Essentially, despite differing source addresses, only one default gateway exists to send traffic to IPs not within the connected subnets and thus, traffic on any “secondary” subnet is rejected as a martian when leaving the host’s interface that is connected to its default gateway.

Some people would use bridged intefaces, although this is sadly not an option for me right now. Whilst the performance of VENET is supposedly better, we also have a large install-base of VENET guests that do not wish to be disturbed. So for now I still need a way to make this work with VENET interfaces (and also VETH if required later).

There are two methods around the return_path filtering, with the first being a terrible hack that should only be used temporarily, if at all… If you echo ’1′ to /proc/sys/net/ipv4/conf/all/log_martians, you will be able to see which interface is filtering martian packets. With that information you can then simply disable the rp_filter function by echoing ’0′ to /proc/sys/net/ipv4/conf/INTERFACE/rp_filter and martians won’t be filtered.

However, this isn’t a sensible option. A better solution is to actually create a routing rule to alter the default gateway used, based on the source subnet. It took me a little bit of digging, but I eventually managed to get this working after combing a few sources (including, but not limited to, the iproute2 man file).

For reference, here’s my routing table showing two networks and two /32 IPs assigned to a guest’s VENET interface (note that the networks are /23′s, not /24′s!):

Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
10.0.9.159      0.0.0.0         255.255.255.255 UH    0      0        0 venet0
10.0.125.53     0.0.0.0         255.255.255.255 UH    0      0        0 venet0
10.0.8.0        0.0.0.0         255.255.254.0   U     0      0        0 br0
10.0.124.0      0.0.0.0         255.255.254.0   U     0      0        0 br1
0.0.0.0         10.0.9.1        0.0.0.0         UG    0      0        0 br0

Start by opening /etc/iproute2/rt_tables in your favourite editor. You’ll need to append a line to the bottom to create a new routing table:

# cat /etc/iproute2/rt_tables
#
# reserved values
#
255    local
254    main
253    default
0    unspec
#
# local
#
#1    inr.ruhep
100    vlan4

As you can see, I’ve appended a new table named ‘vlan4′ (picking a sensible name helps, in my case this is the VLAN name for 10.0.124.0/23)  and given it a priority of 100. As per my understanding, the priority should be decremented for each subsequent table defined.

Now you need to use ip to define the new rules & routing behaviour, taking advantage of the new table we’ve defined. First, create a rule matching traffic from your secondary subnet:

ip rule add from 10.0.124.0/23 iif venet0 table vlan4

For reference, the ‘iif’ attribute is not a mistake; “iif” not “if”. This was also a key part of the setup, as it only classifies traffic originating from the VENET interfaces, no-where else.

Now add a route to define the new default gateway for our new table of classified traffic and apply it:

ip route add default via 10.0.125.1 dev br1 table vlan4
ip route flush cache

You should now find that guest traffic from either network is routed correctly without having to change any rp_filter settings. At any time you can use the following two commands to see your configuration:

ip rule show

ip route show table vlan4

Be sure to re-apply the ‘ip rule’ and ‘ip route’ statements on your next reboot; under Scientific Linux 6.0 I’ve used the /etc/rc.local file, but you can just as easily apply them on ifup in Debian’s network configuration.

I’m currently testing & tweaking an iSCSI setup that utilises a Dell 6224 switch. These are very fast switches for the money (about ~£900 if you’ve got a good account manager!) and provide a lot of features, including stacking, if you have more than one. Their drawbacks, however, are mostly in the lack of documentation and/or the same level of user interface ‘polish’ that you receive from other manufacturers. Most people will say; ‘you get what you pay for’, but for the most part they are great switches for the money you pay.

One such “lack of documentation” has had me annoyed today. I’ve been using LACP to bond ports and, at the same time, have raised the MTU to the maximum of 9216 (which can be done per-port, without a reboot or a switchport up/down event, I might add) across all ports. All this, in an attempt to glean a little more performance (i.e. lower processing overhead) from my iSCSI sessions.

And it seemed to work just fine. However, upon inspecting the interface counters, I noticed a stunning amount of packets being regarded as ‘Oversize Packets’:


switch#show interfaces counters port-channel 1
Alignment Errors: ............................. 0
FCS Errors: ................................... 0
Single Collision Frames: ...................... 0
Multiple Collision Frames: .................... 0
Late Collisions: .............................. 0
Excessive Collisions: ......................... 0
Oversize Packets: ............................. 15829678
Internal MAC Rx Errors: ....................... 0
Received Pause Frames: ........................ 0
Transmitted Pause Frames: ..................... 0

I wasn’t sure whether or not to take this as an error or just a simple ‘count’ of packets. “Oversize” would indicate that they’re bigger than the port was expecting, but I was still hitting around 120MB/sec (out of the theoretical 125MB/sec that Gigabit Ethernet can physically provide) which wouldn’t be conducive to a serious string of frame/packet errors.

I couldn’t find anything online, so I contacted Dell ProSupport to raise a ticket. I had to go through the annoying rigmarole of explaining the problem three times over, but eventually a ‘switch expert’ explained that he wasn’t certain on the use of that counter and that its purpose depended on the firmware version currently in use (in my case, this was 3.2.0.9) and needed to check with his colleagues.

He eventually rang back to inform me that this was not a problem with the switch. The “Oversize Packets” counter merely serves to log packets that have a payload in excess of 1518 bytes. A fixed amount. It doesn’t matter than the MTU was set to 9216, it just continues counting the packets. Utterly useless, then!

As some form of consolation, he also mentioned that it didn’t update in real time.. Owing me to believe that there was some form of port stats analysing process running over the real time output. When it’s this useless, could I please have an option to turn it off? Or better yet, don’t bother logging it by default!

My ADSL provider Andrews & Arnold have provided me with a /48 IPv6 subnet, which seems somewhat wasteful at 2^80 addresses (throw that in your calculator) but certainly useful for testing nevertheless. Whilst slowly getting my head around the task that is variable-length subnetting of IPv6 ranges – painful at best – I decided to just throw in a /64 subnet and set a static gateway address on m0n0wall‘s LAN interface to see if it would ‘just work’.

The result, is a working IPv6 LAN by simply enabling autoconfig from the m0n0wall box and telling Ubuntu’s Network Manager to use it. Et voila:

teh@desktop:~$ ifconfig eth0
eth0 Link encap:Ethernet HWaddr 00:01:29:fc:37:1d
inet addr:81.187.xxx.xxx Bcast:81.187.xxx.xxx Mask:255.255.255.240
inet6 addr: 2001:8b0:ff87:1:201:29ff:fefc:371d/64 Scope:Global
inet6 addr: fe80::201:29ff:fefc:371d/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:1616524 errors:0 dropped:0 overruns:0 frame:0
TX packets:2224946 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:277202062 (277.2 MB) TX bytes:519498762 (519.4 MB)
Interrupt:18

You’ll notice that the last 80 bits of my IPv6 address on this host were assigned via autoconfig, using part of my MAC address (the part that doesn’t correspond to a certain manufacturer, IIRC) as well as some randomly-generated bits, too.

And to make my night, ping6 worked straight away, too:

teh@desktop:~$ ping6 2001:08B0:FF88:0001::1
PING 2001:08B0:FF88:0001::1(2001:8b0:ff88:1::1) 56 data bytes
64 bytes from 2001:8b0:ff88:1::1: icmp_seq=1 ttl=64 time=3.81 ms
64 bytes from 2001:8b0:ff88:1::1: icmp_seq=2 ttl=64 time=0.130 ms
64 bytes from 2001:8b0:ff88:1::1: icmp_seq=3 ttl=64 time=0.132 ms

--- 2001:08B0:FF88:0001::1 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2000ms
rtt min/avg/max/mdev = 0.130/1.358/3.813/1.735 ms

Now to plan how I’m going to roll this out at work…

Even so, due to the impressive compatibility of the Sony Ericsson K800i and Linux, and the subsequent lack of the same DHCP/CDC USB Ethernet adapter functionality in the K850i, I thought it’d be quite cool to see if the K800i could be configured as a back-up WAN interface within m0n0wall.

So I fish my K800i (now retired, although I wish it wasn’t) out of its resting place, find a USB cable, and plug it into the back of the m0n0wall machine. No new interface appeared on the ‘assign interfaces’ page, so I restarted it. Still no new interfaces. Upon checking the kernel messages in the log, I found these lines pertaining to the CDC USB Ethernet device:

Jan 4 20:28:06 kernel: device_attach: cdce0 attach returned 6
Jan 4 20:28:06 kernel: cdce0: could not find data bulk in
Jan 4 20:28:06 kernel: cdce0: Sony Ericsson Sony Ericsson K800, rev 2.00/0.00, addr 2

Which, as a Linux geek, confused me somewhat. Google turned up a number of results for ‘cdce0′ problems or ‘attach returned 6′ regarding various other drivers, but only one really addressed the issue in particular, albeit for a much older SE phone. You’ll notice that there haven’t been any replies, either.

A former colleague pointed me in the direction of a patch that was submitted around October 2008 which enables the proper handling of the CDC USB device within the Nokia N80. Hopefully it should help, but it may be some time before the patch filters down to m0n0wall.

This is just one of those times when I wish I’d followed the world of software development a little more.

So when the K850i came out, I was quite eager to get my upgrade. And so far there’s been only one real drawback to it, that I’ve found: using it as a modem.

When I first moved into my current abode, I didn’t have any ADSL for a few weeks. Predictably one can steal some wireless broadband, or one can attempt to use some form of mobile broadband. Before signing my life away for a few months, I decided to test my phone (which at the time, was the K800i) with Ubuntu. To my sheer delight, the phone presents itself as a USB Ethernet adapter, and Ubuntu’s network-manager simply sent a DHCP request and received an ACK. No messing about here: I had 3G broadband within 5 seconds of plugging the USB cable in!

So obviously when I attempted the same trick with my K850i, I was really quite dismayed to find that you can’t do this any longer. The USB Ethernet device is there (grep -i CDC /var/log/messages) but for the life of me, I cannot find a way to obtain a DHCP lease via the usb0 interface.

Yes, it works perfectly (and with HSDPA speeds, thanks to my city-centre location) if you use wvdial or one of its GUI front-ends (gnome-ppp worked well) and I’ve been able to connect like this..

But I can’t understand why the Sony Ericsson engineers would want to remove such a simple mechanism in favour of the greatest faff-about in history. I’d be interested to hear from anyone that’s managed to get this working.. Although I fear by the time I get an answer, I’ll be back on some ADSL goodness: HSPDA is alright in a pinch, but T-Mobile UK’s data network seems so heavily sensitive to peak times (I suspect insane levels of contention) and the latency is atrocious. Half a second? Ugh. That’ll be the Deep Packet Inspection they do…