Debian/Ubuntu PMTUD & uRPF

I originally started my PMTUD posts using Ubuntu 14.04. Halfway through the post I simply could not get Ubuntu to change it’s MTU on receipt of ICMP fragmentation needed messages. I then tried Debian and it worked. Windows also had no issues changing it’s MTU.

Wanting to finish off the post I switched to Debian and then would investigate the fault later.

Let’s remind ourselves of the original topology:
pmtu-1
Swap out Debian for Ubuntu in the above image.

When I initially started to test, I dropped the MTU between R1 and R2 to 1400. The link between R2 and R4 was kept at 1500. If the user requested a file from the server at this point, Ubuntu would attempt to send at 1500 and get it’s packet dropped at R1. R1 would send a Fragmentation Needed packet back to the Ubuntu server, which would adjust it’s MTU and then send at 1400.

When I changed the MTU between R1 and R2 back up to 1500 and dropped R2-R4 down to 1400, it no longer worked. Debian and Windows did work. I ran tcpdump on Ubuntu and confirmed that it was definitely getting Fragmentation Needed packets. Ubuntu was only acting on Fragmentation Needed packets if it came from it’s default gateway, R1. Any router further along in the path was getting it’s ICMP packets ignored.

In order to understand what the problem is I need to show more about the topology. While the above diagram shows how thins are connected for the most part, it is missing a couple of things. All the devices are running inside virtualbox linked to GNS3. eth1 of all the servers are connected to the above topology, while eth0 was connected via NAT to my host PC so I could install software:
PMTUD-uRPF
Each device had a static route to 192.168.0.0/16 to go out eth1 while their default route was out eth0. Some of you may be sensing what the issue is already…

The point to point links between the virtual routers are using the 10.0.0.0/8 space.

If Ubuntu received an ICMP packet from 192.168.4.1, it’s local default gateway on R1, there were no issues. If it received a packet from R2 or R4’s local interfaces, the packet was dropped. Debian and Windows both didn’t have problems, even though they are configured the same way.

sysctl.conf

I’ve touched on sysctl.conf before the the PMTUD posts, but there is an important difference in the defaults of Ubuntu and Debian. Take a look at this.
Debian:
Screen Shot 2014-09-02 at 9.11.50 am
Ubuntu:
Screen Shot 2014-09-02 at 9.12.01 am

uRPF

Ubuntu has Unicast Reverse Path Forwarding on by default. Debian has it off by default. In sysctl.conf on both machines, the required configuration setting is commented out:
Screen Shot 2014-09-02 at 9.15.44 am
R2 was originating it’s ICMP packets from it’s local interface, 10.0.12.2 in my example. Ubuntu did receive that packet, but it failed the RPF check and so was ignored. To confirm I tested this in two different ways:

  • Add a static route to 10.0.0.0/8 out eth1
  • Disable uRPF check on Ubuntu

Each test individually allowed the original PMTUD to work. What’s odd is that the sysctl.conf file in Ubuntu says that you need to uncomment the lines to turn on uRPF, but it’s on by default. Uncommenting the lines and setting the value to 1 is the same as leaving them commented. In Debian the default is to disable uRPF. In that distro you would need to uncomment the uRPF lines and set the value to 1 to turn the feature on.

Conclusions

  • If a server is multi-homed, PMTUD could break if the ICMP message arrives on an interface that the server is not expecting.
  • If you do have a server multi-homed, it would probably be best to turn off uRPF

© 2009-2019 Darren O'Connor All Rights Reserved -- Copyright notice by Blog Copyright