mVPN, or Multicast VPN, is a pretty big subject. I’d like to go over a lot of details and so this will become a series of posts. How many I don’t know yet, which is why I’m using X in the title.
I’ll try and start with the more basic types of mVPN and then move onto the more complicated stuff. I’m most certainly not going to go over the basics of multicast or l3vpn themselves otherwise this series would stretch to 20 posts long.
Let’s take the following topology into consideration:
R2, R3, R4, and R5 are the ISP routers providing a L3VPN service to Mr Customer. R2, R4, and R5 are the PE routers while R3 is a P router. The core network is currently running OSPF and LDP. The PE routers are exchanging VPNv4 routes via MP-BGP.
Now Mr. Customer would like to start running multicast within their network. These multicast packets will not be able to run natively over the ISP core, as the source address could be a private address in the customer’s VPN. Therefore the P routers would never know the source. Also if we did try to run multicast natively, we would have a problem in that no two customers would be able to run the same group.
One solution is for the customer to run GRE tunnels between his WAN routers. i.e. he would manually set up a full mesh of tunnels from R1 to R6, R1 to R7, and R1 to R8. He’ll also need to ensure all his other WAN edge routers have tunnels to all others. This works, but it’s a lot of work and upkeep for the customer to do.
Why can’t this ISP sell this as an added service?
The first ISP method of doing this is called draft rosen. This draft has now expanded into RFC 6037, but is still called draft rosen by many.
Draft rosen, in a nutshell, makes the PE routers do the hard work. Instead of manually configuring GRE tunnels from CE to CE, the PE routers will automatically set up GRE tunnels. In order to set up a GRE tunnel, you need a source and destination address. In draft rosen the source address will be the PE’s loopback address, while the destination address will be a multicast address dedicated to the customer. Hang on, multicast destination? Yes that’s correct. If we look at customer 1 for example, we can configure the multicast group address of 220.127.116.11 and dedicate it to them. Each PE router will attempt to form GRE tunnels with other PE routers. Each PE router also joins the group 18.104.22.168 in the ISP core. This way they all send GRE tunneled traffic from themselves which will end up at all other PE routers thanks to destination address being a multicast address. These tunnels create whats called the MDT or multicast distribution tree.
This does mean a few things though. For one we need to enable multicast in the ISP core network. In order for the PE routers to create the MDT to make their GRE tunnels. Customer multicast traffic will then be encapsulated in those GRE packets and sent off to the other PE routers in the same mVPN.
Let’s take a look at configuring this.
ip multicast-routing ! interface Loopback0 ip address 22.214.171.124 255.255.255.255 ip pim sparse-mode ip ospf 1 area 0 ! interface GigabitEthernet1/0 ip address 10.0.23.3 255.255.255.0 ip pim sparse-mode ip ospf 1 area 0 mpls ip ! interface GigabitEthernet2/0 ip address 10.0.34.3 255.255.255.0 ip pim sparse-mode ip ospf 1 area 0 mpls ip ! interface GigabitEthernet3/0 ip address 10.0.35.3 255.255.255.0 ip pim sparse-mode ip ospf 1 area 0 mpls ip ! ip pim rp-address 126.96.36.199
To keep things simple, I’m using a static RP address for the core. In the real-world this could be static, auto-rp, or BSR. Either of the three could also use anycast and MSDP if you so wished.
The MDT group will be configured on the PE routers. Each VRF will need to have multicast enabled, and then an MDT address defined for that VRF. PIM will be enabled on the CE-facing interface, and it also needs to be enabled on the loopback interface which it’s using for it’s VPNv4 peering. Finally it will need PIM enabled on the core facing interface itself. This is R2’s relevant config:
vrf definition A rd 100:1 route-target export 100:1 route-target import 100:1 ! address-family ipv4 mdt default 188.8.131.52 exit-address-family ! ip multicast-routing ip multicast-routing vrf A ! interface Loopback0 ip address 184.108.40.206 255.255.255.255 ip pim sparse-mode ip ospf 1 area 0 ! interface GigabitEthernet1/0 ip address 10.0.23.2 255.255.255.0 ip pim sparse-mode ip ospf 1 area 0 mpls ip ! interface FastEthernet2/0 vrf forwarding A ip address 10.0.12.2 255.255.255.0 ip pim sparse-mode ip ospf 2 area 0 ! ip pim rp-address 220.127.116.11
R4 and R5 have a similar config.
At this point, multicast has not yet been enabled in the customer network. What we should see in the core is that all three PE routers are sending traffic to 18.104.22.168 – This is the MDT set up. All three will join the *,22.214.171.124 group. Once the PE routers all get traffic from other PEs over this group, they will all join the S,G group directly. We can verify this on R3:
R3#show ip mroute | beg \( (*, 126.96.36.199), 00:12:46/00:02:45, RP 188.8.131.52, flags: S Incoming interface: Null, RPF nbr 0.0.0.0 Outgoing interface list: GigabitEthernet1/0, Forward/Sparse, 00:09:12/00:02:45 GigabitEthernet2/0, Forward/Sparse, 00:12:46/00:02:30 GigabitEthernet3/0, Forward/Sparse, 00:12:46/00:02:34 (184.108.40.206, 220.127.116.11), 00:12:46/00:03:16, flags: T Incoming interface: GigabitEthernet3/0, RPF nbr 10.0.35.5 Outgoing interface list: GigabitEthernet1/0, Forward/Sparse, 00:09:12/00:02:45 GigabitEthernet2/0, Forward/Sparse, 00:12:46/00:02:37 (18.104.22.168, 22.214.171.124), 00:12:46/00:03:14, flags: T Incoming interface: GigabitEthernet2/0, RPF nbr 10.0.34.4 Outgoing interface list: GigabitEthernet1/0, Forward/Sparse, 00:09:12/00:02:45 GigabitEthernet3/0, Forward/Sparse, 00:12:46/00:02:34 (126.96.36.199, 188.8.131.52), 00:12:46/00:03:01, flags: T Incoming interface: GigabitEthernet1/0, RPF nbr 10.0.23.2 Outgoing interface list: GigabitEthernet2/0, Forward/Sparse, 00:12:46/00:02:34 GigabitEthernet3/0, Forward/Sparse, 00:12:46/00:02:34
The OIL for *,184.108.40.206 is out to all three PE routers. All three are also sources, and so you see three S,G groups, each with a source of their loopbacks. This part is all automatic, so if I add another PE router, I just need to add it to the MDT group, enable PIM, and all the other PE routers will set up new GRE tunnels automatically.
Once the GRE tunnels are setup, the PE routers will form a PIM adjacency automatically over the multipoint tunnel inside the customer’s VRF. Let’s take a look at R2’s PIM neighbours:
R2#sh ip pim vrf A neighbor PIM Neighbor Table Mode: B - Bidir Capable, DR - Designated Router, N - Default DR Priority, P - Proxy Capable, S - State Refresh Capable, G - GenID Capable Neighbor Interface Uptime/Expires Ver DR Address Prio/Mode 10.0.12.1 FastEthernet2/0 00:03:17/00:01:23 v2 1 / S P G 220.127.116.11 Tunnel2 00:01:20/00:01:23 v2 1 / S P G 18.104.22.168 Tunnel2 00:01:49/00:00:53 v2 1 / DR S P G
fa2/0 is the CE-facing interface. Tunnel2 is the multipoint GRE interface going to R4 and R5. Over the tunnel interface R2 has two adjacencies. R5 is elected as the DR (thanks to its highest IP address)
As far as the customer is concerned, they can just run a standard multicast set up. It doesn’t what what version they are running either. I’ll make R9 announce itself as the BSR and RP, and that should filter to all other customer sites.
R9#sh run | inc candidate ip pim bsr-candidate Loopback0 0 ip pim rp-candidate Loopback0 interval 10
Let’s go to R12 in another site to see if we get the RP mapping information:
R12#sh ip pim rp map PIM Group-to-RP Mappings Group(s) 22.214.171.124/4 RP 126.96.36.199 (?), v2 Info source: 188.8.131.52 (?), via bootstrap, priority 0, holdtime 150 Uptime: 00:08:15, expires: 00:02:17
At this point I should be able to join a group on R12 and source traffic from R9. Let’s test this:
R12 interface FastEthernet1/0 ip igmp join-group 184.108.40.206
R9#ping 220.127.116.11 repeat 2 Type escape sequence to abort. Sending 2, 100-byte ICMP Echos to 18.104.22.168, timeout is 2 seconds: Reply to request 0 from 10.0.128.12, 88 ms Reply to request 1 from 10.0.128.12, 132 ms
The CE network shows regular muticast working. Let’s check the PE routers mroute tables:
R5#sh ip mroute vrf A 22.214.171.124 | beg \( (*, 126.96.36.199), 00:11:51/00:03:24, RP 188.8.131.52, flags: S Incoming interface: Tunnel1, RPF nbr 184.108.40.206 Outgoing interface list: FastEthernet2/0, Forward/Sparse, 00:11:51/00:03:24 (220.127.116.11, 18.104.22.168), 00:02:00/00:01:29, flags: T Incoming interface: Tunnel1, RPF nbr 22.214.171.124 Outgoing interface list: FastEthernet2/0, Forward/Sparse, 00:02:00/00:03:28
R5 shows that the incoming RPF interface is Tunnel1, which is the MDT GRE tunnel.
R2#sh ip mroute vrf A 126.96.36.199 | beg \( (*, 188.8.131.52), 00:11:16/00:03:03, RP 184.108.40.206, flags: S Incoming interface: FastEthernet2/0, RPF nbr 10.0.12.1 Outgoing interface list: Tunnel2, Forward/Sparse, 00:11:16/00:03:03 (220.127.116.11, 18.104.22.168), 00:02:23/00:01:06, flags: T Incoming interface: FastEthernet2/0, RPF nbr 10.0.12.1 Outgoing interface list: Tunnel2, Forward/Sparse, 00:02:23/00:03:03
R2 shows Tunnel2 is in the OIL which again is the MDT GRE tunnel.
So our initial draft-rosen configuration is working as expected. Join me for part two where I’ll go a lot deeper into how those multicast packets are going from A to B