Tag Archives: bgp

Full BGP in dynamips

After having trouble with IOS-XRv yesterday, I wondered if an emulated 7200 could take a full table.

In particular I’m running:

Cisco IOS Software, 7200 Software (C7200-ADVIPSERVICESK9-M), Version 12.2(33)SRE7

The ram itself isn’t even high:

Cisco 7206VXR (NPE400) processor (revision A) with 491520K/32768K bytes of memory.

The emulated 7200 has no issue taking the full table:

C7200# sh ip bgp sum
BGP router identifier 1.1.31.253, local AS number 65500
BGP table version is 482914, main routing table version 482914
479027 network entries using 61315456 bytes of memory
479027 path entries using 24909404 bytes of memory
76560/76531 BGP path/bestpath attribute entries using 9493440 bytes of memory
68802 BGP AS-PATH entries using 2856590 bytes of memory
0 BGP route-map cache entries using 0 bytes of memory
0 BGP filter-list cache entries using 0 bytes of memory
BGP using 98574890 total bytes of memory
BGP activity 479427/400 prefixes, 479482/455 paths, scan interval 60 secs

Neighbor        V           AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
1.1.31.252      4        65550   82140      11   482914    0    0 00:07:06   479027

Memory utilisation on the virtual machine I’m running this on is a paltry 160MB – Which includes the OS itself.

Performance isn’t too bad either. I wouldn’t push transit traffic through it, but it would make a good looking glass to be able to see BGP routes and trace through it

FULL BGP Table on IOS-XRv – Seems broken

I’ve attempted to put a full table into IOS-XRv, but I’m getting all kinds of nasty messages.

First the session came up:

RP/0/0/CPU0:XR1#show bgp ipv4 un sum
Thu Feb 20 17:27:07.990 UTC
BGP router identifier 1.1.224.62, local AS number 65500
BGP generic scan interval 60 secs
BGP table state: Active
Table ID: 0xe0000000   RD version: 2
BGP main routing table version 2
BGP scan interval 60 secs

BGP is operating in STANDALONE mode.


Process       RcvTblVer   bRIB/RIB   LabelVer  ImportVer  SendTblVer  StandbyVer
Speaker               2          1          2          0           1           2

Neighbor        Spk    AS MsgRcvd MsgSent   TblVer  InQ OutQ  Up/Down  St/PfxRcd
1.1.31.252     0 39326   78356       3        0    0    0 00:01:52     479556

Soon after I was getting all manner of errors:

RP/0/0/CPU0:XR1#RP/0/0/CPU0:Feb 20 17:27:29.308 : shmwin_svr[68]: %OS-SHMWIN_SVR-3-NOVMEM : Exhausted virtual memory, top 1 consumer, ifc-ipv4: 50305 KBytes
RP/0/0/CPU0:Feb 20 17:27:29.308 : shmwin_svr[68]: %OS-SHMWIN_SVR-3-NOVMEM : Exhausted virtual memory, top 2 consumer, statsd_db_g: 5103 KBytes
RP/0/0/CPU0:Feb 20 17:27:29.308 : shmwin_svr[68]: %OS-SHMWIN_SVR-3-NOVMEM : Exhausted virtual memory, top 3 consumer, aib: 2159 KBytes
RP/0/0/CPU0:Feb 20 17:27:29.308 : shmwin_svr[68]: %OS-SHMWIN_SVR-3-NOVMEM : Exhausted virtual memory, top 4 consumer, statsd_db_l: 1131 KBytes
RP/0/0/CPU0:Feb 20 17:27:29.308 : shmwin_svr[68]: %OS-SHMWIN_SVR-3-NOVMEM : Exhausted virtual memory, top 5 consumer, im_rd: 1131 KBytes
RP/0/0/CPU0:Feb 20 17:27:29.328 : fib_mgr[207]: %OS-SHMWIN-3-ALLOC_ARENA_FAILED : SHMWIN: Failed to allocate new arena from the server : 'SHMWIN_SVR' detected the 'fatal' condition 'VM is exhausted or totally fragmented'
RP/0/0/CPU0:Feb 20 17:27:29.328 : fib_mgr[207]: %ROUTING-FIB-3-PD_FAIL : FIB platform error: fib_leaf_insert 5014 Cannot insert in switching leaf 97.73.48.0/21 [0xafff21a8] type 5 flags 10000001 refcnt 1 prot ipv4 table default tableid e0000000 vrfid 60000000 ---LDI: [0xacbf25a4] type 6 flags 90141 refcnt 1 type 3 depth 1 num_slots 1 num_buckets 1 LDI pl 0xacbd306c ---PL [0xacbd306c] type 7 flags 8020 refcnt 160238 path_cnt 1 max_depth 2 ldi 0xacbf25a4  ---: 0x16 Invalid argument  : pkg/bin/fib_mgr : (PID=417911) :  -Traceback= (TRUNCATED)
RP/0/0/CPU0:Feb 20 17:27:29.328 : fib_mgr[207]: %ROUTING-FIB-3-INTERNAL_ERR : FIB internal error: (!retry)  : pkg/bin/fib_mgr : (PID=417911) :  -Traceback= 98bf26a 42497c4 4241b80 424be26 422e1b2 42f41ee 42fa798 4214150 4211fe4 421427e 4218e75 421923a 8226a4b 8224bdc 4200bd5 421081c
RP/0/0/CPU0:Feb 20 17:27:29.418 : bgp[1048]: %OS-SHMWIN-3-ALLOC_ARENA_FAILED : SHMWIN: Failed to allocate new arena from the server : 'SHMWIN_SVR' detected the 'fatal' condition 'VM is exhausted or totally fragmented'
RP/0/0/CPU0:Feb 20 17:27:29.518 : fib_mgr[207]: %ROUTING-FIB-3-GTRIE : [3] : FIB error in generic trie operation: leaf alloc: Not enough memory

RP/0/0/CPU0:XR1#
RP/0/0/CPU0:XR1#show bgp ipv4 un nexthops RP/0/0/CPU0:Feb 20 17:27:39.337 : fib_mgr[207]: %OS-SHMWIN-3-ALLOC_ARENA_FAILED : SHMWIN: Failed to allocate new arena from the server : 'SHMWIN_SVR' detected the 'fatal' condition 'VM is exhausted or totally fragmented'
RP/0/0/CPU0:Feb 20 17:27:39.527 : fib_mgr[207]: %ROUTING-FIB-3-GTRIE : [3] : FIB error in generic trie operation: leaf alloc: Not enough memory
RP/0/0/CPU0:Feb 20 17:27:40.717 : bgp[1048]: %OS-SHMWIN-3-ALLOC_ARENA_FAILED : SHMWIN: Failed to allocate new arena from the server : 'SHMWIN_SVR' detected the 'fatal' condition 'VM is exhausted or totally fragmented'

The actual resources on the VM itself isn’t too bad:
XRv Full table 300x204 FULL BGP Table on IOS XRv   Seems broken

After that, the BGP session dies, restarts, and the above happens over and over again.

The VM itself has 3Gb of RAM which is more than enough for the full table in both RIB and FIB.

Anyone else managed to get this working?

OSPF as the PE-CE routing protocols deep dive – Part 2 of 3 – The SHAM Link

Read part 1
Read part 2
Read part 3

 
In order to understand the purpose of the sham link, you first need to understand the problem it is trying to fix. If we look at the topology used last time again for a refresh:
RFC4577 12 OSPF as the PE CE routing protocols deep dive   Part 2 of 3   The SHAM Link

The Problem

From the previous post it was clear that it did not matter if the LSA received by a PE from a CE was type1, type2, or type3. That LSA would always be either type3 or type5 on the remote side. While this is perfectly fine most of the time, there are times when this is less than ideal. I’ll add a low-speed serial link between R5 and R6 and enable regular OSPF over the link like so:
RFC4577 4 OSPF as the PE CE routing protocols deep dive   Part 2 of 3   The SHAM Link

R5 config:

interface Serial2/0
 ip address 10.0.56.5 255.255.255.0
 ip ospf 1 area 0

If I check the route to R6′s loopback, it’ll be going over the slow serial link:

R5#sh ip route 6.6.6.6
Routing entry for 6.6.6.6/32
  Known via "ospf 1", distance 110, metric 65, type intra area
  Last update from 10.0.56.6 on Serial2/0, 00:01:26 ago
  Routing Descriptor Blocks:
  * 10.0.56.6, from 6.6.6.6, 00:01:26 ago, via Serial2/0
      Route metric is 65, traffic share count is 1

Changing the metric of the link will have no effect whatsoever:

R5#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
R5(config)#int s2/0
R5(config-if)#ip ospf cost 50000
R5(config-if)#end
R5#
*Jan  6 12:01:52.747: %SYS-5-CONFIG_I: Configured from console by console
R5#sh ip route 6.6.6.6
Routing entry for 6.6.6.6/32
  Known via "ospf 1", distance 110, metric 50001, type intra area
  Last update from 10.0.56.6 on Serial2/0, 00:00:01 ago
  Routing Descriptor Blocks:
  * 10.0.56.6, from 6.6.6.6, 00:00:01 ago, via Serial2/0
      Route metric is 50001, traffic share count is 1

OSPF has it’s own internal route-selection decision. Intra-area routes from type1 LSAs are always preferred over summaries from type3 LSAs. Summaries are also preferred over E1s, then E2, then N1, then finally N2 OSPF routes.

R5 and R6 are in the same area, hence they are currently learning each others prefixes through the type1 LSAs between then. Regardless of metric, this route will always be preferred over the type3 learned over the MPLS cloud.

The Sham Link

RFC 4577 Section 4.2.7 gives us one option to fix this problem. The sham-link essentially allows the PE routers to share OSPF routes via type1 LSAs. When this LSA reaches the PE on the other side, it is still a type1 LSA. That LSA is flooded to the connected PE. This means all internal OSPF routes at one site can appear internal on the other side. The sham-link cost can be adjusted to be lower than the backdoor OSPF link and therefore traffic will prefer going over the MPLS core first.

Unlike the previous post in which IOS and IOS-XR had minor differences in interpreting the RFC, for this second part they are very different indeed.

IOS Sham-Link

Sham-links can be placed into any area you wish. As the CE’s are all in area 0 we’ll just stick to area 0. Both PEs will create a sham-link to each other. Both PEs need to be able to send packets to the other PE over the MPLS cloud. These end-points need to be in the customer’s VRF. Generally the easiest way to do this is to create a new loopback on both PEs in the VRF, and then advertise those addresses via BGP in a VPNv4 address.
RFC4577 5 OSPF as the PE CE routing protocols deep dive   Part 2 of 3   The SHAM Link
R2:

interface Loopback20
 vrf forwarding A
 ip address 20.20.20.20 255.255.255.255
!
router bgp 100
 !
 address-family ipv4 vrf A
  network 20.20.20.20 mask 255.255.255.255

From R2 we should be able to reach the new loopback on R4 through the VRF:

R2#traceroute vrf A 40.40.40.40 so lo20

Type escape sequence to abort.
Tracing the route to 40.40.40.40

  1 10.0.23.3 [MPLS: Labels 16/21 Exp 0] 40 msec 48 msec 44 msec
  2 40.40.40.40 72 msec 64 msec 40 msec

Now that they have connectivity via a label-switched-path we can create the sham-link:

R2#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
R2(config)#router ospf 1 vrf A
R2(config-router)#area 0 sham-link 20.20.20.20 40.40.40.40
R2(config-router)#end

Once both sides are configured we can see the sham-link up:

R2#sh ip ospf 1 sham-links
Sham Link OSPF_SL0 to address 40.40.40.40 is up
Area 0 source address 20.20.20.20
  Run as demand circuit
  DoNotAge LSA allowed. Cost of using 1 State POINT_TO_POINT,
  Timer intervals configured, Hello 10, Dead 40, Wait 40,
    Hello due in 00:00:04
    Adjacency State FULL (Hello suppressed)
    Index 3/3, retransmission queue length 0, number of retransmission 0
    First 0x0(0)/0x0(0) Next 0x0(0)/0x0(0)
    Last retransmission scan length is 0, maximum is 0
    Last retransmission scan time is 0 msec, maximum is 0 msec

Before we continue with the verification of the sham-link, I want you to take a step back and think about how each router in the path learns and forwards traffic from R5 to R6. This is essential when dealing with the differences between IOS and IOS-XR.

No Sham-link

  • R6 originates it’s loopback in a type-1 LSA to R4
  • R4 installs a route to R6 via the type1 LSA in the VRF
  • R4 redistributes that route into BGP, converts it to VPNv4 and advertises it over to R2
  • R2 redistributes the VPNv4 route into OSPF and originates a type3 LSA to R5

Sham-link

  • R6 originates it’s loopback in a type-1 LSA to R4
  • R4 installs a route to R6 via the type1 LSA in the VRF
  • R4 advertises the LSA over the sham-link to R2
  • R2 installs a route based on the LSA and forwards that LSA to R5

What’s interesting about the sham-link here is that there is no redistribution between BGP and OSPF. So do we need to redistribute at all? It’s an interesting question as we shall soon see. Let’s remove all redistribution on R2 and R4 to see what happens.

R2#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
R2(config)#router ospf 1 vrf A
R2(config-router)#no redi bgp 100
R2(config-router)#router bgp 100
R2(config-router)#add ipv4 vrf A
R2(config-router-af)#no red ospf 1
R2(config-router-af)#end

This has been completed on both PEs. Do we see the route to R6′s loopback as a intra-area OSPF route over the MPLS cloud on R5?

R5#sh ip route 6.6.6.6
Routing entry for 6.6.6.6/32
  Known via "ospf 1", distance 110, metric 4, type intra area
  Last update from 10.0.25.2 on FastEthernet1/0, 00:00:04 ago
  Routing Descriptor Blocks:
  * 10.0.25.2, from 6.6.6.6, 00:00:04 ago, via FastEthernet1/0
      Route metric is 4, traffic share count is 1

So our control-plane is working, but as we shall see next the data-plane will not work:

R5#ping 6.6.6.6 so lo0 re 3

Type escape sequence to abort.
Sending 3, 100-byte ICMP Echos to 6.6.6.6, timeout is 2 seconds:
Packet sent with a source address of 5.5.5.5
...
Success rate is 0 percent (0/3)

Section 4.2.7.4 of the RFC tells us why this is happening:

Any other route advertised in an LSA that is transmitted over a sham link MUST also be redistributed (by the PE flooding the LSA over the sham link) into BGP. This means that if the preferred (OSPF) route for a given address prefix has the sham link as its next hop interface, then there will also be a “corresponding BGP route”, for that same address prefix, installed in the VRF. Per Section 4.1.2, the OSPF route is preferred. However, when forwarding a packet, if the preferred route for that packet has the sham link as its next hop interface, then the packet MUST be forwarded according to the corresponding BGP route. That is, it will be forwarded as if the corresponding BGP route had been the preferred route. The “corresponding BGP route” is always a VPN-IPv4 route; the procedure for forwarding a packet over a VPN-IPv4 route is described in [VPN].

The part of section 4.1.2 reffered to in the section above states:

If a VRF contains both an OSPF-distributed route and a VPN-IPv4 route for the same IPv4 prefix, then the OSPF-distributed route is preferred. In general, this means that forwarding is done according to the OSPF route. The one exception to this rule has to do with the “sham link”. If the next hop interface for an installed (OSPFdistributed) route is the sham link, forwarding is done according to a corresponding BGP route. This is detailed in Section 4.2.7.4.

So while R2 has an OSPF-learned route through the sham-link, it does NOT have a BGP-learned route to actually do the forwarding on. R2 and R4 will have to redistribute the OSPF routes into BGP. They do NOT however have to move those BGP routes back into OSPF on the other side.

While it may be a little confusing it makes perfect sense. If R2 needs to send a packet to a VPN attached to R4 it needs two labels. The top-most label is the transport label needed to get the packet through the ISP core. The second label is the VPN label needed to let R4 know which VPN that packet belongs to. MP-BGP is able to advertise a VPN label with it’s VPNv4 NLRI update. OSPF does not have the same capibility. Therefore the BGP route is needed on the PEs so they know which labels to impose on ingress through the core.

If we look at the current CEF table on R2 to get to R6, we’ll see it is doesn’t know how to handle it:

R2#show ip  cef vrf A  6.6.6.6/32 detail
6.6.6.6/32, epoch 0
  recursive via 40.40.40.40 unusable: no label, unresolved

The RFC states that the next-hops need to be resolved via BGP, so I’ll ensure both routers are redistributing OSPF routes into BGP. I won’t, however, redistributes BGP routes back into OSPF as it’s not required:

R2#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
R2(config)#router bgp 100
R2(config-router)#add ipv4 vrf A
R2(config-router-af)#red ospf 1
R2(config-router-af)#end

From R2′s perspective, the route to 6.6.6.6/32 will be an OSPF route through the sham-link, but will be forwarded via the BGP link.

 

  • OSPF Route:
R2#sh ip route vrf A 6.6.6.6

Routing Table: A
Routing entry for 6.6.6.6/32
  Known via "ospf 1", distance 110, metric 3, type intra area
  Redistributing via bgp 100
  Last update from 4.4.4.4 00:01:34 ago
  Routing Descriptor Blocks:
  * 4.4.4.4 (default), from 6.6.6.6, 00:01:34 ago
      Route metric is 3, traffic share count is 1
      MPLS label: 23
      MPLS Flags: MPLS Required
  • Forwarding BGP route:
R2#show bgp vpnv4  un rd  4.4.4.4:1 6.6.6.6
BGP routing table entry for 4.4.4.4:1:6.6.6.6/32, version 48
Paths: (1 available, best #1, no table)
  Not advertised to any peer
  Local
    4.4.4.4 (metric 30) from 4.4.4.4 (4.4.4.4)
      Origin incomplete, metric 2, localpref 100, valid, internal, best
      Extended Community: RT:1:1 OSPF DOMAIN ID:0x0005:0x000000030200
        OSPF RT:0.0.0.0:2:0 OSPF ROUTER ID:40.40.40.40:0
      mpls labels in/out nolabel/23
  • CEF entry showing two label imposition:
R2#show ip  cef vrf A  6.6.6.6/32 detail
6.6.6.6/32, epoch 0, flags rib defined all labels
  recursive via 4.4.4.4 label 23
    nexthop 10.0.23.3 GigabitEthernet1/0 label 16
  • Finally we should now be able to get from R5 to R6:
R5#traceroute 6.6.6.6 so lo0

Type escape sequence to abort.
Tracing the route to 6.6.6.6

  1 10.0.25.2 24 msec 8 msec 44 msec
  2 10.0.23.3 [MPLS: Labels 16/23 Exp 0] 64 msec 88 msec 60 msec
  3 10.0.46.4 [MPLS: Label 23 Exp 0] 56 msec 88 msec 12 msec
  4 10.0.46.6 152 msec 76 msec 128 msec

It does raise a question though. If OSPF had the ability to advertise VPN labels in it’s LSAs, it might be possible to do away with BGP in this specific type of topology. It may be that OSPFv3 and IS-IS, both easily extended, would be able to do this. That will have to be another post for another day.

IOS-XR Sham-Link

IOS-XR, at least in version 3.9.1, has an odd behaviour when it comes to an OSPF sham link. Note that I have only tested version 3.9.1 so if this behaviour changes in newer versions I’m not aware of them yet.

Like the first post in this series, I’ll swap out R4 for an IOS-XR box. R2 will continue to run regular IOS.

I’m going to configure the sham-link between R2 and R4. I’ll also redistribute from OSPF into BGP, but not the other way around. This will match the working configuration on IOS. I’ll show the XR config of R4 for this:

RP/0/0/CPU0:R4#sh run router ospf 100
Mon Jan  6 16:35:46.369 UTC
router ospf 100
 vrf A
  domain-id type 0005 value 000000640200
  area 0
   sham-link 40.40.40.40 20.20.20.20
   !
   interface POS0/6/0/0
   !
  !
 !
!

RP/0/0/CPU0:R4#sh run router bgp
Mon Jan  6 16:36:04.374 UTC
router bgp 100
 address-family vpnv4 unicast
 !
 neighbor 2.2.2.2
  remote-as 100
  update-source Loopback0
  address-family vpnv4 unicast
  !
 !
 vrf A
  rd 1:1
  address-family ipv4 unicast
   network 40.40.40.40/32
   redistribute ospf 100
  !
 !
!

So now the sham-link should come up. But it doesn’t… It never comes up. Doing a debug on R2 shows something interesting:

R2#debug ip ospf hello
OSPF hello events debugging is on
OSPF: Send hello to 40.40.40.40 area 0 on OSPF_SL0 from 20.20.20.20

R2 is sending OSPF hellos to R4, but R4 is simply not responding. It can also be a bit cryptic as R2 considers the sham-link ‘up’ – but there is no neighbourship:

R2#sh ip ospf sham-links
Sham Link OSPF_SL0 to address 40.40.40.40 is up
Area 0 source address 20.20.20.20
  Run as demand circuit
  DoNotAge LSA allowed. Cost of using 1 State POINT_TO_POINT,
  Timer intervals configured, Hello 10, Dead 40, Wait 40,
    Hello due in 00:00:02
R2#
R2#
R2#sh ip ospf 100 neigh

Neighbor ID     Pri   State           Dead Time   Address         Interface
7.7.7.7           1   FULL/DR         00:00:38    10.1.2.1        FastEthernet1/0
5.5.5.5           1   FULL/DR         00:00:33    20.2.4.4        FastEthernet0/0.24

On R4 we see the following:

RP/0/0/CPU0:R4# show ospf 100 vrf A sham-links
Mon Jan  6 16:39:41.808 UTC

Sham Links for OSPF 100, VRF A

Sham Link OSPF_SL0 to address 20.20.20.20 is down
Area 0, source address 40.40.40.40
IfIndex = 2
  Run as demand circuit
  DoNotAge LSA allowed., Cost of using 1
  Transmit Delay is 1 sec, State DOWN,
  Timer intervals configured, Hello 10, Dead 40, Wait 40, Retransmit 5

RP/0/0/CPU0:R4# show ospf 100 vrf A neighbor
Mon Jan  6 16:39:51.085 UTC

* Indicates MADJ interface

Neighbors for OSPF 100, VRF A

Neighbor ID     Pri   State           Dead Time   Address         Interface
6.6.6.6         1     FULL/  -        00:00:31    10.19.20.20     POS0/6/0/0
    Neighbor is up for 00:30:14

Total neighbor count: 1

Each PE only has their neighbourships to their directly connected CEs as fully up. They are not adjacent on the sham-link.

The only way to get the sham-link up on IOS-XR, is to redistribute the VPNv4 routes back into OSPF on the XR side. This makes little sense considering what I have covered above in the IOS-only side.

P/0/0/CPU0:R4#conf
Mon Jan  6 16:42:09.978 UTC
RP/0/0/CPU0:R4(config)#router ospf 100 vrf A
RP/0/0/CPU0:R4(config-ospf-vrf)#redistribute bgp 100
RP/0/0/CPU0:R4(config-ospf-vrf)#end
Uncommitted changes found, commit them before exiting(yes/no/cancel)? [cancel]:yes
RP/0/0/CPU0:Jan  6 16:42:29.374 : ospf[482]: %ROUTING-OSPF-5-ADJCHG : 
Process 100, Nbr 20.20.20.20 on OSPF_SL0 in area 0 from LOADING to FULL, Loading Done,vrf A vrfid 0x60000012

As you can see, the sham-link comes up straight away as soon as this is done.

We can confirm from the CE’s perspective that the route is intra-area over the MPLS cloud and is label-switched that way:

R5#show ip route 6.6.6.6
Routing entry for 6.6.6.6/32
  Known via "ospf 1", distance 110, metric 4, type intra area
  Last update from 20.2.4.2 on FastEthernet0/0.24, 00:01:37 ago
  Routing Descriptor Blocks:
  * 20.2.4.2, from 6.6.6.6, 00:01:37 ago, via FastEthernet0/0.24
      Route metric is 4, traffic share count is 1
R5#traceroute 6.6.6.6

Type escape sequence to abort.
Tracing the route to 6.6.6.6

  1 20.2.4.2 0 msec 4 msec 0 msec
  2 20.2.3.3 [MPLS: Labels 21/16028 Exp 0] 0 msec 4 msec 0 msec
  3 20.3.6.6 [MPLS: Labels 22/16028 Exp 0] 0 msec 4 msec 0 msec
  4 20.6.19.19 [MPLS: Label 16028 Exp 0] 4 msec 0 msec 4 msec
  5 10.19.20.20 4 msec *  4 msec

So why does IOS-XR have this behaviour? I’m not entirely sure, but checking the route table on both PE does give us a hint. Let’s go over the RFC statements once again:

Any other route advertised in an LSA that is transmitted over a sham link MUST also be redistributed (by the PE flooding the LSA over the sham link) into BGP. This means that if the preferred (OSPF) route for a given address prefix has the sham link as its next hop interface, then there will also be a “corresponding BGP route”, for that same address prefix, installed in the VRF. Per Section 4.1.2, the OSPF route is preferred. However, when forwarding a packet, if the preferred route for that packet has the sham link as its next hop interface, then the packet MUST be forwarded according to the corresponding BGP route. That is, it will be forwarded as if the corresponding BGP route had been the preferred route. The “corresponding BGP route” is always a VPN-IPv4 route; the procedure for forwarding a packet over a VPN-IPv4 route is described in [VPN].

If a VRF contains both an OSPF-distributed route and a VPN-IPv4 route for the same IPv4 prefix, then the OSPF-distributed route is preferred. In general, this means that forwarding is done according to the OSPF route. The one exception to this rule has to do with the “sham link”. If the next hop interface for an installed (OSPFdistributed) route is the sham link, forwarding is done according to a corresponding BGP route. This is detailed in Section 4.2.7.4.

The RFC states that each PE should be learning a BGP and OSPF route. The OSPF route should be installed into the RIB, while the BGP route is used for the actual forwarding thanks to it’s label carrying capability. What we see on IOS-XR is different.

  • IOS:
R2#sho ip route vrf A 6.6.6.6

Routing Table: A
Routing entry for 6.6.6.6/32
  Known via "ospf 100", distance 110, metric 3, type intra area
  Redistributing via bgp 100
  Advertised by bgp 100 match internal external 1 & 2
  Last update from 4.4.4.4 00:04:28 ago
  Routing Descriptor Blocks:
  * 4.4.4.4 (default), from 6.6.6.6, 00:04:28 ago
      Route metric is 3, traffic share count is 1
      MPLS label: 16028
      MPLS Flags: MPLS Required

The active route on R2 is the OSPF sham-link route as expected.

  • IOS-XR:
RP/0/0/CPU0:R4#sh route vrf A 5.5.5.5
Mon Jan  6 16:47:44.109 UTC

Routing entry for 5.5.5.5/32
  Known via "bgp 100", distance 200, metric 2, type internal
  Installed Jan  6 16:42:29.984 for 00:05:14
  Routing Descriptor Blocks
    2.2.2.2, from 2.2.2.2
      Nexthop in Vrf: "default", Table: "default", IPv4 Unicast, Table Id: 0xe0000000
      Route metric is 2
  No advertising protos.

The active route on R4 is the BGP route, not the OSPF route as the RFC tells us it should be. Even if R4 uses the BGP route as an active route and forwarding route, it still makes no sense for it to have to redistribute into OSPF first. The PE router already has the VPNv4 update. It has no need to share that information with it’s directly connected CEs.

From R6′s perspective, it still has a type 1 intra-area route to R5:

P/0/3/CPU0:R6#show route ipv4 5.5.5.5
Mon Jan  6 16:51:22.808 UTC

Routing entry for 5.5.5.5/32
  Known via "ospf 1", distance 110, metric 4, type intra area
  Installed Jan  6 16:42:29.798 for 00:08:53
  Routing Descriptor Blocks
    10.19.20.19, from 5.5.5.5, via POS0/7/0/0
      Route metric is 4
  No advertising protos.

R4 has two valid routes to 5.5.5.5/32, an OSPF route and a BGP route. Both prefix-lengths are the same. OSPF has a lower AD than BGP, so I would expect to see the OSPF route in the VRF table as the active route.

What’s even more odd is that R4 simply needs to redistribute. But it doesn’t have to redistribute an actual route. As an example I’ll create a policy that blocks everything and use that :

route-policy BLOCK
  drop
end-policy
!
router ospf 100
 vrf A
  redistribute bgp 100 route-policy BLOCK
 !
!
end
RP/0/0/CPU0:R4#clear ospf 100 process
Mon Jan  6 17:08:20.284 UTC
Reset OSPF process 100? [no]: yes

RP/0/0/CPU0:R4#show ospf 100 vrf A neigh
Mon Jan  6 17:10:20.440 UTC

* Indicates MADJ interface

Neighbors for OSPF 100, VRF A

Neighbor ID     Pri   State           Dead Time   Address         Interface
20.20.20.20     1     FULL/  -           -        20.20.20.20     OSPF_SL0
    Neighbor is up for 00:04:37
6.6.6.6         1     FULL/  -        00:00:36    10.19.20.20     POS0/6/0/0
    Neighbor is up for 01:00:44

Total neighbor count: 2

IOS-XR seems to simply want the redistribute command configured, regardless of whether its doing anything…

Regardless of all of that, the sham-link now works in both directions and boths CEs are forwarding over the MPLS cloud.

Sham-link conclusions

  • Know the difference between the behaviour of IOS and IOS-XR
  • Sham-links are point-to-point. If you had to create sham-links between four PEs you are going to need six sham-links
  • A loopback in a VRf can be the end-point for multiple sham-links
  • Avoid sham-links if you can! Might be easier to just give a VPLS solution and let the customer run OSPF directly between their CE’s over the VPLS

OSPF as the PE-CE routing protocols deep dive – Part 1 of 3 – Redistribution

Read part 1
Read part 2
Read part 3

 
When doing L3VPN, using OSPF is actually one of the more complicated options. Vector-based protocols like RIP, EIGRP, and BGP are comparatively simple.

RFC4577 is a great RFC that goes over how OSPF and BGP should operate when it comes to using OSPF as the PE-CE routing protocol.

I wanted to go into detail some of what is noted on the RFC to see just how both IOS and IOS-XR interpret the RFC. Also it makes it a bit fun by purposely trying to break the RFC and seeing what happens.

First, a quick refresh of how PE-CE protocols work when not using BGP as the PE-CE routing protocol. I’m going to brush very lightly over this.

Consider the following network. R2, R3, and R3 are ISP routers in which R2 and R4 are PE routers. R7, R5, and R6 belong to the customer. R7 and R5 are both connected to the same PE while R6 is connected to another PE.
RFC4577 12 OSPF as the PE CE routing protocols deep dive   Part 1 of 3   Redistribution

The CE routers are running OSPF with the PE routers. The PE routers redistribute these OSPF routes into BGP and then converts them to VPNv4 NLRI. These VPNv4 NLRIare advetised to other PE routers via BGP. The PE also converts these VPNv4 routes back into OSPF and then off to the CE router:
RFC4577 22 OSPF as the PE CE routing protocols deep dive   Part 1 of 3   Redistribution

LSA Translation

Taking the above image as an example. R7 is running OSPF with R2. R2 is also running OSPF with R5 and so any LSA updates are sent to R5 from R7 as per standard OSPF rules. When R2 needs to advertise the route over to R4, that LSA needs to be converted to a VPNv4 route. R4 will then convert that VPNv4 route back to an OSPF route on the other side. So how does the RFC state this LSA must be translated?

Section 4.2.6 of the RFC states:

For every address prefix that was installed in the VRF by one of its associated OSPF instances, the PE must create a VPN-IPv4 route in BGP. Each such route will have some of the
following Extended Communities attributes:

- The OSPF Domain Identifier Extended Communities attribute. If the OSPF instance that installed the route has a non-NULL primary Domain Identifier, this MUST be present; if that OSPF instance has only a NULL Domain Identifier, it MAY be omitted. This attribute is encoded with a two-byte type field, and its type is 0005, 0105, or 0205. For backward compatibility, the type 8005 MAY be used as well and is treated as if it were 0005. If the OSPF instance has a NULL Domain Identifier, and the OSPF Domain Identifier Extended Communities attribute is present, then the attribute’s value field must be all zeroes, and its type field may be any of 0005, 0105, 0205, or 8005.

- OSPF Route Type Extended Communities Attribute. This attribute MUST be present. It is encoded with a two-byte type field, and its type is 0306. To ensure backward compatibility, the type 8000 SHOULD be accepted as well and treated as if it were type 0306. The remaining six bytes of the Attribute are encoded as follows:

Area Number – Route Type – Options

In the test network I have already configured mutual redistribution between OSPF and BGP on both PE routers. Let’s see if the VPNv4 routes match what we expect from the RFC. R7 is advertising it’s loopback into OSPF. R2 converts this to a VPNv4 route. Let’s dig into the VPNv4 route itself:

R2#show bgp vpnv4 un all  7.7.7.7
BGP routing table entry for 2.2.2.2:1:7.7.7.7/32, version 28
Paths: (1 available, best #1, table A)
  Advertised to update-groups:
     1
  Local
    10.0.27.7 from 0.0.0.0 (2.2.2.2)
      Origin incomplete, metric 2, localpref 100, weight 32768, valid, sourced, best
      Extended Community: RT:1:1 OSPF DOMAIN ID:0x0005:0x000000010200
        OSPF RT:0.0.0.0:2:0 OSPF ROUTER ID:2.2.2.2:0
      mpls labels in/out 26/nolabel

The route has a number of extended communities. The first one we’ll look at is the domain id value of

OSPF DOMAIN ID:0x0005:0x000000010200

IOS has encoded a type 005 domain ID with a value of 000000010200. This is interesting as I have not hard-coded a domain ID. Section 4.2.4 of the RFC states:

Each OSPF instance MUST be associated with one or more Domain Identifiers. This MUST be configurable, and the default value (if none is configured) SHOULD be NULL.

I have not configured one yet there is one. This means IOS is configuring one automatically even though it SHOULD be null.

The second community we’ll look at is the Route Type Extended Communities Attribute:

OSPF RT:0.0.0.0:2:0

The RFC states that the RT is broken up as follows:

  1. 32-bit Area number
  2. Route-type
  3. Options

From our value above we can see that the original OSPF LSA is from area 0. Our RT says that this route comes from a type-2 LSA, but that’s incorrect as 7.7.7.7 is coming in via a type-1 LSA so that is a bit odd (as we shall see in a bit, it doesn’t actually matter whether this value is 1, 2, or 3 at the end of day). The final byte is the Options byte which is currently zero.

This VPNv4 update is now sent over to R4, who needs to take that information and create a new OSPF LSA and advertise it to R6. What does the RFC say about how the PE needs to do this?

VPNv4 routes received via BGP

Sescion 4.2.8.1 of the RFC states:

With respect to a particular OSPF instance associated with a VRF, a VPN-IPv4 route that is installed in the VRF and then selected as the preferred route is treated as an External Route if one of the following conditions holds:

- The route type field of the OSPF Route Type Extended Community has an OSPF route type of “external”

- The route is from a different domain from the domain of the OSPF instance

What this means is that if a route comes into a PE as an External or NSSA-External , it will always be so. It can never change. If a route comes in with a type of 1, 2, or 3; and the domain-id matches – then the local PE will originate a new type-3 LSA. i.e. the route will appear inter-area on the other customer sites.
If a route comes in with a type of 1, 2, or 3; and the domain-id does not match, then it becomes an external route.
All my routers are currently running IOS and OSPF process ID 100. This means currently all the domain-ids match. This means that R4 should be originating a new type-3 LSA. We can verify this on R6:

R6#sh ip ospf database summary 7.7.7.7

            OSPF Router with ID (6.6.6.6) (Process ID 1)

                Summary Net Link States (Area 0)

  Routing Bit Set on this LSA in topology Base with MTID 0
  LS age: 638
  Options: (No TOS-capability, DC, Downward)
  LS Type: Summary Links(Network)
  Link State ID: 7.7.7.7 (summary Network Number)
  Advertising Router: 4.4.4.4
  LS Seq Number: 80000001
  Checksum: 0x1EDF
  Length: 28
  Network Mask: /32
        MTID: 0         Metric: 2

We see the 7.7.7.7/32 LSA coming from 4.4.4.4. This means the OSPF route should be inter area:

R6#sh ip route 7.7.7.7
Routing entry for 7.7.7.7/32
  Known via "ospf 1", distance 110, metric 3, type inter area
  Last update from 10.0.46.4 on FastEthernet1/0, 00:11:25 ago
  Routing Descriptor Blocks:
  * 10.0.46.4, from 4.4.4.4, 00:11:25 ago, via FastEthernet1/0
      Route metric is 3, traffic share count is 1

Let’s change the domain-id on R4 to Null to see if this will change the route-type:

R4#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
R4(config)#router ospf 1
R4(config-router)#domain-id Null
R4(config-router)#end

Verify:

R6#sh ip route 7.7.7.7
Routing entry for 7.7.7.7/32
  Known via "ospf 1", distance 110, metric 2
  Tag Complete, Path Length == 1, AS 100, , type extern 2, forward metric 1
  Last update from 10.0.46.4 on FastEthernet1/0, 00:00:08 ago
  Routing Descriptor Blocks:
  * 10.0.46.4, from 4.4.4.4, 00:00:08 ago, via FastEthernet1/0
      Route metric is 2, traffic share count is 1
      Route tag 3489661028

R6#sh ip ospf database external 7.7.7.7

            OSPF Router with ID (6.6.6.6) (Process ID 1)

                Type-5 AS External Link States

  Routing Bit Set on this LSA in topology Base with MTID 0
  LS age: 69
  Options: (No TOS-capability, DC)
  LS Type: AS External Link
  Link State ID: 7.7.7.7 (External Network Number )
  Advertising Router: 4.4.4.4
  LS Seq Number: 80000001
  Checksum: 0x863A
  Length: 36
  Network Mask: /32
        Metric Type: 2 (Larger than any link state path)
        MTID: 0
        Metric: 2
        Forward Address: 0.0.0.0
        External Route Tag: 3489661028

As expected, the route is now external.

IOS-XR

I’ve swapped out R4 with an IOS-XR box and configured it the same. How has R6′s loopback been converted into a VPNv4 route?

R2#show bgp vpnv4 un all 6.6.6.6
BGP routing table entry for 1:1:6.6.6.6/32, version 4
Paths: (1 available, best #1, table A)
  Not advertised to any peer
  Local
    4.4.4.4 (metric 4) from 4.4.4.4 (4.4.4.4)
      Origin incomplete, metric 2, localpref 100, valid, internal, best
      Extended Community: RT:1:1 OSPF RT:0.0.0.0:2:0 OSPF ROUTER ID:4.4.4.4:0
      mpls labels in/out nolabel/16012

What’s interesting here is that IOS-XR follows the RFC a little bit more closely in that there is no implicit default Domain-ID. This means a L3VPN where some of your routers are IOS and some are IOS-XR, their Domain-IDs are not going to match unless you change the defaults. This should also mean on R6 I should be seeing external routes from R7 and R5:

RP/0/3/CPU0:R6#sho route ipv4 7.7.7.7
Thu Jan  2 17:05:59.526 UTC

Routing entry for 7.7.7.7/32
  Known via "ospf 1", distance 110, metric 2
  Tag 3489661028, type extern 2
  Installed Jan  2 17:01:21.626 for 00:04:38
  Routing Descriptor Blocks
    10.19.20.19, from 4.4.4.4, via POS0/7/0/0
      Route metric is 2
  No advertising protos.
RP/0/3/CPU0:R6#sho route ipv4 5.5.5.5
Thu Jan  2 17:06:05.378 UTC

Routing entry for 5.5.5.5/32
  Known via "ospf 1", distance 110, metric 2
  Tag 3489661028, type extern 2
  Installed Jan  2 17:01:21.625 for 00:04:43
  Routing Descriptor Blocks
    10.19.20.19, from 4.4.4.4, via POS0/7/0/0
      Route metric is 2
  No advertising protos.

Let’s hard-code the Domain-ID on R4 to ensure they now match:

RP/0/0/CPU0:R4#conf
Thu Jan  2 17:06:40.703 UTC
RP/0/0/CPU0:R4(config)#router ospf 100 vrf A domain-id type 0005 value 0000006$
RP/0/0/CPU0:R4(config)#end
Uncommitted changes found, commit them before exiting(yes/no/cancel)? [cancel]:yes
RP/0/3/CPU0:R6#sho route ipv4 5.5.5.5
Thu Jan  2 17:08:01.164 UTC

Routing entry for 5.5.5.5/32
  Known via "ospf 1", distance 110, metric 3, type inter area
  Installed Jan  2 17:07:33.737 for 00:00:27
  Routing Descriptor Blocks
    10.19.20.19, from 4.4.4.4, via POS0/7/0/0
      Route metric is 3
  No advertising protos.

Knowing the implicit defaults on both platforms can certainly save you from headaches.

Multiple Domain-IDs

IOS gives you the option to have secondary domain-IDs. The configuration guide doesn’t give all that information on what exactly it does, so it’s time to break out Wireshark. First I’ll configure multiple secondary domain-ids on R2:

R2#sh run | sec router ospf
router ospf 1 vrf A
 domain-id type 0005 value 000000010200
 domain-id type 0005 value 000000020200 secondary
 domain-id type 0005 value 000000030200 secondary
 domain-id type 0005 value 000000040200 secondary
 log-adjacency-changes
 redistribute bgp 100 subnets

Will this make R2 generate VPNv3 update with multiple extended OSPF communities? I’m capturing BGP traffic on R4′s core interface and done a route refresh:
RFC4577 3 OSPF as the PE CE routing protocols deep dive   Part 1 of 3   Redistribution
No. The VPNv4 update still only has a single domain-id. Secondary domain-ids are for a receiving PE to look at. If it receives OSPF updates from multiple different domain-id’s, if the ID matches any of the local secondary IDs, then it is considered a match. In order for this to work, all sides will need to match multiple IDs to consider everything internal as each PE can only originate a single ID outbound.

VPLS on Junos signalled via LDP or BGP

Continuing on from the L2VPN on Junos post, let’s switch focus to VPLS. CCC is a point to point technology and so out of the question. That leaves both LDP and BGP to do our VC label signalling. As always, you can use either LDP or RSVP for your transport label signalling.

Slightly different topology this time, as I’m using to test different ways for the CE to attach to the VPLS. For now we’ll simply focus on T1, C2, and T2:
VPLS1 VPLS on Junos signalled via LDP or BGP

All three CE WAN interfaces are in the same subnet running OSPF. The goal is for them to be able to reach each other’s loopbacks. As far as the CE devices are concerned, they are simply plugged into a ‘big switch’

LDP

I’ll concentrate on the PE R3 for this example. We first need to let the router know that the interface pointing towards T1 will in fact be a VPLS interface:

darreno@M7i> show configuration interfaces fe-0/0/1
encapsulation ethernet-vpls;
unit 0;

Our regular RSVP MPLS config, nothing special. Note that LDP is configured for the loopback interface:

darreno@M7i> show configuration protocols
rsvp {
    interface all;
}
mpls {
    label-switched-path TO-R6 {
        to 6.6.6.6;
        no-cspf;
    }
    label-switched-path TO-R7 {
        to 7.7.7.7;
        no-cspf;
    }
    interface all;
}
ospf {
    traffic-engineering;
    area 0.0.0.0 {
        interface all;
    }
}
ldp {
    interface lo0.3;
}

Finally the LDP VPLS config itself. As there is no auto-discovery you need to let Junos know what other PE routers are participating in this VPLS:

darreno@M7i> show configuration routing-instances
VPLS1 {
    instance-type vpls;
    interface fe-0/0/1.0;
    protocols {
        vpls {
            vpls-id 1;
            neighbor 6.6.6.6;
            neighbor 7.7.7.7;
        }
    }
}

I’ve matched the above configs on R6 and R7. Let’s take a look at the network from T1′s perspective:

USERT1@M7i:T1> show ospf neighbor
Address          Interface              State     ID               Pri  Dead
192.168.0.2      fe-0/1/0.0             Full      12.12.12.12      128    37
192.168.0.3      fe-0/1/0.0             Full      14.14.14.14      128    36

USERT1@M7i:T1> show route protocol ospf

inet.0: 9 destinations, 9 routes (9 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

12.12.12.12/32     *[OSPF/10] 00:05:43, metric 1
                    > to 192.168.0.2 via fe-0/1/0.0
14.14.14.14/32     *[OSPF/10] 00:27:15, metric 1
                    > to 192.168.0.3 via fe-0/1/0.0
224.0.0.5/32       *[OSPF/10] 2d 06:44:41, metric 1
                      MultiRecv

USERT1@M7i:T1> ping 12.12.12.12 rapid count 30
PING 12.12.12.12 (12.12.12.12): 56 data bytes
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
--- 12.12.12.12 ping statistics ---
30 packets transmitted, 30 packets received, 0% packet loss
round-trip min/avg/max/stddev = 1.085/1.338/6.357/0.935 ms

USERT1@M7i:T1> ping 14.14.14.14 rapid count 30
PING 14.14.14.14 (14.14.14.14): 56 data bytes
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
--- 14.14.14.14 ping statistics ---
30 packets transmitted, 30 packets received, 0% packet loss
round-trip min/avg/max/stddev = 1.081/1.496/11.077/1.784 ms

T1 considers T2 and C2 to be directly connected via L2. There is an OSPF neighbourship between all three and routes are learned. The data plane is also functioning correctly.

BGP

Let’s turn our attention now to BGP. There are a number of advantages to using BGP, especially if you already run BGP in the SP network. There is another address family which will not only advertise VC labels between PE routers, it will also allow PE routers to auto-discover any other PE configured in the same VPLS.

I’ll keep the interface config the same as above. You may notice that there is more configuration for the BGP version, but in the long run there is less config as that same BGP session is good for all your VPLS instances on the PE.

Let’s start with our BGP config:

darreno@M7i> show configuration routing-options autonomous-system
100;

darreno@M7i> show configuration protocols bgp
group iBGP {
    local-address 3.3.3.3;
    family l2vpn {
        signaling;
    }
    peer-as 100;
    neighbor 6.6.6.6;
    neighbor 7.7.7.7;
}

The BGP VPLS config is slightly different. We now have site-identifiers, but no manual neighbour config. As with our L3VPN set up, we need both RD and RTs configured.

darreno@M7i> show configuration routing-instances
VPLS1 {
    instance-type vpls;
    interface fe-0/0/1.0;
    route-distinguisher 100:200;
    vrf-target target:100:200;
    protocols {
        vpls {
            site T1 {
                site-identifier 1;
            }
        }
    }
}

We test from our CE once again:

USERT1@M7i:T1> show ospf neighbor
Address          Interface              State     ID               Pri  Dead
192.168.0.2      fe-0/1/0.0             Full      12.12.12.12      128    34
192.168.0.3      fe-0/1/0.0             Full      14.14.14.14      128    36

USERT1@M7i:T1> show route protocol ospf

inet.0: 9 destinations, 9 routes (9 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

12.12.12.12/32     *[OSPF/10] 00:03:34, metric 1
                    > to 192.168.0.2 via fe-0/1/0.0
14.14.14.14/32     *[OSPF/10] 00:04:26, metric 1
                    > to 192.168.0.3 via fe-0/1/0.0
224.0.0.5/32       *[OSPF/10] 2d 07:00:30, metric 1
                      MultiRecv

USERT1@M7i:T1> ping 12.12.12.12 rapid count 30
PING 12.12.12.12 (12.12.12.12): 56 data bytes
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
--- 12.12.12.12 ping statistics ---
30 packets transmitted, 30 packets received, 0% packet loss
round-trip min/avg/max/stddev = 1.061/1.480/10.779/1.728 ms

USERT1@M7i:T1> ping 14.14.14.14 rapid count 30
PING 14.14.14.14 (14.14.14.14): 56 data bytes
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
--- 14.14.14.14 ping statistics ---
30 packets transmitted, 30 packets received, 0% packet loss
round-trip min/avg/max/stddev = 1.079/1.183/1.394/0.088 ms

LDP & BGP

There is another way to get this to work. You can use BGP for auto-discovery, while using LDP to advertise the VC labels. This is the same way Brocade Netiron boxes do this, and inter-op is the only reason I would do it this way. If you have BGP running already, why not just let it do both discovery and VC assignment?

The configuration on PE R3 has been changed as follows:

darreno@M7i> show configuration protocols bgp
group iBGP {
    local-address 3.3.3.3;
    family l2vpn {
        auto-discovery-only;
    }
    peer-as 100;
    neighbor 6.6.6.6;
    neighbor 7.7.7.7;
}

darreno@M7i> show configuration routing-instances
VPLS1 {
    instance-type vpls;
    interface fe-0/0/1.0;
    route-distinguisher 100:200;
    l2vpn-id l2vpn-id:100:200;
    vrf-target target:100:200;
    protocols {
        vpls;
    }
}

CE-CE connectivity has been tested as above with no issues at all.