Remote Triggered Black Hole Filtering and Flowspec

RTBH is a mature technology widely used to lower the effects of a DDOS attack against a customer of yours. While it works well, it’s a bit of a sledgehammer. Flowspec is a new technology that gives you a lot more control over what is blocked and as such it’s a lot more powerful.

I’ll be using the following diagram for this post:

P1 and P2 are edge routers peering with transit peers. R3 is a route-reflector which is peered to both P1 and P2. C1 is a customer attached to P3 originating their own address space (172.16.0.0/16)

RTBH

RTBH works on the concept of black-holing traffic towards an IP host/subnet. It does this by advertising a statically injected static route which has been pre-defined to have a next-hop to null0/discard.

As an example, let’s assume a host with the address 172.16.200.10 is under attack. R3, the RR is the route-injector, but it can be any of the internal iBGP routers. There is quite a bit of upfront config with RTBH, but most of this config only needs to be done once.

On all BGP routers in the core you need a route that will be discarded:

[email protected]> show configuration routing-options
static {
    route 192.0.2.1/32 discard;
}

On all routers I want routes learned with a certain community to have their next-hop pointing to the discard route:

[email protected]> show configuration policy-options
policy-statement BLACK-HOLE-FILTER {
    term 1 {
        from community BLACK_HOLE;
        then {
            next-hop 192.0.2.1;
        }
    }
}
community BLACK_HOLE members 65401:666;

I’m going to apply this an an inbound filter on my iBGP sessions:

[email protected]> show configuration protocols bgp group ISP1
import BLACK-HOLE-FILTER;

Basically we are saying that any routes learned via BGP with the above community, set your next-hop to discard. On the route injector we set up an export policy matching static routes with a tag of 666. Any route matching will have the black hole community added. As this will be a specific route we need to ensure it doesn’t leave the confines of our AS and so we also tag no-export:

[email protected]_RR> show configuration policy-options
policy-statement RTBH {
    term BLACK-HOLE {
        from {
            protocol static;
            tag 666;
        }
        then {
            local-preference 5000;
            community add no-export;
            community add BLACK_HOLE;
            next-hop 192.0.2.1;
            accept;
        }
    }
}
community BLACK_HOLE members 65401:666;
community no-export members no-export;

The above policy is then applied outbound on the iBGP session on the route-injector:

[email protected]_RR> show configuration protocols bgp group ISP1
local-address 192.168.0.3;
export RTBH;

RTBH testing and verification

From a router out on the internet I can currently ping the affected host:

[email protected]> ping 172.16.200.10 interface lo0.0 rapid
PING 172.16.200.10 (172.16.200.10): 56 data bytes
!!!!!
--- 172.16.200.10 ping statistics ---
5 packets transmitted, 5 packets received, 0% packet loss
round-trip min/avg/max/stddev = 7.836/10.747/14.091/2.430 ms

I’ll now implement a black hole static on the route-injector:

set routing-options static route 172.16.200.10/32 next-hop 192.0.2.1 resolve tag 666

[edit]
[email protected]_RR# commit and-quit
commit complete
Exiting configuration mode

If we ping from the internet again:

[email protected]> ping 172.16.200.10 interface lo0.0 rapid
PING 172.16.200.10 (172.16.200.10): 56 data bytes
.....
--- 172.16.200.10 ping statistics ---
5 packets transmitted, 0 packets received, 100% packet loss

All packets lost. We can ensure only this /32 is affected by pinging another host in the subnet:

[email protected]> ping 172.16.200.50 interface lo0.0 rapid
PING 172.16.200.50 (172.16.200.50): 56 data bytes
!!!!!
--- 172.16.200.50 ping statistics ---
5 packets transmitted, 5 packets received, 0% packet loss
round-trip min/avg/max/stddev = 5.582/7.116/10.018/1.658 ms

Looking at the edge routers we see the learned /32, and the next-hop of discard:

[email protected]> show route 172.16.200.10 extensive

inet.0: 17 destinations, 17 routes (17 active, 0 holddown, 0 hidden)
172.16.200.10/32 (1 entry, 1 announced)
TSI:
KRT in-kernel 172.16.200.10/32 -> {indirect(262143)}
        *BGP    Preference: 170/-5001
                Next hop type: Indirect
                Address: 0x97106d0
                Next-hop reference count: 3
                Source: 192.168.0.3
                Next hop type: Discard
                Protocol next hop: 192.0.2.1
                Indirect next hop: 94781d0 262143
                State: 
                Local AS: 65401 Peer AS: 65401
                Age: 2:15       Metric2: 0
                Task: BGP_65401.192.168.0.3+64669
                Announcement bits (2): 0-KRT 4-Resolve tree 1
                AS path: I
                Communities: 65401:666 no-export
                Accepted
                Localpref: 5000
                Router ID: 192.168.0.3
                Indirect next hops: 1
                        Protocol next hop: 192.0.2.1 Metric: 0
                        Indirect next hop: 94781d0 262143
                        Indirect path forwarding next hops: 0
                                Next hop type: Discard
                        192.0.2.1/32 Originating RIB: inet.0
                          Metric: 0                       Node path count: 1
                          Forwarding nexthops: 0
                                Next hop type: Discard

The /32 route has been learned through BGP from the route-injector. The correct communities are set. The next-hop goes to a route that is discard, and hence any packets going to this host are now discarded.

Adding and removing hosts are are simple as adding or removing routes on the route-injector.

The above works extremely well, but until the attack is finished and routes removed, that IP address is unroutable over the internet. Any traffic at all going towards it will be black-holed.

Flowspec

There is a more subtle way of doing the above. RFC5575 is the definition of a new filtering mechanism called flowspec. Oddly half the RFC authors are Cisco employess, yet as of today I can only find support for flowspec on Junos.

Essentially flowspec allows routers to advertise firewall filters to your edge BGP devices directly through BGP. Because this is a filter, it allows you to use all the actions of a regular firewall filter. Do you want to police DNS traffic only in a DNS amplification attack? Simple. Flowspec gives you the flexibility to do so.

The first part of enabling flowspec is to configure BGP to carry the NLRI. This will be done on all your internal routers:

[email protected]> configure
Entering configuration mode

[edit]
[email protected]# set protocols bgp group ISP1 family inet flow

[edit]
[email protected]# commit and-quit
commit complete
Exiting configuration mode

Now let’s suppose 172.16.200.10 is under some kind of ICMP attack. I want to block all ICMP traffic to this host from the edge routers, but still allow other traffic through to the host:

[email protected]_RR> show configuration routing-options flow
route BLOCK-ICMP-172.16.200.10 {
    match {
        destination 172.16.200.10/32;
        protocol icmp;
    }
    then discard;
}
term-order standard;

This router will now advertise this filter to all other iBGP peers.

Flowspec testing and verification

We can test this from the internet by trying to ping to this address, and then trying to FTP. Ping should fail, while FTP should be let through:

[email protected]> ping 172.16.200.10 source 192.168.50.1 rapid
PING 172.16.200.10 (172.16.200.10): 56 data bytes
.....
--- 172.16.200.10 ping statistics ---
5 packets transmitted, 0 packets received, 100% packet loss
[email protected]> ftp 172.16.200.10 source 192.168.50.1
Connected to 172.16.200.10.
220 C1 FTP server (Version 6.00LS) ready.
Name (172.16.200.10:root): darreno
331 Password required for darreno.
Password:
230 User darreno logged in.
Remote system type is UNIX.
Using binary mode to transfer files.
ftp>

This works exactly as expected.

You can verify the flow NLRI coming in and applied as a filter on the edge routers:

[email protected]> show route table inetflow.0

inetflow.0: 1 destinations, 1 routes (1 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

172.16.200.10,*,proto=1/term:1
                   *[BGP/170] 00:12:57, localpref 100, from 192.168.0.3
                      AS path: I, validation-state: unverified
                      Fictitious

[email protected]> show firewall

Filter: __default_bpdu_filter__

Filter: __flowspec_default_inet__
Counters:
Name                                                Bytes              Packets
172.16.200.10,*,proto=1                              2352                   28

172.16.200.10,* – meaning destination address 172.16.200.10/32 with any source – proto=1 is ICMP

Conclusion

  • Flowspec gives you a lot more options when it comes to filtering out DDOS attacks. Instead of isolating an IP you are able to filter specific traffic only. These firewall filters are then advertised via BGP to all your iBGP speakers.
  • As this is a firewall filter, you don’t have to specify a discard action. You can just as easily set a policing action.
  • Currently junos supports flowspec on both the inet and family-inet-vpn familes. So no v6 support yet
  • Most other vendors still don’t have working implementations

© 2009-2019 Darren O'Connor All Rights Reserved -- Copyright notice by Blog Copyright