Building my topology

So now it’s time to actually build my topology. There are a number of issues I’d like to get my head around. It helps to know what the planned set up is going to be.

I plan to use my laptop for my studies. From my laptop I’ll be connecting to my dynamips box. My dynamips box is connected to 4 switches. I need to be able to console into all 4 switches remotely, but I don’t want to buy a terminal server. I also want to be able to telnet into all the routers running on the dynamips box.

This is how it’ll look:

This is all possible of course, and I’ll be showing how I did it.

I needed 12 ‘breakout’ ports on the system. Essentially dynamips can map emulated router ports to real ports, allowing you to connect your emulated routers to real switches. I went and bought 3 of these on ebay (Sun Quad Fast PCI Ethernet Card 501-4366):

However, first issue. I can only fit 2 NIC’s in my box. The cards are long, and the 3rd simply down not fit in the box:You can see that there is a heatsinked chip in the way as these cards are very long.

So I went and bought 4 of these:
But these things are pretty awful. They are more bulky than they look, and Ubuntu just doesn’t like to see more than one of them. It’s also messy.

I then decided to find a smaller 4 port NIC that would work. I looked around and found the Dlink DFE-580TX.

Will this fit? It does indeed!

I’ve downloaded and installed the latest version of Ubuntu 64bit server (At this time, 10.10)

Does Ubuntu see all my NICs?

[email protected]:~$ lspci | grep Ethernet
02:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller (rev 03)
04:04.0 Ethernet controller: D-Link System Inc DL10050 Sundance Ethernet (rev 15)
04:05.0 Ethernet controller: D-Link System Inc DL10050 Sundance Ethernet (rev 15)
04:06.0 Ethernet controller: D-Link System Inc DL10050 Sundance Ethernet (rev 15)
04:07.0 Ethernet controller: D-Link System Inc DL10050 Sundance Ethernet (rev 15)
05:00.1 Ethernet controller: Sun Microsystems Computer Corp. Happy Meal 10/100 Ethernet [hme] (rev 01)
05:01.1 Ethernet controller: Sun Microsystems Computer Corp. Happy Meal 10/100 Ethernet [hme] (rev 01)
05:02.1 Ethernet controller: Sun Microsystems Computer Corp. Happy Meal 10/100 Ethernet [hme] (rev 01)
05:03.1 Ethernet controller: Sun Microsystems Computer Corp. Happy Meal 10/100 Ethernet [hme] (rev 01)
06:00.1 Ethernet controller: Sun Microsystems Computer Corp. Happy Meal 10/100 Ethernet [hme] (rev 01)
06:01.1 Ethernet controller: Sun Microsystems Computer Corp. Happy Meal 10/100 Ethernet [hme] (rev 01)
06:02.1 Ethernet controller: Sun Microsystems Computer Corp. Happy Meal 10/100 Ethernet [hme] (rev 01)
06:03.1 Ethernet controller: Sun Microsystems Computer Corp. Happy Meal 10/100 Ethernet [hme] (rev 01)

How has Ubuntu numbered those interfaces? You can find out like this:

[email protected]:~$ sudo vi /etc/udev/rules.d/70-persistent-net.rules
# PCI device 0x10ec:0x8168 (r8169)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:24:21:de:ed:1e",
 ATTR{dev_id}=="0x0", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0"

# PCI device 0x108e:0x1001 (hme)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="08:00:20:8d:49:19",
 ATTR{dev_id}=="0x0", ATTR{type}=="1", KERNEL=="eth*", NAME="eth6"

# PCI device 0x1186:0x1002 (sundance)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:0d:88:cd:4f:7a",
 ATTR{dev_id}=="0x0", ATTR{type}=="1", KERNEL=="eth*", NAME="eth4"

You can see that eth0 is my onboard NIC. eth6 is one of the Sun’s ports and eth4 is one of the D-Link’s ports.

I need to set up my networking interfaces file so ifconfig knows that they are there. I’m going to be putting them in manual mode, and use a script to start them up when I need to run my topologies.

[email protected]:~$ sudo vi /etc/network/interfaces
# The primary network interface
auto eth0
iface eth0 inet static
address 10.20.30.12
netmask 255.255.255.0
gateway 10.20.30.254
#
# Sun
auto eth1
iface eth1 inet manual

# D-Link
auto eth2
iface eth2 inet manual
#
auto eth3
iface eth3 inet manual
#
auto eth4
iface eth4 inet manual
#
auto eth5
iface eth5 inet manual

# Sun
auto eth6
iface eth6 inet manual
#
auto eth7
iface eth7 inet manual
etc....

I have no terminal server, so I’ve just done this: Why buy a terminal server when an old PC will do

Right. So now I have my 12 NIC ports. I have 4 serial cables connected for my switches. The only thing left now is the topology itself. Dynamips allows you to breakout your emulated routers to real switches. This is simple to do in the .net file.

Usually in a .net file, you specify that a particular router port is connected to another router port like so:

[[Router CR1]]
  model = 3725
  console = 2001
  slot1 = NM-4T
  slot2 = NM-1FE-TX
  s1/0 = AR1 s1/0
  s1/2 = AR3 s1/2
  Fa0/0 = CR3 Fa0/0
  Fa2/0 = CR2 Fa2/0

In the above configuration, I’m telling dynamips that R1’s S1/0 interface is connected to AR1’s S1/0 interface. R1’s Fa0/0 interface is connected to CR3’s Fa0/0 interface and so on.

Instead of doing it that way, you could do it this way:

[[Router CR1]]
  model = 3725
  console = 2001
  slot1 = NM-4T
  slot2 = NM-1FE-TX
  s1/0 = AR1 s1/0
  s1/2 = AR3 s1/2
  Fa0/0 = NIO_linux_eth:eth3
  Fa0/1 = NIO_linux_eth:eth2

In the above I’m telling dynamips to map R1’s Fa0/0 interface to eth3, and Fa0/1 to eth2. This then allows me to run a cat5 cable from eth3 on the server to a real switch. It’s also important to note that you can mix and match both modes so you can get very complex topologies.

Let’s cook up a quick bash script that will bring up my interfaces and start the dynamips process.

[email protected]:~$ sudo vim /etc/ccie.sh
#!/bin/bash
#Bring interfaces up
ifconfig eth1 up
ifconfig eth2 up
ifconfig eth3 up
ifconfig eth4 up
ifconfig eth5 up
ifconfig eth6 up
ifconfig eth7 up
ifconfig eth8 up
ifconfig eth9 up
ifconfig eth10 up
ifconfig eth11 up
ifconfig eth12 up

#Start Hypervisor
dynamips -H 7200 &
sudo chmod +x /etc/ccie.sh
[email protected]:~$ sudo /etc/ccie.sh
[email protected]:~$ Cisco Router Simulation Platform (version 0.2.8-RC2-amd64)
Copyright (c) 2005-2007 Christophe Fillot.
Build date: May  9 2009 18:06:28

ILT: loaded table "mips64j" from cache.
ILT: loaded table "mips64e" from cache.
ILT: loaded table "ppc32j" from cache.
ILT: loaded table "ppc32e" from cache.

A quick look via ifconfig shows all 12 interfaces up.

Let’s test all of this with a simple topology. 3 routers to break out to a single 3560 switch:

autostart = False
[10.20.30.12:7200]
    workingdir = /data/dynamips/working
    [[3725]]
        image = /data/dynamips/ios/3725/c3725-adventerprisek9-mz.124-15.T14.UNCOMPRESSED.bin
        ram = 142
        idlepc = 0x6026be14
        ghostios = True
    [[ROUTER R5]]
        model = 3725
        console = 2005
        f0/0 = nio_linux_eth:eth4
        f0/1 = nio_linux_eth:eth5
    [[ROUTER R1]]
        model = 3725
        console = 2001
        f0/0 = nio_linux_eth:eth1
    [[ROUTER R3]]
        model = 3725
        console = 2003
        f0/0 = nio_linux_eth:eth2
        f0/1 = nio_linux_eth:eth3

I’ve started the topology up and connected the correct eth ports to the 3560. Let’s have a look at CDP:

R3#sh cdp neighbors
Capability Codes: R - Router, T - Trans Bridge, B - Source Route Bridge
                  S - Switch, H - Host, I - IGMP, r - Repeater

Device ID        Local Intrfce     Holdtme    Capability  Platform  Port ID
3560TOP          Fas 0/1            158          S I      WS-C3560- Fas 0/6
3560TOP          Fas 0/0            129          S I      WS-C3560- Fas 0/4

What about on the switch itself?

[email protected]:~$ telnet localhost 3000
3560TOP#sh cdp neighbors
Capability Codes: R - Router, T - Trans Bridge, B - Source Route Bridge
                  S - Switch, H - Host, I - IGMP, r - Repeater, P - Phone,
                  D - Remote, C - CVTA, M - Two-port Mac Relay

Device ID        Local Intrfce     Holdtme    Capability  Platform  Port ID
R3               Fas 0/6           152             R S I  3725      Fas 0/1
R3               Fas 0/4           151             R S I  3725      Fas 0/0
R1               Fas 0/5           150             R S I  3725      Fas 0/0
R5               Fas 0/7           149             R S I  3725      Fas 0/0

Finally, let’s do a layer 3 test between 2 routers going through the 3560:
R5

interface FastEthernet0/0
 ip address 10.1.1.5 255.255.255.0

R1

interface FastEthernet0/0
 ip address 10.1.1.1 255.255.255.0

R1#ping 10.1.1.5 repeat 100

Type escape sequence to abort.
Sending 100, 100-byte ICMP Echos to 10.1.1.5, timeout is 2 seconds:
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Success rate is 100 percent (100/100), round-trip min/avg/max = 1/4/24 ms

Perfect. Everything works just as expected :)

Author: Darren

Network Architect. Dual CCIE and JNCIE-SP.

24 thoughts on “Building my topology”

  1. Thanks for the tip.At least on linux you can use a single network card with vlans added to it.
    On linux: vconfig add eth0 2 (vlan 2)
    In dynamips: use eth0.2

  2. How u going to access your Dynamips box from your Laptop?
    Simply using remote desktop feature or are u going to run multiple hypervisors ?

  3. This is one of the best step by step posts that I ever come accross on this topic…Thumbs Up man

  4. SMMS – I run a single hypervisor and I connect via SSH to my linux box. Once it’s all running I telnet onto each emulated router from my laptop

    Motunfolahan – Thanks! :D

  5. You re welcom Darren…I dont have much/any knowledge of Linux and I really want to start my labbing. Any help from you? Will be glad to receive one

  6. Thanks for your kind reply.
    I also try to follow you.
    One more thing, is your vlan trunking work perfectly with these multi port NIC
    cards ?

  7. That’s great !!!!!!!!
    you mean your NICs (both Sun Quad Fast PCI Ethernet Card 501-4366 and Dlink DFE-580TX ) supports vlan trunking with out any issue on Linux environment ? (just to ensure)
    Thanks again for your support.

  8. Yup. Both cards do everything I’ve ever tried on linux. Including raising the MTU far higher than 1500.

    I’ve not yet run into a single thing that cannot be done, and I’ve done some very advanced layer2 stuff

  9. Agree with Alexandru Ulici.On Linux one NIC without VLAN does work.
    In dynamips:use eth0:1 like e0/0 = nio_linux_eth:eth0:1
    gns3 can’t find the sub interface, so vethd is needed to create some virtual NIC binding to eth0 such as veth0,veth1…, then u can find them in Cloud

  10. Hi Darren,

    I am trying to install D-Link DFE-580TX card on the motherboard which is Intel DH67CL. But its not working with win 7 ,even not detecting the cards.

    Now I want to use it in ubuntu which is in vmware within Win7.

    What are the ways to install the DFE-580TX driver in ubuntu…
    Please suggest me…Or which motherboard I should use???????

  11. However I don’t know if it’ll work inside vmware/windows as vmware would be running on top of windows which does not recognise the card. I would recommend running Ubuntu straight.

  12. I’m still currently using my MSI 770T-C45 motherboard – But I build it a couple of years back. I don’t see any motherboard having an issue with it

  13. So you might be using 3x PCI slots for DFE-580TX and not the PCI express
    slots

    Your MOBO specification…

    Expansion Slot(s) 3.0 x PCI ( 3.3 V/ 5 V ),
    4.0 x PCIe 2.0 x16 ( 1.8 V ),
    1.0 x PCIe x1,
    2.0 x CPU,
    1.0 x DIMM 240-pin

Leave a Reply

Your email address will not be published. Required fields are marked *