Isp Workshop PDF

ISP Workshop Lab
Module 1a – Basic Topology and OSPF

Objective: Create a basic physical lab interconnection with one OSPF Area. Ensure that all
routers, interfaces, cables and connections are working properly.
Prerequisites: Knowledge of Cisco router CLI, previous hands on experience.
The following will be the common topology used for the first series of labs.
Figure 1 – ISP Lab Basic Configuration
1
Wednesday, August 06, 2014
Lab Notes
This workshop is intended to be run on a Dynamips server with the appropriate lab topologies set up.
The routers in the Dynamips environment are using service provider IOS. The configurations and
configuration principles discussed below will work on all Cisco IOS Release 12.4 onwards. Earlier
Cisco IOS releases are not supported but will mostly work using the notes below; they will miss some
of the features covered.
The purpose of this module is to construct the workshop lab and introduce everyone to the basic
principles of constructing and configuring a network. An important point to remember, and one that
will be emphasised time and again through out this workshop, is that there is a distinct sequence to
building an operational network:
! After the physical design is established, the connections between the hardware should be built and
verified.
! Next, the routers should have the base configuration installed, and basic but sufficient security
should be set up.
! Next the basic IP connectivity be tested and proven. This means assigning IP addresses on all
links which are to be used, and testing the links to the neighbouring devices.
! Only once one router can see its neighbour does it make sense to start configuring routing
protocols. And start with the IGP (OSPF is chosen for this workshop). There is no purpose to
building BGP while the chosen IGP (in this case OSPF) is not functioning properly. BGP relies on
OSPF to find its neighbours and next hops, and an improperly or non-functioning OSPF will result
in much time wasted attempting to debug routing problems.
! Once the IGP is functioning properly, the BGP configuration can be started, first internal BGP,
then external BGP.
! Remember to RTFM. What is RTFM? It is critical that ISP Network Engineers fully utilise all
information resources. The #1 source is the documentation. Read The F#$% Manual (RTFM) is
the traditional phase used to inform engineers that the answer is in the documentation and go read
it.
! Finally, documentation. Documentation is often overlooked or forgotten. It is an ongoing process

in this workshop. If the instructor asks you to document something, either on the whiteboard in the
class, or at the back of this booklet, it is in your best interests to do so. There can never be too
much documentation, and documentation at the time of network design and construction can
usually saves much frustration at a future date or event.
2
ISP Workshop Lab
Lab Exercise
1. Routers and the Workshops participants. This workshop is laid out such that a group of two
students will operate a single router. 14 routers generally imply at least 28 participants. For
workshops with larger numbers of participants, groups of three should configure a single router.
The Workshop Instructors will divide the routers amongst the workshop participants. In the
following notes, a “router team” refers to the group assigned to one particular router.
2. Introducing the lab. This workshop uses Cisco IOS routers running IOS, but on the Dynamips
systems – Dynamips translates the Cisco 7200 router PowerPC processor instructions in IOS to
those of the host system, allowing Cisco IOS images, and therefore network configurations, to be
run on a host PC system (usual Linux or MacOS based).
The lab will have been preconfigured by the instructors, allowing participants to enter the
following exercises directly. Please read the following steps carefully.
3. Accessing the lab. The instructors will assign routers to each class group, and will indicate the
method of access to the Dynamips server. This will usually be by wireless – if this is the case,
make a note of the SSID and any password required. Also make a note of the IP address (IPv4, as
Dynamips only supports IPv4 access) of the Dynamips server.
Access to Dynamips will be by telnet, to a high port, which the instructor will specify. Each
participant should ensure that their device has a suitable telnet client. Linux and MacOS system
have access to a shell command prompt (or Terminal) programme, which allows telnet at the
command line. Windows users can use the Windows “Command Prompt” with the telnet client
there, but it’s notoriously unreliable. Better to install software such as Putty, TeraTerm,
HyperTerm or similar third party telnet client.
Using the client, connect to the router you have been assigned; for example, to connect to the
console port of Router 1:
telnet 10.0.2.1 2001
or to Router 12:
telnet 10.0.2.1 2012
Once connected, you will see the Dynamips response, followed by the login or command prompt
of the router:
bash-3.2$ telnet 10.0.2.1 2001

Trying 10.0.2.1...
Connected to dynamips.
Escape character is '^]'.
Connected to Dynamips VM "r1" (ID 0, type c7200) - Console port
User Access Verification
Username:
3
If the “Connected to Dynamips VM” won’t appear, even after hitting the Return key several times,
please request help from the workshop instructors.
4. Router Hostname. Each router will be named according to the table location, Router1, Router2,
Router3, etc. Documentation and labs will also refer to Router1 as R1. At the router prompt, first
go into enable mode, then enter “config terminal”, or simply “config” by itself:
Router> enable
Router# config terminal
Enter configuration commands, one per line. End with CNTL/Z.
Router(config)# hostname Router1
Router1(config)#
5. Turn Off Domain Name Lookups. Cisco routers will always try to look up the DNS for any
name or address specified in the command line. You can see this when doing a trace on a router
with no DNS server or a DNS server with no in-addr.arpa entries for the IP addresses. We will turn
this lookup off for the labs for the time being to speed up traceroutes.
Router1 (config)# no ip domain-lookup
6. Disable Command-line Name Resolution. The router by default attempts to use the various
transports it supports to resolve the commands entered into the command line during normal and
configuration modes. If the commands entered are not part of Cisco IOS, the router will attempt to
use its other supported transports to interpret the meaning of the name. For example, if the
command entered is an IP address, the router will automatically try to connect to that remote
destination. This feature is undesirable on an ISP router as it means that typographical errors can
result in connections being attempted to remote systems, or time outs while the router tries to use
the DNS to translate the name, and so on.
Router1 (config)# line con 0

Router1 (config-line)# transport preferred none
Router1 (config-line)# line vty 0 4
7. Disable Source Routing. Unless you really believe there is a need for it, source routing should be
disabled. This option, enabled by default, allows the router to process packets with source routing
header options. This feature is a well-known security risk as it allows remote sites to send packets
with different source address through the network (this was useful for troubleshooting networks
from different locations on the Internet, but in recent years has been widely abused for miscreant
activities on the Internet).
Router1 (config)# no ip source-route
8. Usernames and Passwords. All router usernames should be isplab and all passwords should be
lab-PW. Please do not change the username or password to anything else, or leave the password
unconfigured (access to vty ports is not possible if no password is set). It is essential for a smooth
operating lab that all participants have access to all routers.
Router1 (config)# username isplab secret lab-PW

Router1 (config)# enable secret lab-PW
Router1 (config)# service password-encryption
4
ISP Workshop Lab
The service password-encryption directive tells the router to encrypt all passwords stored in the
router’s configuration (apart from enable secret which is already encrypted).
Note A: There is the temptation to simply have a username of cisco and password of cisco as a
lazy solution to the username/password problem. Under no circumstances must any service
provider operator ever use easily guessable passwords as these on their live operational network1.
Note B: for IOS releases prior to 12.3, the username/secret pair is not available, and operators will
have to configure username/password instead. The latter format uses type-7 encryption, whereas
the former is the slightly more secure md5 based encryption.
9. Enabling login access for other teams. In order to let other teams telnet into your router in future
modules of this workshop, you need to configure a password for all virtual terminal lines.
Router1 (config)# aaa new-model

Router1 (config)# aaa authentication login default local
Router1 (config)# aaa authentication enable default enable
This series of commands tells the router to look locally for standard user login (the username
password pair set earlier), and to the locally configured enable secret for the enable login. By
default, login will be enabled on all vtys for other teams to gain access.
10. Configure system logging. A vital part of any Internet operational system is to record logs. The
router by default will display system logs on the router console. However, this is undesirable for
Internet operational routers, as the console is a 9600 baud connection, and can place a high
processor interrupt load at the time of busy traffic on the network. However, the router logs can
also be recorded into a buffer on the router – this takes no interrupt load and it also enables to
operator to check the history of what events happened on the router. In a future module, the lab
will configuration the router to send the log messages to a SYSLOG server.
Router1 (config)# no logging console

Router1 (config)# logging buffer 8192 debug
which disables console logs and instead records all logs in a 8192byte buffer set aside on the
router. To see the contents of this internal logging buffer at any time, the command “sh log”
should be used at the command prompt.
11. Save the Configuration. With the basic configuration in place, save the configuration. To do this,
exit from enable mode by typing “end” or “<ctrl> Z”, and at the command prompt enter “write
memory”.
Router1(config)#^Z
Router1# write memory
Building configuration...
[OK]
Router1#
It is highly recommended that the configuration is saved quite frequently to NVRAM, especially in
the workshop environment where it is possible for power cables to become dislodged. If the
1
This sentence cannot be emphasized enough. It is quite common for attackers to gain access to networks simply because
operators have used familiar or easily guessed passwords.
5
configuration is not saved to NVRAM, any changes made to the running configuration will be lost
after a power cycle.
Log off the router by typing exit, and then log back in again. Notice how the login sequence has
changed, prompting for a “username” and “password” from the user. Note that at each checkpoint
in the workshop, you should save the configuration to memory – remember that powering the
router off will result in it reverting to the last saved configuration in NVRAM.
12. IP Addresses. This Module will introduce the basic concepts of putting together a sensible
addressing plan for an ISP backbone. We are building one autonomous system out of the 14
routers we have in the lab. The RIRs are typically handing out IPv4 address space in /20 chunks
(depends on which RIR region) – we assume for the purposes of this lab that our ISP has received
a /20. Rather than using public address space, we are going to use a portion of 10/8 (RFC1918 or
private address space) for this lab. In the real world Internet, we would use public address space
for our network infrastructure.
The typical way that ISPs split up their allocated address space is to carve it into three pieces. One
piece is used for assignments to customers, the second piece is used for infrastructure point-to-
point links, and the final piece is used for loopback interface addresses for all their backbone
routers. The schematic in Figure 2 shows what is typically done.
10.0.0.0/20 network block
10.0.0.0 10.0.14.255 10.0.15.0 10.0.15.223 10.0.15.224 10.0.15.255
Customer Address Space Network Infrastructure Loopbacks
Figure 2 – Dividing allocated block of /20 into Customer, Infrastructure and Loopbacks
Study the address plan which was handed out as an addendum to this workshop module. Notice
how the infrastructure addressing starts at 10.0.15.0 and carries on up to 10.0.15.70 – this leave us
room to grow the network by more point-to-point links, up to 10.0.15.223 in fact. Notice how we
have set a side just a single /27 for the router loopbacks – but we have only used the 14 addresses
from 241 up to 254 for our network, leaving some spare for future growth (not that we have future
growth planned for the workshop), an entirely realistic proposition for an ISP backbone. Indeed,
ISPs tend to document their addressing plans in flat text files or in spreadsheets – Figure 3 below
shows an extract from a typical example (using our addressing scheme here).
6
ISP Workshop Lab
Figure 3 – Extract from an ISP addressing plan
13. Back-to-Back Serial Connections. Connect the serial connections as in Figure 1. The DCE side
of a back-to-back serial connection is configured with the clock rate command that drives the
serial circuit. (Older versions of IOS used the clockrate command, now hidden but still
functional.) Physically check the cable to see which side is DCE and which is DTE. On some
routers, the command show controller <interface> will show DCE/DTE status. For
example, on a Cisco 3620 router, show controllers serial 0/0 will produce a result that
will display whether the cable connected to serial 0/0 is a DTE or DCE.
Once the DTE and DCE cables have been determined and the clock rate command has been
applied, configure the IP address (as per the addressing plan discussed earlier) and other
recommended BCP commands that are recommended for each ISP’s Interface:
Router2(config)# interface serial 1/0

Router2(config-if)# ip address 100.1.17.1 255.255.255.252
Router2(config-if)# description 2 Mbps Link to Router4 via DTE/DCE Serial
Router2(config-if)# bandwidth 2000
Router2(config-if)# clock rate 2000000
Router2(config-if)# no ip redirects
Router2(config-if)# no ip directed-broadcast
Router2(config-if)# no ip proxy-arp
Router2(config-if)# no shutdown
NOTE: The lab instructors will have drawn a large network map on the white-board in the
workshop lab. When the IP addresses are assigned, please annotate them and inform the instructor.
All the point to point links MUST be annotated there so that other Router Teams can document
and understand the links and routing in this and future modules.
Q: What network mask should be used on point-to-point links?
A: On serial interfaces, the network mask should be /30 (or 255.255.255.252 in dotted quad
format). There is no point in using any other size of mask as there are only two hosts on such a
7
link. A 255.255.255.252 address mask means 4 available host addresses, of which two are usable
(the other two representing network and broadcast addresses).
14. Ethernet Connections. The Ethernet links between the routers will be made using cross-over RJ-
45 cables – these will directly connect the Ethernet ports on the two routers without the
requirement for an Ethernet switch. IP subnets will again be taken from the Addressing Plan.
Don’t make the mistake of assigning a /24 mask to the interface address – there are only two hosts
on the Ethernet connecting the two routers, so a /30 mask should be entirely sufficient.
15. Ping Test #1. Ping all physically connected subnets of the neighbouring routers. If the physically
connected subnets are unreachable, consult with your neighbouring teams as to what might be
wrong. Don’t ignore the problem – it may not go away. Use the following commands to
troubleshoot the connection:
show arp : Shows the Address resolution protocol

show interface <interface> <number> : Interface status and configuration
show ip interface : Brief summary of IP interface status and configuration
16. Create Loopback Interfaces. Loopback interfaces will be used in this workshop for many things.
These include generating routes (to be advertised) and configuring some BGP peerings. As
discussed earlier in Step 12, we will use part of the allocated IP address block for loopback
interfaces. Most ISPs tend to set aside a contiguous block of addresses for use by their router
loopbacks. For example, if an ISP had 20 routers, they would need a /27 (or 32 host addresses) to
provide a loopback address for each router. We have 14 routers in our lab – to be prudent and
allow for growth, we will set aside a /27 (allows us 32 loopbacks) but only use 14 of them. The
assigned loopback addresses are:
R1 10.0.15.241/32 R8 10.0.15.248/32
R2 10.0.15.242/32 R9 10.0.15.249/32
R3 10.0.15.243/32 R10 10.0.15.250/32
R4 10.0.15.244/32 R11 10.0.15.251/32
R5 10.0.15.245/32 R12 10.0.15.252/32
R6 10.0.15.246/32 R13 10.0.15.253/32
R7 10.0.15.247/32 R14 10.0.15.254/32
For example, Router Team 1 would assign the following address and mask to the loopback on
Router 1:
Router1(config)#interface loopback 0
Router1(config-if)#ip address 10.0.15.241 255.255.255.255
Q: Why do we use /32 masks for the loopback interface address?
A: There is no physical network attached to the loopback so there can only be one device there. So
we only need to assign a /32 mask – it is a waste of address space to use anything else.
Checkpoint #1: call lab assistant to verify the connectivity. Demonstrate that you can ping and telnet
to the adjacent routers.
8
ISP Workshop Lab
17. OSPF with one area in the same AS – activate the OSPF process. Each router Team should
enable OSPF on their router. The OSPF process identifier should be 41 (see example). (The OSPF
process identifier is just a number to uniquely identify this OSPF process on this router. It is not
passed between routers.)
Router1(config)#router ospf 41
The default IOS configuration should be changed so that all interfaces are marked as passive for
OSPF by default. This suppresses routing updates on all router interfaces and stops the router from
unintentionally forming OSPF adjacencies over external facing interfaces, and the potential
problems this may bring2.
Router1(config-router)#passive-interface default
Any interfaces over which OSPF adjacencies should be formed need to be marked with the no
passive-interface subcommand.
Router1(config-router)#no passive-interface fastethernet 0/0
Router1(config-router)#no passive-interface fastethernet 0/1
Router1(config-router)#no passive-interface serial 1/0
18. Activating OSPF on each interface. Now that the OSPF process is configured, each team should
activate OSPF on the individual router interfaces as required. Unlike previous releases of IOS, IOS
12.4 and later releases also allow OSPF to be run on a link (rather than just on a subnet). Rather
than using the older (and confusing) “network” statement, we now activate OSPF on each interface
that will form an adjacency:
Router1(config)#interface serial 1/0

Router1(config-if)#ip ospf 41 area 0
!
Router1(config-if)#interface fastethernet 0/0
!
Router1(config-if)#interface fastethernet 0/1
.
19. Announcing the Loopback /32. The loopback interface also requires OSPF to be activated on it.
Even though there is no adjacency to be formed (because there is no physical neighbour and the
interface is marked as passive by default in the previous step), we need to declare OSPF on the
loopback interface so that the IP address used for the loopback is placed into the OSPF RIB.
20. OSPF Adjacencies. Each team should enable logging of OSPF adjacency changes. (Note: From
IOS 12.4 onwards, log-neighbor-changes is activated by default when OSPF is first configured).
This is so that a notification is generated every time the state of an OSPF neighbour changes, and
is useful for debugging purposes:
2
It is a common error in many ISP configurations to have the IGP active on all interfaces on the router. There have been
many documented accidents where a customer IGP has established a connection with the ISP’s IGP, resulting in a cross
pollution of routing information, and the resulting traffic chaos. Switching off this ability by marking all interfaces passive
by default helps avoid forgetfulness or errors at a later date.
9
Router2(config-router)#log-adjacency-changes
21. Avoiding Traffic Blackhole on Reboot. When a router restarts after being taken out of service,
OSPF will start distribute prefixes as soon as adjacencies are established with its neighbours. In
the next part of the workshop lab, we will be introducing iBGP. So if a router restarts, OSPF will
start up well before the iBGP mesh is re-established. This will result in the router landing in the
transit path for traffic, with out the routing table being completed by BGP. There will not be
complete routing information on the router, so any transit traffic (from customer to peer or
upstream, or vice-versa) will be either dropped, or resulting in packets bouncing back and forth
between adjacent routers. To avoid this problem, we require the router to not announce it is
availability until the iBGP mesh is up and running. To do this, we have to provide the following
command:
Router1(config-router)#max-metric router-lsa on-startup wait-for-bgp
This sets up OSPF such that all routes via this router will be marked as unreachable (very high
metric) until iBGP is up and running. Once iBGP is running, the prefixes distributed by OSPF will
revert to standard metric values, and the router will pass transit traffic as normal.
22. Ping Test #2. Ping all loopback interfaces in the classroom. This will ensure the OSPF IGP is
connected End-to-End. If there are problems, use the following commands to help determine the
problem:
show ip route : see if there is a route for the intended destination

show ip ospf : see general OSPF information
show ip ospf interface : Check if OSPF is enabled on all intended interface
show ip ospf neighbor : see a list of OSPF neighbours that the router sees
Checkpoint #2: call lab assistant to verify the connectivity. Save the configuration as it is on the
router – use a separate worksheet, or the workspace at the end of this Module. You will require this
configuration several times throughout the workshop.
23. Traceroute to all routers. Once you can ping all the routers, try tracing routes to all the routers
using trace x.x.x.x command. For example, Router Team 1 would type:
Router1# trace 10.0.15.252
to trace a route to Router R12. If the trace times out each hop due to unreachable destinations, it is
possible to interrupt the traceroute using the Cisco break sequence CTRL-^.
Q. Why do some trace paths show multiple IP addresses per hop?
A. If there are more than one equal cost paths, OSPF will “load share” traffic between those paths.
Router1>trace router12
Type escape sequence to abort.

Tracing the route to router12.workshop.net (10.0.15.224)
10
ISP Workshop Lab
1 fe0-0.router2.workshop.net (10.0.15.2) 4 msec

fe0-1.router13.workshop.net (10.0.15.6) 0 msec
3 ser0-0.router12.workshop.net (10.0.15.69) 4 msec * 4 msec
Router1>
24. Other Features in OSPF. Review the documentation or use command line help by typing ? to see
other show commands and other OSPF configuration features.
25. Advanced Configuration. Those router teams who have completed this module should refer to
Module 11 of the Advanced BGP Workshop. The set-up steps have been extended to include all
the basic requirements of a router being used in an ISP backbone. While waiting for the module to
complete, now would be a good time to review the advanced Module and incorporate the additions
to the configuration used here.
Review Questions
1. What IP Protocol does Ping and Traceroute use?
2. Ping the IP address of your neighbour’s router (for example 10.0.15.2). Look at the time it took for
the ping to complete. Now Ping the IP address of your router on the same segment (for example
10.0.15.1). Look at the time it took to complete a ping. What are the results? Why is there a
difference?
3. What IOS show command(s) will display the router’s forwarding table?
4. What IOS show command(s) will display the router’s OSPF database?
11
ISP Workshop Lab
Module 1b – Basic Topology and ISIS

Objective: Create a basic physical lab interconnection with one ISIS Area. Ensure that all
routers, interfaces, cables and connections are working properly.
Prerequisites: Knowledge of Cisco router CLI, previous hands on experience.
1
Lab Notes
verified.
should be set up.
protocols. And start with the IGP (ISIS is chosen for this workshop). There is no purpose to
building BGP while the chosen IGP (in this case ISIS) is not functioning properly. BGP relies on
ISIS to find its neighbours and next hops, and an improperly or non-functioning ISIS will result in
much time wasted attempting to debug routing problems.
then external BGP.
it. We will use RTFM through out these exercises to highlight areas where the student should use
the documentation for further deepening. There will be many new commands. Please refer to the
IOS Command Reference on Cisco Connection Online (CCO.cisco.com) for details on each of
these commands.

2
ISP Workshop Lab
Lab Exercise
1. Routers and the Workshops participants. This workshop is laid out such that a group of two
students will operate a single router. 14 routers generally imply at least 28 participants. For
workshops with larger numbers of participants, groups of three should configure a single router.
The Workshop Instructors will divide the routers amongst the workshop participants. In the
following notes, a “router team” refers to the group assigned to one particular router.
2. Introducing the lab. This workshop uses Cisco IOS routers running IOS, but on the Dynamips
systems – Dynamips translates the Cisco 7200 router PowerPC processor instructions in IOS to
those of the host system, allowing Cisco IOS images, and therefore network configurations, to be
run on a host PC system (usual Linux or MacOS based).
The lab will have been preconfigured by the instructors, allowing participants to enter the
following exercises directly. Please read the following steps carefully.
3. Accessing the lab. The instructors will assign routers to each class group, and will indicate the
method of access to the Dynamips server. This will usually be by wireless – if this is the case,
make a note of the SSID and any password required. Also make a note of the IP address (IPv4, as
Dynamips only supports IPv4 access) of the Dynamips server.
Access to Dynamips will be by telnet, to a high port, which the instructor will specify. Each
participant should ensure that their device has a suitable telnet client. Linux and MacOS system
have access to a shell command prompt (or Terminal) programme, which allows telnet at the
command line. Windows users can use the Windows “Command Prompt” with the telnet client
there, but it’s notoriously unreliable. Better to install software such as Putty, TeraTerm,
HyperTerm or similar third party telnet client.
Using the client, connect to the router you have been assigned; for example, to connect to the
console port of Router 1:
telnet 10.0.2.1 2001
or to Router 12:
telnet 10.0.2.1 2012
Once connected, you will see the Dynamips response, followed by the login or command prompt
of the router:
bash-3.2$ telnet 10.0.2.1 2001

Trying 10.0.2.1...
Connected to dynamips.
Escape character is '^]'.
Connected to Dynamips VM "r1" (ID 0, type c7200) - Console port
User Access Verification
Username:
3
If the “Connected to Dynamips VM” won’t appear, even after hitting the Return key several times,
please request help from the workshop instructors.
Router3, etc. Documentation and labs will also refer to Router1 as R1. At the router prompt, first
go into enable mode, then enter “config terminal”, or simply “config” by itself:
Router> enable
Router# config terminal
Enter configuration commands, one per line. End with CNTL/Z.
Router(config)# hostname Router1
Router1(config)#
5. Turn Off Domain Name Lookups. Cisco routers will always try to look up the DNS for any
name or address specified in the command line. You can see this when doing a trace on a router
with no DNS server or a DNS server with no in-addr.arpa entries for the IP addresses. We will turn
this lookup off for the labs for the time being to speed up traceroutes.
Router1 (config)# no ip domain-lookup
Router1 (config)# line con 0

Router1 (config-line)# line vty 0 4
Router1 (config)# no ip source-route
Router1 (config)# username isplab secret lab-PW

Router1 (config)# enable secret lab-PW
Router1 (config)# service password-encryption
4
ISP Workshop Lab
Note A: There is the temptation to simply have a username of cisco and password of cisco as a
lazy solution to the username/password problem. Under no circumstances must any service
provider operator ever use easily guessable passwords as these on their live operational network1.
Note B: for IOS releases prior to 12.3, the username/secret pair is not available, and operators will
have to configure username/password instead. The latter format uses type-7 encryption, whereas
the former is the more secure md5 based encryption.
9. Enabling login access for other teams. In order to let other teams telnet into your router in future
modules of this workshop, you need to configure a password for all virtual terminal lines.
Router1 (config)# aaa new-model

Router1 (config)# aaa authentication login default local
Router1 (config)# aaa authentication enable default enable
router by default will display system logs on the router console. However, this is undesirable for
Internet operational routers, as the console is a 9600 baud connection, and can place a high
processor interrupt load at the time of busy traffic on the network. However, the router logs can
also be recorded into a buffer on the router – this takes no interrupt load and it also enables to
operator to check the history of what events happened on the router. In a future module, the lab
will configuration the router to send the log messages to a SYSLOG server.
Router1 (config)# no logging console

Router1 (config)# logging buffer 8192 debug
which disables console logs and instead records all logs in a 8192byte buffer set aside on the
router. To see the contents of this internal logging buffer at any time, the command “sh log”
should be used at the command prompt.
11. Save the Configuration. With the basic configuration in place, save the configuration. To do this,
exit from enable mode by typing “end” or “<ctrl> Z”, and at the command prompt enter “write
memory”.
Router1(config)#^Z
Router1# write memory
Building configuration...
[OK]
Router1#
It is highly recommended that the configuration is saved quite frequently to NVRAM, especially in
the workshop environment where it is possible for power cables to become dislodged. If the
1
5
configuration is not saved to NVRAM, any changes made to the running configuration will be lost
after a power cycle.
Log off the router by typing exit, and then log back in again. Notice how the login sequence has
changed, prompting for a “username” and “password” from the user. Note that at each checkpoint
in the workshop, you should save the configuration to memory – remember that powering the
router off will result in it reverting to the last saved configuration in NVRAM.
12. IP Addresses. This Module will introduce the basic concepts of putting together a sensible
addressing plan for an ISP backbone. We are building one autonomous system out of the 14
routers we have in the lab. The RIRs are typically handing out IPv4 address space in /20 chunks
(depends on which RIR region) – we assume for the purposes of this lab that our ISP has received
a /20. Rather than using public address space, we are going to use a portion of 10/8 (RFC1918 or
private address space) for this lab. In the real world Internet, we would use public address space
for our network infrastructure.
The typical way that ISPs split up their allocated address space is to carve it into three pieces. One
piece is used for assignments to customers, the second piece is used for infrastructure point-to-
point links, and the final piece is used for loopback interface addresses for all their backbone
routers. The schematic in Figure 2 shows what is typically done.
10.0.0.0 10.0.14.255 10.0.15.0 10.0.15.223 10.0.15.224 10.0.15.255
Study the address plan which was handed out as an addendum to this workshop module. Notice
how the infrastructure addressing starts at 10.0.15.0 and carries on up to 10.0.15.70 – this leave us
room to grow the network by more point-to-point links, up to 10.0.15.223 in fact. Notice how we
have set a side just a single /27 for the router loopbacks – but we have only used the 14 addresses
from 241 up to 254 for our network, leaving some spare for future growth (not that we have future
growth planned for the workshop), an entirely realistic proposition for an ISP backbone. Indeed,
ISPs tend to document their addressing plans in flat text files or in spreadsheets – Figure 3 below
shows an extract from a typical example (using our addressing scheme here).
6
ISP Workshop Lab
Figure 3 – Extract from an ISP addressing plan
13. Back-to-Back Serial Connections. Connect the serial connections as in Figure 1. The DCE side
of a back-to-back serial connection is configured with the clock rate command that drives the
serial circuit. (Older versions of IOS used the clockrate command, now hidden but still
functional.) Physically check the cable to see which side is DCE and which is DTE. On some
routers, the command show controller <interface> will show DCE/DTE status. For
example, on a Cisco 3620 router, show controllers serial 0/0 will produce a result that
will display whether the cable connected to serial 0/0 is a DTE or DCE.
Once the DTE and DCE cables have been determined and the clock rate command has been
applied, configure the IP address (as per the addressing plan discussed earlier) and other
recommended BCP commands that are recommended for each ISP’s Interface:

Router2(config-if)# ip address 10.0.15.17 255.255.255.252
Router2(config-if)# description 2 Mbps Link to Router4 via DTE/DCE Serial
Router2(config-if)# clock rate 2000000
Router2(config-if)# no ip redirects
Router2(config-if)# no ip directed-broadcast
Router2(config-if)# no ip proxy-arp
Router2(config-if)# no shutdown
NOTE: The lab instructors will have drawn a large network map on the white-board in the
workshop lab. When the IP addresses are assigned, please annotate them and inform the instructor.
All the point to point links MUST be annotated there so that other Router Teams can document
Q: What network mask should be used on point-to-point links?
A: On serial interfaces, the network mask should be /30 (or 255.255.255.252 in dotted quad
format). There is no point in using any other size of mask as there are only two hosts on such a
link. A 255.255.255.252 address mask means 4 available host addresses, of which two are usable
(the other two representing network and broadcast addresses).
7
14. Ethernet Connections. The Ethernet links between the routers will be made using cross-over RJ-
45 cables – these will directly connect the Ethernet ports on the two routers without the
requirement for an Ethernet switch. IP subnets will again be taken from the Addressing Plan.
Don’t make the mistake of assigning a /24 mask to the interface address – there are only two hosts
on the Ethernet connecting the two routers, so a /30 mask should be entirely sufficient.
wrong. Don’t ignore the problem – it may not go away. Use the following commands to
troubleshoot the connection:
show arp : Shows the Address resolution protocol

show interface <interface> <number> : Interface status and configuration
show ip interface : Brief summary of IP interface status and configuration
16. Create Loopback Interfaces. Loopback interfaces will be used in this workshop for many things.
These include generating routes (to be advertised) and configuring some BGP peerings. As
discussed earlier in Step 12, we will use part of the allocated IP address block for loopback
interfaces. Most ISPs tend to set aside a contiguous block of addresses for use by their router
loopbacks. For example, if an ISP had 20 routers, they would need a /27 (or 32 host addresses) to
provide a loopback address for each router. We have 14 routers in our lab – to be prudent and
allow for growth, we will set aside a /27 (allows us 32 loopbacks) but only use 14 of them. The
assigned loopback addresses are:
R1 10.0.15.241/32 R8 10.0.15.248/32
R2 10.0.15.242/32 R9 10.0.15.249/32
R3 10.0.15.243/32 R10 10.0.15.250/32
R4 10.0.15.244/32 R11 10.0.15.251/32
R5 10.0.15.245/32 R12 10.0.15.252/32
R6 10.0.15.246/32 R13 10.0.15.253/32
R7 10.0.15.247/32 R14 10.0.15.254/32
For example, Router Team 1 would assign the following address and mask to the loopback on
Router 1:
Router1(config-if)#ip address 10.0.15.241 255.255.255.255
Q: Why do we use /32 masks for the loopback interface address?
A: There is no physical network attached to the loopback so there can only be one device there. So
we only need to assign a /32 mask – it is a waste of address space to use anything else.
Checkpoint #1: call lab assistant to verify the connectivity. Demonstrate that you can ping and telnet
to the adjacent routers.
17. ISIS with one area and one level (level-2) within the same AS. Each router team should enable
ISIS on their router, and use workshop as the ISIS ID in the configuration. In this module, we use
8
ISP Workshop Lab
level-2 in one area (49.0001) and use wide metrics (IOS default is the historical narrow metric and
is not considered good practice). The NET should be 49.0001.x.x.x.x.00, where x.x.x.x represents
the router loopback IP address. For example, the loopback for Router1 is 10.0.15.241 which will
make the NSAP address 49.0001.0100.0001.5241.00.
Router1(config)# router isis workshop
Router1(config-router)#net 49.0001.0100.0001.5241.00
Router1(config-router)#is-type level-2-only
Q: Why do you have is-type level-2-only configured? Write your answer here:
Hint: A nice trick for converting the loopback interface address into the NSAP address is to take
the loopback address and put the missing leading zeroes in. For example, Router 5 loopback
address is 10.0.15.245; this is rewritten to 010.000.015.245 putting in the missing zeroes. Then
rather than having the dot after every third character, move it to be after every fourth character. So
010.000.015.245 becomes 0100.0001.5245.
18. Setting Wide Metrics. We also set the metric-style to wide. ISIS supports two types of metric,
narrow (historic now and not suitable for modern networks) and wide. IOS still defaults to narrow
metrics, so we need to enter explicit configuration to change this to wide. We will set the metric
style to be wide for level-2:
Router1(config-router)#metric-style wide level-2
19. Activating ISIS on each interface. Now that the ISIS process is configured, all connected point to
point and shared ethernet interfaces need to be configured with ISIS. Else, you will not be able to
see network advertisements via ISIS from routers two or more hops away. Here is an example
configuration as would be used on Router1:
Router1(config)# interface fastethernet 0/0

Router1(config-if)# ip router isis workshop
!
!
Note: the ISIS ID on the interfaces must be matched with the router’s ISIS ID.
20. ISIS Circuit Type and ISIS Metrics. Now each team needs to set the circuit type and ISIS metric
on each physical interface.
The default circuit type is level-1-2 even though the router has been defined to be a level-2-only
router.
The default ISIS metric for all interface types is 10. Unlike OSPF in IOS, ISIS has no automatic
scheme to convert the interface bandwidth into a metric value. ISPs deploying ISIS have to come
up with their own scheme (as in fact many ISPs using OSPF now also do). In the lab we will use
metric 2 for the Ethernet interfaces and metric 20 for the Serial interfaces.
9
Combining the above, gives the example:

Router1(config-if)# isis metric 2 level-2
Router1(config-if)# isis circuit-type level-2-only
!
!
21. Announcing the Loopback /32. We do not need to set up ISIS adjacencies on the loopback
interface as there are no neighbours there, so we mark it as passive:

Router1(config-router)# passive-interface Loopback0
Note that this will tell ISIS to install the loopback interface address in the ISIS RIB. We do NOT
need to add an ip router isis statement onto the loopback interface itself. This is different
from the required OSPF configuration, and often catches many engineers out, especially those who
are learning ISIS after gaining experience with OSPF.
22. ISIS Adjacencies. Enable logging of ISIS adjacency changes. This is so that a notification is
generated every time the state of a CLNS neighbor changes, and is useful for debugging purposes.
(Note: From IOS 12.4 onwards, log-adjacency-changes is activated by default when ISIS is first
configured.)
Router1(config)#router isis workshop

Router1(config-router)#log-adjacency-changes
23. Avoiding Traffic Blackhole on Reboot. When a router restarts after being taken out of service,
ISIS will start distribute prefixes as soon as adjacencies are established with its neighbours. In the
next part of the workshop lab, we will be introducing iBGP. So if a router restarts, ISIS will start
up well before the iBGP mesh is re-established. This will result in the router landing in the transit
path for traffic, with out the routing table being completed by BGP. There will not be complete
routing information on the router, so any transit traffic (from customer to peer or upstream, or
vice-versa) will be either dropped, or resulting in packets bouncing back and forth between
adjacent routers. To avoid this problem, we require the router to not announce it is availability
until the iBGP mesh is up and running. To do this, we have to provide the following command:
Router1(config)#router isis workshop

Router1(config-router)#set-overload-bit on-startup wait-for-bgp
This sets ISIS’ overload bit such that all routes via this router will be marked as unreachable (very
high metric) until iBGP is up and running. Once iBGP is running, the prefixes distributed by ISIS
will revert to standard metric values, and the router will pass transit traffic as normal.
10
ISP Workshop Lab
24. Ping Test #2. Ping all loopback interfaces. This will ensure the ISIS IGP is connected End-to-End.
If there are problems, use the following commands to help determine the problem:

show clns neighbor : see a list of CLNS-IS neighbors that the router sees
show clns interface : see if ISIS is configured and see the IS type
show isis database : see ISIS link state database that the router has learned
show isis rib : see ISIS routes that the router has learned
show isis topology : see the ISIS topology as learned by the router
router – use a separate worksheet, or the workspace at the end of this Module. You will require this
configuration several times throughout the workshop.
25. Traceroute to all routers. Once you can ping all the routers, try tracing routes to all the routers
using trace x.x.x.x command. For example, Router Team 1 would type:
Router1# trace 10.0.15.252
to trace a route to Router R12. If the trace times out each hop due to unreachable destinations, it is
possible to interrupt the traceroute using the Cisco break sequence CTRL-^.
Q. Why do some trace paths show multiple IP addresses per hop?
A. If there are more than one equal cost paths, ISIS will “load share” traffic between those paths.
Router1>trace router12


Router1>
26. Other Features in ISIS. Review the documentation or use command line help by typing ? to see
other show commands and other ISIS configuration features.
11
Review Questions
1. What IP Protocol does Ping and Traceroute use?
2. Ping the IP address of your neighbour’s router (for example 10.0.15.2). Look at the time it took for
the ping to complete. Now Ping the IP address of your router on the same segment (for example
10.0.15.1). Look at the time it took to complete a ping. What are the results? Why is there a
difference?
3. What IOS show command(s) will display the router’s forwarding table?
4. What IOS show command(s) will display the router’s ISIS database?
12
ISP Workshop Lab
Module 1c – iBGP
Objective: Create a basic physical lab interconnection with IGP and one BGP AS number.
Ensure that all routers, interfaces, cables and connections are working properly.
Prerequisites: Module 1a (OSPF) or Module 1b (ISIS).
1
Monday, March 24, 2014
Lab Notes
verified.
should be set up.
then external BGP.
it. We will use RTFM through out these exercises to highlight areas where the student should use
the documentation for further deepening. There will be many new commands. Please refer to the
IOS Command Reference on Cisco Connection Online (CCO.cisco.com) for details on each of
these commands.

2
ISP Workshop Lab
Lab Exercise
1. Configuring iBGP. Before we set up iBGP with our neighbours in our AS, we need to do some
basic preparation on the router. The IOS defaults are not optimised for Service Provider networks,
so before we bring up BGP sessions, we should set the defaults that we require.
The default distance for eBGP is 20, the default distance for iBGP is 200, and the default distance
for ISIS is 115. This means that there is a potential for a prefix learned by eBGP to override the
identical prefix carried by ISIS. Recall from the Routing presentation that there is a distinct
separation between BGP and ISIS processes – prefixes present in ISIS will never be found in BGP,
and vice-versa. To protect against accidents1, the eBGP distance is set to 200 also. The command
to do this is the bgp distance subcommand, syntax is:
distance bgp <external-routes> <internal-routes> <local-routes>
Note: This should be included in all future BGP configurations in this workshop. For
example, for Router2, the configuration might be:
Router2(config)#router bgp 10
Router2(config-router)#distance bgp 200 200 200
2. Logging of BGP Adjacency State. Enable logging of BGP neighbour changes. This is so that a
notification is generated every time the state of a BGP neighbour changes state, and is useful for
debugging purposes:
Router2(config-router)# bgp log-neighbor-changes
Note: From IOS 12.3 onwards, bgp log-neighbor-changes is activated by default when BGP is
first configured.
3. Configuring iBGP Neighbours. All Routers will be in Autonomous System (AS) 10 for this first
lab. Use the show ip bgp summary to check the peering. The BGP peering will be established using
the loopback interfaces’ IP address.
Router2 (config-router)#neighbor 10.0.15.241 remote-as 10
Router2 (config-router)#neighbor 10.0.15.241 update-source loopback 0
Router2 (config-router)#neighbor 10.0.15.241 description iBGP with Router1
Router2 (config-router)#
1
There have been several incidents in the past where denial of service attacks on ISP networks have been successful
because ISPs have omitted basic routing protocol security. Setting the BGP distances to be greater than any IGP is one of
the mitigation methods available.
3

Q. Why is update-source loopback 0 necessary on iBGP?
Use show ip bgp summary to check the status of the iBGP neighbour connections. If the iBGP
session is not up and/or no updates are being sent, work with the Router Team for that neighbour
connection to troubleshoot the problem.
4. Sanity Check. Remember to use the following commands to ensure you are getting the
information you are suppose to be getting:
show ip bgp summary : see a list of BGP peers that the router sees
show ip bgp : see a list of BGP paths that the router sees
show ip route : see all the routes that the router has installed
Q. Are there routes seen via show ip bgp? If not, why not? Are there any routes tagged "B" when
you do a show ip route?
5. Add Networks via BGP. Each Router Team will use BGP to advertise the address block used for
the Module. For example, Router Team 1 would add:
4
ISP Workshop Lab
Router1 (config)#router bgp 10

Router1 (config-router)#network 10.0.0.0 mask 255.255.240.0
Use show ip bgp on neighbour’s router to see if you are advertising your network via BGP.
Q. Does the network show up via BGP? If not, why?
Enter a static route for the CIDR block. For example, Router 1 would use:
Router1 (config)#ip route 10.0.0.0 255.255.240.0 Null0
Q. Does the network show up via a neighbour’s BGP? Use the command show ip bgp
neighbor <neighbour’s IP address> advertised-routes to see what you are exporting to the
other router. Physically go to one of your neighbour’s routers and check their BGP Table.
Explain what you see.
Q. Does the network appear in the router’s forwarding table? Use the command show ip route
to check the local forwarding table. If not, why not?
6. For Routers with IOS prior to 12.3 add the following commands to BGP:
Router1 (config)#router bgp 10

Router1 (config-router)# no synchronization
Router1 (config-router)# no auto-summary
Q. Does the network appear in the router’s forwarding table? Use the command show ip route
to check the local forwarding table. What does the no synchronisation command do in BGP?
How does it effect the router’s forwarding table?
Note: As from IOS 12.3, synchronization and auto-summarisation are disabled by default and do
not appear in the default BGP configuration. These two features have not been required in Service
Provider networks since the classless routing system was introduced to the Internet in 1994.
Checkpoint #1: call the lab assistant to verify the connectivity.
7. Adding a “customer” route into BGP (Background & Example). We are now going to add a
“customer” route into BGP on each router. Now in the lab we don’t have any “customers” as such
connected to our routers, so we are going to simulate the connectivity by simply using a Null0
interface. As an example, in real life, the configuration to connect a customer would look
something like this.
ip route 172.16.4.0 255.255.255.128 Serial 0/5/2 permanent

!
router bgp 64509
network 172.16.4.0 mask 255.255.255.128
!
This would add a static route pointing 172.16.4.0/25 towards Serial 0/5/2 – the latter interface
would be a fixed link connecting to the customer site. 172.16.4.0/25 would be the address space
5
that the ISP had assigned to the customer. The BGP network statement would then add the
customer address block into the ISP’s iBGP.
Note: the permanent keyword ensures that the static route is always in the routing table, even if
the interface physically goes down. Many ISPs use this to ensure they don’t have iBGP churn
when their customer links go down.
8. Adding a “customer” route into BGP. The “customer” address space that each router team will
introduce into the iBGP is listed below – we will each use a /26, for simplicity’s sake.
R1 10.0.0.0/26 R8 10.0.1.192/26
R2 10.0.0.64/26 R9 10.0.2.0/26
R3 10.0.0.128/26 R10 10.0.2.64/26
R4 10.0.0.192/26 R11 10.0.2.128/26
R5 10.0.1.0/26 R12 10.0.2.192/26
R6 10.0.1.64/26 R13 10.0.3.0/26
R7 10.0.1.128/26 R14 10.0.3.64/26
Each team should now set up a static route pointing to the NULL0 interface for the /26 that they
are to originate. Once the static is set up, the team should then add an entry into the BGP table.
Here is an example for Router11:
Router11 (config)# ip route 10.0.2.128 255.255.255.192 Null0

Router11 (config)# router bgp 10
Router11 (config-router)# network 10.0.2.128 mask 255.255.255.192
9. Check the BGP table. Are there routes seen via show ip bgp? If not, why not? Once every team in
the class has done their configuration, each team should see the aggregate as well as the fourteen
/26s introduced in the previous step. If this is not happening, work with your neighbours to fix the
problem.
Checkpoint #2: call the lab assistant to demonstrate the current BGP table.
10. Other Features in iBGP. Review the documentation or use command line help by typing ? to see
other show commands and other BGP configuration features.
6
ISP Workshop Lab
Review Questions
1. What IOS show command(s) will display the router’s BGP route table?
2. Why change the eBGP distance from the default 20 to 200?
3. Why is the static pull up route necessary in relation to inserting a prefix into iBGP?
7
ISP Workshop Lab
Module 2 – OSPF Areas

Objective: To migrate from one flat OSPF topology to OSPF areas, plus introduce neighbour
authentication and area summarisation in the lab network.
Prerequisite: Module 1 and OSPF presentation

Topology:
Figure 1 – OSPF Areas

1
Lab Notes
This module provides an introduction to areas in OSPF as used in ISP networks. The module forms
the basis for the Route Reflector Module which follows. It is strongly recommended that the OSPF
presentation is reviewed prior to starting this module. Prerequisites for this Module are Module 1, and
the OSPF presentation. As before, ask the workshop instructors or refer to online documentation if
there is any doubt.
The topology shown in Figure 1 allows the student to configure the best combination for the study of
OSPF areas.
Functional assignments of routers in Figure 1:

! Routers 5 to 10 represent the “core network” and the core interfaces are all in OSPF area 0. In a
typical ISP backbone, these routers would carry all the internal link routes known in the ISPs
network. Routers 5, 6, 9 and 10 are Area Border Routers, whereas Routers 7 and 8 are Internal
Routers.
! Routers 1 and 3 are completely in OSPF area 10. Router 5 is the boundary between area 0 and area
10, so requires configuration for both areas.
! Routers 2 and 4 are completely in OSPF area 20. Router 6 is the boundary between area 0 and area
20, so requires configuration for both areas.
! Routers 11 and 13 are completely in OSPF area 30. Router 9 is the boundary between area 0 and
area 30, so requires configuration for both areas.
! Routers 12 and 14 are completely in OSPF area 40. Router 10 is the boundary between area 0 and
area 40, so requires configuration for both areas.
Lab Exercise
1. OSPF Migration. This module is going to show the student how to migrate from a single area
OSPF backbone to one using multiple areas. As ISP networks grow, quite often a single OSPF
area ends up with an unmanageably large number of routers, resulting in reduced convergence
rates impacting network performance. Of course, standard design advice is to start off with areas
from day one, but many network designers unfortunately choose to ignore lessons well learnt by
their predecessors.
Before starting ensure that the configuration is exactly as it was at the end of Module 1. OSPF
adjacencies should all be properly established and iBGP should be running with each neighbour
adjacency established and active. Also, shutdown or disconnect interfaces which are not being
used for this Module. These include connections between Router1 and Router2, Router1 and
Router13, Router2 and Router14, and Router13 and Router14. It makes no sense to have a
physical connection between non-zero OSPF areas as traffic between two non-zero areas must
always transit Area 0.
Checkpoint #1: Call the lab assistant and show the router configuration and connectivity.
2. Migration strategy. Review the OSPF for ISPs presentation where one possible migration
strategy was discussed. The best strategy is to start at the edge of the network and work inwards.
We want to migrate to using OSPF areas without causing too much downtime in the network – in
other words, we want to try and ensure that all the iBGP sessions remain active. To this end, the
outermost links of the network are changed from OSPF Area 0 to the sub area first. We want to
2
ISP Workshop Lab
avoid having any islands of Area0 in the network – this will result in partition of the network and
disconnectivity for the “stub” Area 0. The following steps work through the process in detail.
3. Step 1: Network Outer Edge. The first links we will move from Area 0 are the links between
Router1 and Router3, Router2 and Router4, Router11 and Router13, and Router12 and Router14.
The Router teams at either side of each link MUST work together to coordinate the change. For
example, Router1 and Router3 should coordinate to change the area on the link between them from
Area0 to Area10 at exactly the same time.
Example for Router1 (IOS ≥ 12.4):
interface serial 1/0

ip ospf 41 area 10
interface loopback 0
ip ospf 41 area 10

ip ospf 41 area 10
ip ospf 41 area 10
Once these two teams have changed the area for the link between them (don’t forget to do the
loopback interface as well!), the OSPF adjacency will re-establish in the new area, and the rest of
the network will become visible again. The other teams mentioned above should do a similar thing
for their interconnecting links and respective areas.
4. Step 2: Network Inner Edge. Once the outer edge links have been moved to the new areas, we
are ready to change the area of the next links in from the outer edge; these are the links between
Router3 and Router5, Router4 and Router6, Router11 and Router9, and Router12 and Router10.
Again the Router teams at either side of each link MUST work together to coordinate the change.
For example, Router4 and Router6 should coordinate to change the area on the link between them
from Area0 to Area20 at exactly the same time.

ip ospf 41 area 20
ip ospf 41 area 20

ip ospf 41 area 20
Once these two teams have changed the area for the link between them, the OSPF adjacency will
re-establish in the new area, and the rest of the network will become visible again. The other teams
mentioned above should do a similar thing for their interconnecting links and respective areas.
Note that Routers 5, 6, 9 and 10 should only change the Area of the link to their outer neighbour –
they should NOT change the area of the loopback interface.
3
5. Further steps. Our lab network is not very large or complex, so the migration was quite simple.
However, for larger networks, we would repeat the above steps, working from the outside in to the
backbone Area, taking great care to ensure that no Area 0 islands are created as part of the process.
Such a migration should have negligible impact on the network’s operation.
6. All router teams should check their routing table. The routing table should look the same as it
did in Module 1. All the prefixes in Area 0 will be available in Areas 10 through to 40. Although
notice now that several of the routes will be marked as Inter Area (IA) routes. Here is an example
of the typical output from show ip route as should be seen from Router2:
Gateway of last resort is not set
10.0.0.0/8 is variably subnetted, 32 subnets, 3 masks

O 10.0.15.4/30 [110/2] via 10.0.15.1, 01:07:57, FastEthernet0/0
C 10.0.15.0/30 is directly connected, FastEthernet0/0
S 10.0.0.0/20 is directly connected, Null0
C 10.0.15.16/30 is directly connected, Serial1/0
O IA 10.0.15.28/30 [110/130] via 10.0.15.1, 01:07:47, FastEthernet0/0
C 10.0.15.24/30 is directly connected, FastEthernet0/1
Traces through the network. Once you have analysed the routing table as it is now, and
compared with any notes you made in Module 1, try some traceroutes through the network.
Checkpoint #2: Call the lab instructors and show the function of your router. The inter-area OSPF
“peerings” will now be up. If you have an area border router, demonstrate the neighbour
relationships using “sh ip ospf neigh”. If you are not in an area border, you should now have a more
complete routing table.
STOP AND WAIT HERE
7. Intra Area Authentication – Part 1. OSPF supports router authentication within areas. This is
quite important inside ISP networks to prevent the introduction of improperly configured or
unintended equipment.
Each area will turn on authentication within that area. Routers which are ABRs will naturally have
to enter configuration to cover all areas the router has interfaces in. This first step will enable each
area to support authentication using the area N authentication message-digest command.
An example configuration for Router6 might be:
router ospf 41
area 0 authentication message-digest
!
4
ISP Workshop Lab
Note that this does not affect the actual adjacencies on the routers – it only tells the router that the
area mentioned will use authentication, if it is configured.
8. Intra Area Authentication – Part 2. Now that support for authentication in each area has been
configured, the second step is to actually set the authentication password to be used, and the
interface it has to be used on. The password that should be used for all areas in this example is
cisco. MD5 encryption should be used rather than the standard simple encryption – to do this, use
the message-digest-key sub-interface command.
router ospf 41
!
interface ethernet 0/0
ip ospf message-digest-key 1 md5 cisco
!
Notice now that the OSPF adjacencies do not come up unless the neighbouring router has also
entered the same configuration and key. Notice also how the OSPF adjacencies were reset as the
configuration was entered – security is being introduced, so the adjacencies are reset.
Note: the message-digest-key allows up to 255 keys to be set per interface. It is generally not
recommended to set more than one per interface, as the router will try and communicate with its
neighbours using all keys. If a key needs to be upgraded, common practice then is to set a second
key, allowing a graceful changeover without compromising the functioning of the network. Once
all the routers on the network are using the new key, the old one should be removed.
Note: Wherever an OSPF session is configured from now on in the workshop, all Router
Teams MUST use passwords on these OSPF sessions.
Checkpoint #3: Call the lab instructors and demonstrate the OSPF is still working following the
addition of neighbour authentication and that the OSPF routing table is still the same as it was
previously.
9. Inter-Area summarisation. It is fairly important, especially for larger ISP networks, to try and
summarise the announcements going from sub-areas into Area 0 as much as is possible.
Unfortunately we don’t have an extensive lab network here, so it’s going to be hard to see much
benefit from summarisation, but we will try with the limited resources we have.
Areas 10, 20, 30 and 40 can summarise the point-to-point links they announce into Area 0.
Routers5, 6, 9 and 10, respectively, will introduce this configuration into the network. If we take
the example of Area 20 and Router 6.
5
Area 20 point-to-point subnets are 10.0.15.16/30 (Router2-Router4) and 10.0.15.20/30 (Router4-

Router6). These can be aggregated into 10.0.15.16/29. The configuration which has to be
introduced on Router6 therefore is:
router ospf 41
area 20 range 10.0.15.16 255.255.255.248
Once the team operating Router6 has done this, check the routing table. Everyone (apart from Area
20 routers) will see that the entries for 10.0.15.16/30 and 10.0.15.20/30 will have been replaced by
the aggregate, 10.0.15.16/29.
The other three router teams (5, 9 and 10) should do a similar thing for their areas, aggregating the
point to point links between their area routers. Once complete, eight /30s in the routing table
should have been replaced by four /29 aggregates.
10. Final check. Use the various “show ip ospf” commands to see the OSPF status of the lab network
now. Check the routing and the routing table. If you are missing any adjacencies, work with your
neighbouring routers to work out why, and what might have gone wrong with the neighbour
authentication.
Checkpoint #4: Call the lab instructors and show the routing table after the area summarisation was
applied.
6
ISP Workshop Lab
Module 3 – BGP Route Reflector Lab

Objective: To migrate the network to BGP route reflectors as an alternative to using a fully-
meshed iBGP.
Prerequisites: Module 1, the Scaling BGP presentation and (optionally) Module 2

Topology:
Figure 1 – Route Reflector Clusters
1
Thursday, July 17, 2014
Lab Notes
While not a pre-requisite, it is helpful if the OSPF Areas Module is completed before this Module. The
design of this route-reflector network assumes the network layout and configuration used in the OSPF
Area Module. While it will work without the OSPF area configuration (for example, if ISIS is chosen
as the IGP instead), this is a good exercise in explaining by example how many ISPs overlay route
reflectors on their IGP.
The topology shown in Figure 1 allows the student to study the best combination of route reflector
clusters.
Routers 5 to 10 represent the “core network” running a fully meshed iBGP, with Routers 5, 6, 9 and
10 being route reflectors. In a typical ISP backbone, these routers would carry all the routes known in
the ISPs network, and possibly all the Internet routes too. Routers 1, 3 and 5 represent a cluster – say
the “distribution network”, carrying out the function of aggregating customer connections. Routers 2,
4 and 6 form another cluster. Routers 9, 11 and 13 form another cluster. And Routers 10, 12 and 14
form a cluster.
Lab Exercise
1. Retain the configuration used in the previous Module. This means that all routers are using the
same IGP (OSPF or ISIS). Before we start with this module, we need to ensure that the BGP
configuration is exactly as it was coming from Module 1 – ensure that all the iBGP sessions are up
and running.
2. Migrating from full mesh iBGP to Route Reflector based iBGP. The process for migration is
relatively straightforward, as long as simple rules are adhered to. The key is to configure a client to
be a route reflector client before removing the redundant iBGP sessions. This ensures that the
client iBGP sessions are up and running and passing on prefixes, meaning that removing the
surplus sessions should not cause any break in iBGP routing information. The next steps in the
module work through the process, and each Router Team affected should follow them carefully.
Teamwork is important – with good teamwork it is more than likely that the network can be
migrated without any major connectivity impact.
3. Configure the iBGP peerings with the Route Reflector Clients. Routers 1 to 4 and Routers 11
to 14 will become route reflector clients. So the teams operating Routers5, 6, 9 and 10 should
reconfigure their peerings with Routers 1 and 3, 2 and 4, 11 and 13, and 12 and 14 respectively to
be route-reflector clients. To configure a neighbour as a client, we simply add in the route-
reflector-client BGP neighbour command as per the following example:
Example for Router 5:
router bgp 10
no synchronization
bgp log-neighbor-changes
network 10.0.0.0 mask 255.255.240.0
neighbor 10.0.15.241 remote-as 10
neighbor 10.0.15.241 description iBGP with Router1
neighbor 10.0.15.241 update-source loopback0
neighbor 10.0.15.241 route-reflector-client
2
ISP Workshop Lab

neighbor 10.0.15.243 description iBGP with Router3
neighbor 10.0.15.243 update-source loopback0
neighbor 10.0.15.243 route-reflector-client
Remember that if you have multiple address families configured for iBGP, you will need to enter
the IPv4 address family to add these extra lines (if this workshop is done in conjunction with the
IPv6 materials). The configuration to do this would be:
router bgp 10
address-family ipv4
neighbor 10.0.15.241 ...
and then the rest of the configuration above would apply.
Checkpoint #1: Once the teams operating Router 5, 6, 9 and 10 have completed adding the two route
reflector client lines, they should call the lab instructors and show the function of their router. The lab
instructors will then tell the class to proceed with remaining steps in this module.
STOP AND WAIT HERE
4. Removing redundant iBGP sessions – Step 1. Once the teams operating Routers5, 6, 9 and 10
have ALL completed step 3, they should remove the redundant iBGP sessions with route reflector
clients which are not in their cluster.
This means that:

• Router5 removes iBGP sessions with Routers 2, 4, 11, 12, 13 and 14
router bgp 10
no neighbor 10.0.15.241
5. Remove redundant iBGP sessions – Step 2. The clients of the route reflectors can also remove
their redundant iBGP sessions. Each client should have only ONE iBGP session – the other 12 set
up in Module 1 can now be removed.
router bgp 10
3
Thursday, July 17, 2014
no neigbbor 10.0.15.253
Router 1 only retains its iBGP session with Router5 – every other one should be removed as in the
example above. The other teams operating route reflector clients (Routers 2, 3, 4, 11, 12, 13 and
14) should do the same thing.
6. Remove redundant iBGP sessions – Step 3. Finally Router7 and Router8, the two core routers
which are not route reflectors, can remove their redundant iBGP sessions – they no longer need to
run iBGP with any of the 8 clients (Routers 1 to 4, 11 to 14). They still need to run iBGP with
other core routers though, as they are part of the core iBGP mesh.
router bgp 10
7. Final Check. The network has now been migrated from full mesh IBGP with each router having
13 neighbours, to a Route Reflector set up, with core routers being route reflectors, and edge
routers being clients. Use “show ip bgp <address>” to see how reflected prefixes show up on the
clients. How do you explain the path choices which you see?
Checkpoint #2: Call the lab instructors and show the function of your router. You should have
peerings with all the routers in your cluster, and any peers/clients. You should also demonstrate the
output from “show ip route” so that you can see which routes you are hearing from which routers.
4
ISP Workshop Lab
Module 4 – Essential Router Configuration

Objective: To configure VTY filters, SSH, Out of Band Access, Network Time Protocol,
TACACS+ and other Essential Features on the ISP Workshop Lab Network.
Prerequisite: Module 1, the IOS Essentials presentation, and Module 2
This module is an optional module which can be inserted into the ISP/IXP Workshop programme at
any stage after the completion of Module One. However, it assumes aspects of the network design
used in the OSPF Area module, and the BGP Route Reflector module.
The aim of the module is to introduce the student to certain essential aspects of Cisco’s IOS which are
not introduced as part of the standard modules. Most features introduced elsewhere in the lab are
involved with routing – however, IOS has a rich set of features which are often overlooked, but
required, in an ISP’s network.
The structure of this module will follow the one worked on in the Advanced OSPF Module, and the
Route Reflector Module. It is a typical design of ISP backbones, and gives good hints about how to
configure Out of Band Management, structure an NTP hierarchy in the network, and deploy
TACACS+ for system management access.
1. VTY Security. The first step is to demonstrate how to secure the vtys on a router. This is the most
often forgotten security feature on service provider routers, and creates a huge security risk for
these devices when connected to the public Internet. The vtys on the router will now be secured so
that only connections from recognised addresses will be permitted.
2. VTY Security – telnet source address. The first step in doing this is to configure telnet so that it
uses the loopback interface as the source address for all telnet packets originated by the router.
ip telnet source-interface loopback 0
To check that this has worked, telnet from your router to a neighbouring router and then enter the
“who” command. You will see that you are logged in, and the source address will be displayed.
For example, using telnet from Router1 to Router3 gives:
Router3>who
Line User Host(s) Idle Location
* 2 vty 0 philip idle 00:00:00 10.0.15.241
3. VTY Security – constructing an access-list. Next, an access-list with all the loopback interfaces
in the lab is constructed. Given we carefully constructed our addressing plan in Module 1 to ensure
that the loopbacks all fitted into a single contiguous range of addresses, this access-list might be
something like:
access-list 3 permit 10.0.15.240 0.0.0.15 ! Router loopback range

access-list 3 permit host 192.168.1.4 ! Workshop laptop
access-list 3 deny any
which covers all routers in the workshop lab and the workshop laptop (nameserver etc).
1
Saturday, May 28, 2011
4. VTY Security – applying the filter to the vtys. Now that the access-list has been constructed it
can be applied to the vtys on the router. To do this, use the access-class command:
line vty 0 4
access-class 3 in
5. Checking the filters. Try using telnet to connect to a neighbouring router. Do you still get access?
If not, why not? (If you don’t, then the access-list is either wrong, or you have forgotten to do the
step which set telnet source address to be that of the loopback interface.) Now remove the “ip
telnet source-interface loopback 0” command set earlier and try using telnet to connect to a
neighbouring router. If that team has implemented the access-list you will now find that access is
barred.
It is strongly recommended that all ISPs implement vty filters on all their routers in their network.
Absence of vty filters has usually led to service provider networks being compromised by
intruders.
A final point: on the IOS images used in this workshop, there are 5 vtys available. On some
releases or versions of software with extended features in them, a greater number of vtys may be
available. Often these are not displayed in the configuration. It is worth checking how many vtys
are supported by your version of IOS when you are configuring these filters. Only protecting the
first 5 is not good security – people attempting to break into such a router will occupy the first 5
vtys, then gain a login prompt on the 6th vty. To check for extra vtys on your router, enter the
following in configuration mode:
Router1#conf t
Router1(config)#line vty 0 ?
<1-63> Last Line number
<cr>
Router1(config)#line vty 0 63
Router1(config-line)#access-class 3 in
The ? option will display how many vtys the router supports. When applying the access-list, make
sure that all vtys are included as per the example above.
6. VTY Transport support. The final step is to change the transports permitted on the vtys from the
defaults in IOS to those which are applicable to a service provider backbone. We will also change
the supported transports on the console port and the auxiliary port of the router. The only required
input transport for an ISP backbone router is telnet (and Secure Shell if you have an image with
SSH support). The only required output transport for an ISP backbone router is also telnet. And the
preferred transport should be none (otherwise the router CLI will try and use all transports to
resolve what has been typed in at the command prompt). For example:
line con 0
transport output telnet
transport preferred none
line aux 0
transport input none
line vty 0 4
transport input telnet
2
ISP Workshop Lab

7. Setting a domain name. The next step takes the lab participants through setting up SSH on their
routers. Before we can do that, we need to set a domain-name – this is used by SSH key generation
to define the hooks for a suitable key to identify the router. We’ll just set the domain as follows:
ip domain-name workshop.net
8. Using SSH for router access. The router software images have SecureShell support available in
them, so this step will enable SSH support for access to and from the routers. You can recognise
an image which has SSH in it as it will have either “k4” or “k9” in the name, signifying 3DES
crypto support; for example, c2600-advipservicesk9-mz.124-25a is a crypto service provider image
for the 2600XM series routers. If you do a show version at the command prompt, you’ll see what
IOS release the router is running.
To enable support for SSH on the router, first the key needs to be set. To do this enter the
following IOS command in configuration mode:
crypto key generate rsa
which will generate an RSA crypto key for the router. The prompt asks what key modulus should
be chosen, in a range from 360 (more or less useless) to 2048 (the best). This key will be
automatically stored in a file in NVRAM – this file is not readable by any user on the router.
Next, set the source ip address for all SSH connections from the router to be the loopback
interface:
ip ssh source-interface Loopback 0
SSH is now available for use on the router. Early IOS images only permitted SSH access to the
router, but more recent images also allow outbound SSH connections. And from 12.3 onwards,
SSHv2 is also supported.
9. Adding SSH to VTY Transport. We now need to change the transports permitted on the vtys to
allow SSH inbound and outbound. This is a simple case of adding ssh to the lines which have
telnet in them.
NOTE WELL: ISPs who care about their infrastructure security completely disable telnet
support on their routers – industry best practice considers telnet an obsolete and archaic
protocol, ill-suited for use for management of public network infrastructure equipment.
For example:
line con 0
transport output telnet ssh
line aux 0
transport input none
line vty 0 4
transport input ssh
3

10. Using SSH. Finally, test SSH on your router. This is simply a case of using the ssh command at
the command prompt, as you would use the telnet command to connect to a remote neighbour.
Checkpoint #1: Check the configuration and operation of the vty filters as suggested in the previous
steps. Check that SSH connections work, and demonstrate this to the lab assistant.
11. Out of Band Management. One of the often forgotten aspects of any operational network is out
of band access to the router and switch equipment. To manage remote sites, ISPs never rely on
simply being able to telnet to the router – they configure some other means of access, usually
connecting the console port of the router to a modem or a device called a “terminal server”. With
this facility it is possible to get access to the remote equipment when network connectivity is
unavailable (due to misconfiguration, or failure of WAN links) or console access is required to
support functions which are incompatible with telnet access (such as upgrading the operating
system images on some devices).
The lab instructors have configured an access router to be used as the out of band management
system for the lab network. The router is at IP address 192.168.1.253, connected to the secondary
LAN on Router6. The router has several asynchronous connections on it (the 2511RJ has 16, the
AS2611 has 32). In most circumstances, the access routers are used to connect modems. However,
this feature can be reversed so that the router provides a means of accessing the devices connected
to each port, the so-called “terminal server” function.
12. Configuring Out of Band Access. The standard lab set up has the COM1: port of the supplied
computer connected to the console port of the router. This connection should now be moved to the
auxiliary port (usually labelled AUX). Locate the cable from the access router which has been run
to your station and connect it to the console port of the router. The layout is represented in Figure
1.
13. Using Out of Band Access. With the connections in place, each team should log into their router
through the auxiliary port. Check that the OOB router is pingable. If not, try and work out why
not. Each tty line on the OOB router is referenced through a particular TCP port. In general, line X
is referenced through port 200X. To access a particular line, telneting to the router and specifying
port 200X will give access to line X. For the purpose of this lab, Router1 has been connected to
Line1, Router2 to Line2, etc. The following table lists these assignments:
Router1 Line1 telnet 192.168.1.253 2001

…etc…
Each router team should now access the console of their router via the OOB router. They should
enter the appropriate telnet command and check that the login functions as expected.
4
ISP Workshop Lab
The instructors will log into the Out of Band router and show the class the configuration. Observe
especially the configuration used for each physical line on the router.
Figure 1 – Out of Band connections

Checkpoint #2: Check the configuration and operation of the OoB access. Each router team should
use this for the rest of this module. The lab instructor will explain the OoB router configuration to the
class.
5
References: IOS Essentials – the section on Network Time Protocol (NTP). Cisco Documentation
– NTP Commands.
14. Setting up NTP. A hierarchy will be used for setting up NTP. This avoids fully meshing the NTP
connections on the routers, and is just as effective. The objective is to have a synchronised time
source throughout the network. This way every router’s time is synchronised with their peers –
among other things, it makes comparison of log files so much easier.
Most ISPs choose a few of their core routers as those which will act as the main NTP peers for the
rest of the routers in their network. In other words, a hierarchy is used. The steps are this:
• The ISP chooses a few external sources to synchronise time against. This can either be by
asking permission from the external sources, or by using public servers.
• The ISP chooses a few internal routers or other systems which will provide the main time
synchronisation for their networks. These are usually core routers.
• The ISP then configures the remaining routers in the network to peer with these “core time”
routers.
The instructor will have configured some systems to be NTP stratum 1 devices. These devices will
have the “official” time in the workshop lab, and will be the devices which all other routers will
synchronise their time with. The Advanced OSPF Module introduced OSPF areas – the Area
Border Routers (ABRs) will be the only routers which peer directly with “official” workshop time
sources. The remaining routers will get their time from the ABRs. (Notice that the ABRs are also
configured as Route Reflectors in the Route Reflector Module.)
15. Initial Preparation. The lab time sources are located on the ethernet LAN connected to Router6.
Router6 will announce this network to the rest of the classroom using OSPF. Configuration
required for Router6 is:
router ospf 41
network 192.168.1.0 0.0.0.255
The LAN uses the network 192.168.1.0/24 – the time sources have IP addresses 192.168.1.1/24
and 192.168.1.2/24. All routers in the lab should check that they can ping both IP addresses. If you
cannot, please inform the lab instructors.
16. Set up NTP Security on all routers. MD5 will be used to encrypt the NTP sessions between all
NTP peers in the network. The word “cisco” will be the MD5 key for this lab exercise.
ntp authenticate
ntp authentication-key 1 md5 cisco
ntp trusted-key 1
17. Set NTP to use the router’s loopback interface as the NTP source used for the peering session.
ntp source loopback 0
18. Set the NTP Server Session to the systems acting as the master stratum 1 clock source. (This
command is applied to the routers in Area 0 only – in other words, Routers 5, 6, 7, 8, 9 and 10
only.)
6
ISP Workshop Lab
ntp server 192.168.1.1 key 1

ntp server 192.168.1.2 key 1
19. Use the “show ntp” commands to check the status of the NTP synchronisation and the clock
setting on the router.
show clock
show ntp associations
show ntp status
20. Set the NTP peer connections between the routers in Area 0. Use the loopback interface for
your router as the NTP peering point. Notice the choice of loopback interface rather than any other
active interface on the router. Routers 5, 6, 7, 8, 9 and 10 should all configure each other as NTP
peers. An example configuration for Router 5 might be:
ntp peer 10.0.15.246 key 1

ntp peer 10.0.15.247 key 1
ntp peer 10.0.15.248 key 1
ntp peer 10.0.15.249 key 1
ntp peer 10.0.15.250 key 1
Doing this ensures that the Area 0 routers are all in sync with each other at the same stratum, and
allows the lab to build a hierarchy.
21. Again use the “show ntp” commands to check the status of the NTP synchronisation and the
clock setting on the router.
Checkpoint #3: The teams with the routers in Area 0 should demonstrate their configuration and its
operation to the lab instructors.
22. Set NTP synchronisation for the remaining routers in the network. Again use the loopback
interface for each router as the NTP synchronisation source.
The remaining router teams should now configure ntp server statements for their ntp
synchronisation. The Area Border Routers should be the time source for each non-zero area. For
example, the teams in OSPF Area 10 should set up synchronisation with the ABR of Area 10.
Teams in OSPF Area 30 should set up synchronisation with the ABRs of Area 30. For example,
Router 11 might be configured with:
ntp server 10.0.15.249 key 1 ! Router 9
23. Set the NTP peer connections between the remaining routers in each area. In this case,
Router1 and Router3 should set up an ntp peering with each other, Router2 and Router4 should set
up an ntp peering with each other, Router11 and Router13 should set up ntp peering with each
other, and Router12 and Router14 should set up ntp peering with each other. An example for
Router14:
ntp peer 10.0.15.252 key 1 ! Router 12
7
This is done to ensure that if the ntp server for these routers disappears, the time synchronisation
within the OSPF area remains consistent.
24. Use show ntp commands to check the NTP status, the NTP time synchronisation, and the setting
of the clock on the router.
show clock
show ntp associations
show ntp status
25. Set the time zone on the router. The router can be set with a time zone offset from GMT. The
timezone command takes a string of characters – obviously set it to the local timezone. Note that
only the first seven characters are used in any time display. The following sets the time zone for
Singapore with a GMT offset of +8.
clock timezone SST 8
or
clock timezone GMT+8 8
26. (Optional) Daylight savings time setting. If the local time zone has daylight savings time (called
summer time), this can be configured on the router too. For example, in the UK:
clock timezone GMT 0

clock summer-time BST recurring last Sun Mar 2:00 last Sun Oct 3:00
UK standard time zone is GMT (Greenwich Mean Time). However, in the summer months, the
UK changes to BST (British Summer Time), where the time becomes GMT+1. The “summer-
time” configuration tells the router this, and between 1am on the first Sunday in March and 1am
on the last Sunday in October, it will automatically add one hour to the time and time zone set on
the router. Alter the start and finish of summertime according to local conditions. Notice that in the
Southern Hemisphere, daylight savings time is usually between November and April.
27. Set time stamps for all logs on the router.
service timestamps debug datetime localtime show-timezone msec

service timestamps log datetime localtime show-timezone msec
28. (Optional) Additional security can be added with ACLs. An Access List (ACL) and Access
Group can be created and applied to all NTP peer sessions. Since the loopback interface is used,
the ACLs will mirror the NTP peer statements. For example, Router 14 would have the following
ACL #5 and Access Group created. Notice that the 4 routers being used as core NTP peers need an
ACL which will allow access for all routers in the network.
access-list 5 permit 10.0.15.250

access-list 5 permit 10.0.15.252
!
ntp access-group peer 5
29. (Optional for routers with a separate real time clock.) Set the real time clock.
Use the ntp update-calendar command to update the router’s real time clock.
8
ISP Workshop Lab
ntp update-calendar
30. (Optional) Set up NTP for the ISP’s customers. NTP broadcast can be set on any interface to
allow an ISP’s customers to use NTP or Simple NTP (SNTP) to pick up time packets and
synchronise their systems.
interface ser 0/0

ntp broadcast
Checkpoint #4: Demonstrate your configuration and its operation to the lab instructors. Save the
configuration to NVRAM. Do you notice any extra information at the top of the configuration saved in
NVRAM?
31. Login Banner. IOS by default has a simple welcome message when a new administrative
connection to the router is opened. Most ISPs tend to customise this banner to be appropriate to
their business – see the example given in the ISP Essentials presentation. We will now set up a
login banner for the routers in the workshop lab. Use an appropriate greeting – consult the ISP
Essentials presentation to see examples of appropriate and inappropriate greetings. If you use an
inappropriate greeting, expect the lab instructors to ask you to change it. Use the following
example:
login banner ^
Cisco Systems ISP/IXP Workshop Lab
^
32. Logging. Routers by default capture syslog data produce locally by various features in the IOS.
However, the default logging set up is probably not optimal for ISPs. Each router team should
configure logging defaults on their router to be as follows:
no logging console
logging source-interface Loopback 0
logging trap debugging
logging buffered 16384 debugging
logging facility local4
logging 192.168.1.4
This command set will set the log source interface to the Loopback 0 interface, trap level to debug
(i.e. most detailed), create a 16K buffer on the router and store the most detailed logs there, and
any logs sent to the 192.168.1.4 loghost should be sent using syslog facility local4.
As mentioned in Module 1, it is highly desirable (if not best practice) to disable logging to the
router console. If you still haven’t done this then the command to do so is no logging console.
Console logging is on by default in IOS.
NOTE: For ISP operations, it is strongly recommended to DISABLE console logging – router
consoles are usually connected to terminal servers as we have just seen, and if the router is under
some stress due to events taking place on it, or on the network, it will waste considerable CPU
cycles by sending log messages to the 9600baud console port, thereby further slowing it down.
Virtually all ISPs disable console logging, and alternatively have the syslog messages sent to the
local syslog host, as per the configuration above.
9
Checkpoint #5: The lab instructors will show the log information gathered on the loghost PC. They
will also show how to configure a Linux box to gather router loghosts – for example, files such as
syslog.conf and the set up of syslogd.
33. Configuring TACACS+ – background.
The lab instructors have configured a TACACS+ server in the network. Its IP address is
192.168.1.4. They have also configured usernames and passwords on the TACACS+ server for
each router team.
The first account created simulates a general ISP staff account. The person can log into the router
and check the system status, but he or she cannot configure the router or enter privilege (enable)
mode. The naming convention is “router#” where # is the number of the router. For example,
Router2’s staff’s username would be “router2”.
The second account created simulates a NOC or Engineering staff account. The person can log into
the router and have full access to all commands. The naming convention is “nocrouter#” where “#”
is the number of the router. For example, Router 10’s NOC username would be “nocrouter10”.
34. Configuring TACACS+ – router configuration.
To configure your router to use the TACACS+ server for user authentication, enter the following
sequence of commands.
ip tacacs source-interface loopback 0

!
tacacs-server host 192.168.1.4
tacacs-server key cisco
!
aaa new-model
aaa authentication login default tacacs+ enable
aaa authentication enable default tacacs+ enable
aaa authorization commands 0 default tacacs+ none
aaa accounting commands 15 default start-stop tacacs+
aaa accounting exec default start-stop tacacs+
!
The first aaa command tells the router to use the AAA model of system security – basically this
tells the router to use the “authentication, authorisation, accounting” system rather than the default
legacy system.
The next two commands “aaa authentication” define the process a router goes through to
authenticate a connecting user. “login default” says that by default a user logging in to the router
must first be passed to the tacacs+ server for the user’s name and password before looking for the
local enable secret configuration. The enable configuration is necessary just in case the TACACS+
server is unavailable. Notice that if a user fails to authenticate on the TACACS+ server, the session
is terminated. The router only fails over to the enable secret in the event of the TACACS+ server
being unavailable.
Likewise, enable default tells the router that by default a user requesting to go into enable mode
must first be passed to the tacacs+ server before checking the local enable secret for a password.
10
ISP Workshop Lab
The advantage of checking a remote server for the enable password is that the enable password can
be changed for an entire network, on the server, without having to log into each router. This greatly
simplifies administration.
The aaa authorization command enables command authorisation on the router. The commands 0
directive states that standard commands when a user logs in are checked to see if the user is
authorised to use them. The authorisation is specified by the tacacs+ none commands – the former
tells the router to check the TACACS+ server, while none tells the router not to use command
authorisation when the TACACS+ server is unavailable.
The final two commands are aaa accounting instructions, which sends aaa accounting records to
the TACACS+ server. The start-stop option logs to the TACACS+ server when commands were
executed, and completed.
Checkpoint #6: Demonstrate your configuration and its operation to the lab instructors. The lab
instructors will show the TACACS+ information gathered on the TACACS+ server PC. They will also
show how to configure a Linux box to be a TACACS+, where to get the software, and the various
configuration files.
35. Setting up DNS. The lab instructors will have configured a name server for the workshop
network. This step will now configure name and address resolution for the workshop network.
Recall that in Module 1, ip domain-lookups were turned off as there was no nameserver for the lab
network. Domain lookups can now be turned back on. We already set the domain name earlier on
in the module when we activated SSH (Step 7). The remaining commands required to enable DNS
on the router are:
ip name-server 192.168.1.4
!
ip domain-lookup
!
36. Testing DNS Configuration. The domain name used is “workshop.net” – the nameserver has
been configured to provide name and address resolution for this domain.Try running a traceroute
from your router to the other side of the workshop network. Notice how fully qualified domain
names are now present in the traceroute, rather than addresses only. (Fully qualified domain name
(or FQDN) is a complete hostname and domain name pairing.) An example of a traceroute from
Router1 to Router13 is given here (based on the topology of the OSPF Area Module).
Router1>traceroute router13

1 ser0-1.router3.workshop.net (10.0.15.10) 0 msec 4 msec 4 msec

Router1>
11
Notice from the traceroute what the format of the names is. The reverse DNS has been configured
such that the addresses map into a name with format InterfaceNumber.RouterNumber.Domain.
This is very common practice on the Internet today, and makes debugging of network problems
much easier.
Important: It is strongly recommended that all ISPs populate the reverse DNS for their
infrastructure equipment. It makes it extremely hard to debug your own backbone if you do not use
the reverse DNS. There is no security risk for doing this – router security is implemented via
access-lists on the interfaces and vtys, not by omitting content from the reverse DNS.
Checkpoint #7: Demonstrate your configuration and its operation to the lab instructors. The
instructors will demonstrate a traceroute through the network. Notice the detailed use of interface
name as part of the router name – this is common practice amongst many ISPs.
37. Netflow (Local). One of the very important features which has been available on Cisco IOS for
many years is Netflow. Netflow is basically an information gatherer on the router – it records
source and destination address, source and destination ports, the protocol types, and the amount of
traffic between those destinations.
As can be imagined, this type of information is invaluable for the service provider. They can
monitor traffic flows in their network, spot popular destinations, deal with attacks on the network,
do sensible bandwidth and capacity planning, and manage the operational network more
effectively. The ISP Security Workshop covers the use of Netflow as a Network Security Tool in
much more detail.
The Router Teams should now enable Netflow on their routers. Netflow is enabled per interface,
and captures flow information inbound on that interface. Netflow should be enabled on active
interfaces only (obviously). An example might be:
interface fastethernet 0/0

ip flow ingress
ip flow egress
!
which will gather flow statistics for traffic inbound (ingress) and outbound (egress) on fastethernet
0/0.
Once this has been done, display the flow statistics of the router on the console. The command to
do this is show ip cache flow. What do you see? Output will look similar to:
Beta-7200-2>sh ip cache flow

IP packet size distribution (17093 total packets):
1-32 64 96 128 160 192 224 256 288 320 352 384 416 448 480
.000 .735 .088 .054 .000 .000 .008 .046 .054 .000 .009 .000 .000 .000 .000
512 544 576 1024 1536 2048 2560 3072 3584 4096 4608
.000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000
IP Flow Switching Cache, 1257536 bytes

3 active, 15549 inactive, 12992 added
12
ISP Workshop Lab
210043 ager polls, 0 flow alloc failures

last clearing of statistics never
Protocol Total Flows Packets Bytes Packets Active(Sec) Idle(Sec)
-------- Flows /Sec /Flow /Pkt /Sec /Flow /Flow
TCP-Telnet 35 0.0 80 41 0.0 14.5 12.7
UDP-DNS 20 0.0 1 67 0.0 0.0 15.3
UDP-NTP 1223 0.0 1 76 0.0 0.0 15.5
UDP-other 11709 0.0 1 87 0.0 0.1 15.5
ICMP 2 0.0 1 56 0.0 0.0 15.2
Total: 12989 0.0 1 78 0.0 0.1 15.4
SrcIf SrcIPaddress DstIf DstIPaddress Pr SrcP DstP Pkts

Et1/1 144.254.153.10 Null 144.254.153.127 11 008A 008A 1
Et1/1 144.254.153.112 Null 255.255.255.255 11 0208 0208 1
Et1/1 144.254.153.50 Local 144.254.153.51 06 701D 0017 63
The first section highlighted above shows a summary of the IP packet size distribution of traffic
crossing the router. This example is from a lab, so is a bit unexciting. But it shows that 73.5% of
all traffic is between 33 and 64 bytes in size.
The second section highlighted above shows a break down of the number of flows, their size and
their activity for certain well known ports.
And the final section shows the current snapshot of traffic crossing the router, source and
destination interfaces, source and destination addresses, protocol, source and destination ports, and
the number of packets.
Note: Netflow does create a 5-10% CPU hit on all the software based routers. In normal
circumstances ISPs adding Netflow shouldn’t cause too much impact on the router’s forwarding
speed. Adding Netflow to a router running at 90% CPU isn’t a good idea, but then again industry
experience deems that a router running at 90% CPU is very definitely performing suboptimally
anyhow.
38. Netflow (Export). Netflow records saved by the router can also be exported. While we can look at
this configuration in the lab, it’s fairly unlikely that the instructors will have a working Netflow
collector system available for testing. If they do, consider yourself lucky!
Netflow records are exported using UDP to the remote device. The router exports the flow
information in a stream of UDP packets to a device specified as flow collector. If the instructors
have a Netflow collector device in the lab, you can configure your router to export records there. A
sample configuration for this might be:
ip flow-export source loopback 0

ip flow-export version 5 origin-as
ip flow-export destination 192.168.1.5 2055
The first line makes the source address of flow records the IP address of the loopback interface of
the router. The second line specifies version 5 flow records (used by IOS routers) and that the
origin-as should be exported (the other option is the peer-as, which isn’t really much use in an ISP
backbone). The third line specifies that flow records should go to UDP port 2055 (fairly widely
used port for Netflow collectors) and the host 192.168.1.5.
13
If the collector system is running cflowd1 it’s possible, using analysis tools such as Flowscan, to
produce graphs and statistical summaries of what is happening on the ISP’s network. If the lab
instructors have a computer with such software running they will demonstrate the results from the
lab. Otherwise, they will demonstrate typical examples of data analysis from Netflow statistics.
39. Saving configurations on the router. There are a variety of ways of storing router configurations.
In Module One you will have used the command “write mem” or simply “write” to save the
configuration to the router’s non-volatile memory (NVRAM).
However, most ISPs also choose to store their router configurations on a system in the NOC or
engineering departments. There are many reasons, but the major one is to have a back up just in
case the router NVRAM is somehow damaged or deleted. Other reasons include having an audit
trail of changes made to the router configuration, back out in case of configuration error, and so
on.
The lab instructors have set up a TFTP server with an IP address of 192.168.1.4. This system can
be used to store the current router configuration. But first, enter the following configuration
command:
ip tftp source-interface loopback 0
This command tells the router that all tftp packets originating from the router will have source IP
address of the Loopback 0 interface.
40. Saving the configuration. The commands to save the configuration are of the format copy
<source> <destination> where the source and destinations can be any of the following options:
ftp, lex, null, nvram, rcp, running-config, startup-config, system, tftp. To save the configuration to
the TFTP server, use the “copy system:/running-config tftp:” command sequence. If the TFTP
server is unreachable, “.”s followed by an error message will be displayed rather than “!”s. (Note
that the previous “write net” is still supported but may be removed at a future release.)
An example of saving the configuration for Router 1 might be:
Router1#copy system:/running-config tftp:

Address or name of remote host[]? 192.168.1.4
Destination filename [running-config]? router1-confg
!!
2259 bytes copied in 2.920 secs (1129 bytes/sec)
Router1#
Checkpoint #8: Demonstrate to the lab instructors how you saved your router configuration on the
TFTP server. Ask the instructors and assistants to check that your router configuration is on the
server.
1
Cflowd is a software package available from CAIDA (not supported by them any more, but it is still available for
download from www.caida.org). Flowscan is also on their archive. The most common Netflow collection tool used today
is one called nfsen (http://nfsen.sourceforge.net).
14
ISP Workshop Lab
Module 5 – Policy Based Routing

Objective: Using interesting lab exercises, the student will implement some of the fundamental
techniques of Policy Routing, as applied in Cisco IOS.
Prerequisite: Module 1 and the Policy Routing Presentation
REFERENCES
Cisco Systems Documentation CD.
INTRODUCTION
The workshop lab exercises which accompany this module appear after the following discussion and
documentation about Policy Routing. Please read this text first, referring to the provided CD
Documentation in the event of any questions.
WHAT IS POLICY ROUTING?
Policy routing is a more flexible mechanism for routing packets than destination routing. It is a
process whereby the router puts packets through a route map before routing them. The route map
determines which packets are routed to which router next. You might enable policy routing if you
want certain packets to be routed some way other than the obvious shortest path. Some possible
applications for policy routing are to provide equal access, protocol-sensitive routing, source-sensitive
routing, routing based on interactive versus batch traffic, or routing based on dedicated links.
To enable policy routing, you must identify which route map to use for policy routing and create the
route map. The route map itself specifies the match criteria and the resulting action if all of the match
clauses are met. These steps are described in the following three task tables.
To enable policy routing on an interface, indicate which route map the router should use by
performing the following task in interface configuration mode. All packets arriving on the specified
interface will be subject to policy routing. This command disables fast switching of all packets
arriving on this interface.
1. Identify the route map to use for policy routing.

ip policy route-map map-tag
2. Define a route map to control where packets are output.

route-map map-tag[permit | deny] [sequence-number]
3. Define the criteria by which packets are policy routed. The next step is to define the criteria by
which packets are examined to see if they will be policy routed. No match clause in the route map
indicates all packets. Perform one or more of the following tasks in route-map configuration mode:
• Match the Level 3 length of the packet:

match length min max
1
• Match the destination IP address that is permitted by one or more standard or extended
access lists:
match ip address {access-list-number | name} [...access-list-number |
name]
4. Pick Destination Port. The last step is to specify where the packets that pass the match criteria are
output. To do so, perform one or more of the following tasks in route-map configuration mode:
• Specify the next hop to which to route the packet Next-hop must be adjacent, early
documentation is NOT correct on this point.
set ip next-hop ip-address [...ip-address]
• Specify the output interface for the packet.
set interface type number [... type number]
• Specify the next hop to which to route the packet, if there is no explicit route for this
destination.
set ip default next-hop ip-address [... ip-address]
• Specify the output interface for the packet, if there is no explicit route for this destination.
set default interface type number [...type number]
The set commands can be used in conjunction with each other. They are evaluated in the order shown
in the previous task table. A usable next hop implies an interface. Once the local router finds a next
hop and a usable interface, it routes the packet.
Enable Local Policy Routing
Packets that are generated by the router are not normally policy-routed. To enable local policy routing
for such packets, indicate which route map the router should use by performing the following task in
global configuration mode. All packets originating on the router will then be subject to local policy
routing.
Identify the route map to use for local policy routing.
ip local policy route-map map-tag
Use the show ip local policy command to display the route map used for local policy routing, if one
exists.
Caveats
• Minimum version is 11.0(1) – policy routed traffic is process switched, so can be a processor
intensive operation for high data rates.
• Make sure "ip policy route-map" is applied on the INPUT interface.
2
ISP Workshop Lab
• Make sure that IP routing or something similar is used to get the packets back to the source.
Policy routing is a static route to the next hop – the next hop must know how to route the
packet onwards.
• Policy routed traffic can be fast-switched (see below) as from 11.3 and 12.0 software releases.
Note that ip route-cache policy is off by default and needs to be applied to the input interface.
• No IP options work with policy routing, e.g. IP record route ...
Fast-Switched Policy Routing
IP policy routing can now be fast-switched. Prior to this feature, policy routing could only be process
switched, which meant that on most platforms, the switching rate was approximately 1,000 to 10,000
packets per second. This was not fast enough for many applications or the size of trunks being used.
Users who need policy routing to occur at faster speeds can now implement policy routing without
slowing down the router.
Restrictions
Fast-switched policy routing supports all of the match commands and most of the set commands,
except for the following restrictions:
• The set ip default command is not supported.
• The set interface command is supported only over point-to-point links, unless a route-cache
entry exists using the same interface specified in the set interface command in the route map.
Also, at process level, the routing table is consulted to determine if the interface is on a
reasonable path to the destination. During fast switching, the software does not make this
check. Instead, if the packet matches, the software blindly forwards the packet to the specified
interface.
Configuration Example
Fast switching of policy routing is disabled by default. The following example enables policy
routing to be fast switched on an ethernet interface.

ip route-cache policy
LIST OF RELATED COMMANDS
• ip policy route-map
• ip local policy
• match ip address
• match length
• route-map
• set default interface
• set interface
• set ip default next-hop
• set ip next-hop
• set ip precedence (undocumented as of 11.2(7)P )
• set ip tos (undocumented as of 11.2(7)P )
• ip route-cache policy
• show ip cache policy
3
LABS & EXERCISES
1. Check Physical Connectivity. The connectivity for this workshop should be as in Figure 1.
Ensure that all physical connections are complete and pingable. The addresses used for links
between routers should be left the same as those chosen for the previous Modules.
Figure 1 – Network Configuration

2. Clean up from past labs. Delete any remaining OSPF, BGP configuration, route maps,
communities, access-lists and AS-Path access-lists (if configured).
4
ISP Workshop Lab
3. Create OSPF PID 41, area 0 on all routers. All Router Teams will configure one OSPF domain
– Area 0 – for the network. This configuration should be as was used in Module 1 checkpoint #1.
4. Ping Test. The system specified by the instructors (usually Router15 or the time server
ntp0.workshop.net) will be used for the target of this lab. All routers should be able to ping and
traceroute to the Time Server. Capture the trace to the Time Server. This will be used as a baseline.
Checkpoint #1: Call the workshop instructor, explain the OSPF configuration, and demonstrate the
connectivity to the Time Server.
5. Policy Assignment – Create a Ping/Traceroute Ring. Using policy routing, configure interfaces
on the entire network so that a traceroute from Router6 to the Time Server would take the
following path (see Figure 2):
Router6 → Router8 → Router10 → Router12 → Router14 → Router13 → Router11 → Router9 → Router7 →

Router5 → Router3 → Router1 → Router2 → Router4 → Router6 → Router15/TimeSource
6. Approaches to applying policy routing. Policy-Based Routing is as dangerous as static routes.

Hence, you need to approach the application of a Policy Route very methodically. OSPF must be
configured properly before any of consideration is made to implement policy routing. The first
step is to change the characteristic of the trace leaving the router. If the trace to the Time Server
originally went out one port, change it to go out another. To accomplish this you use the ip local
policy route-map command (see explanation above).
Hint: Work out which interface is the outbound interface and which is inbound, and where the
policy configuration needs to be applied. Think before typing in configuration.
Example:
! destination address to match against
!
access-list 100 permit ip any host 192.168.1.1
!
! configuration to determine policy for packets generated on router
!
route-map redirect-local-traffic permit 10
match ip address 100
set interface serial 0/0
!
ip local policy route-map redirect-1ocal-traffic
!
! configuration to determine policy for incoming packets
!
route-map redirect-incoming-traffic permit 10
match ip address 100
set interface serial 0/0
!
ip policy route-map redirect-incoming-traffic
HINT: For two routers that interconnect via serial links – the set interface command may be
easier. Set interface will not work for broadcast interconnects like ethernet.
5
Figure 2 – Traceroute Ring from Router 6 to Time Server (Router15)
Question: What should be used in lieu of set interface on broadcast mediums like ethernet?
Answer: Use “set ip next-hop <ip address>”.
6
ISP Workshop Lab
Question: What differences are there between the configuration used for Router 6 and the remaining
routers in the workshop? Why?
Answer: Remember that the trace from the console has to go round the class, not direct to the Time
Server…
Q. Will Policy Routing drop my packets if I set in on a router's interface, but have no set next-hop
configured?
A. Policy Routing will first look for another set command first. If there is no other set command
configured or if the set command doesn't make sense, then the router falls back to the normal routing
(destination-based).
Checkpoint #2: The lab instructor will now demonstrate a trace from Router6 to the Time Server (or
substitute device). If the policy routing has been set up successfully, a trace will follow the paths
shown in Figure 2.
Q. What happens if the connection between one of the routers in the path of the policy routed
connection to the Time Server is broken? Why?
Checkpoint #3: The lab instructor will now break the connection between Router4 and Router6.
Observe what happens to the paths followed now.
A. If the policy destination is down, the router reverts to using the routing table for its forwarding
decisions.
7
ISP Workshop Lab
Module 6 – More iBGP, and Basic eBGP Configuration

Objective: Simulate four different interconnected ISP backbones using a combination of ISIS,
internal BGP, and external BGP.
Prerequisites: Module 1 (ISIS)
Topology :
Figure 1 – BGP AS Numbers
1
Saturday, August 16, 2014
Lab Notes
The purpose of this module is to introduce the student to external BGP (eBGP). This is the
relationship between different autonomous systems in an “Internet”. The classroom is split into four
distinct networks, and the teams belonging to each network work together as a typical ISP. Each AS
has two links to its neighbouring ASes, and this feature will used throughout a significant portion of
this workshop.
The connectivity shown in the diagrams represents links between AS’s. It is assumed that all the
routers within an AS are physically connected to each other as per Figure 1.
Lab Exercises
1. Connect routers as shown in Figure 1. All routers within an AS must be physically connected and
reachable. The relationship between the ASes is as drawn in Figure 2 and gives a view which can
be related to the “real world”.
Figure 2 – AS relationship
2. Remove the IP addressing. Before we proceed to even think about configuring the routing
protocols to match the above AS layout, we need to tidy up the addressing from previous
modules. In this step we remove the IP addresses from all physical interfaces, and the loopback
interface. This effectively takes the lab back to the configuration it had prior to step 10 in Module
1. Don’t forget to remove all the IP addressing.
(The alternative is to simply erase the entire router configuration using write erase, and then
reload the router and start again, completing all steps of Module 1 up to Step 9.)
3. Re-configure BGP and ISIS. On each router, remove the BGP and ISIS processes from Module 1
by using the following two commands:
2
ISP Workshop Lab
Router1 (config)# no router bgp 10

Router1 (config)# no router isis workshop
This will clear the BGP and ISIS configuration for the current module.
4. IP Addressing. As we did in step 10 of Module 1, we need to come up with a sensible and

scalable addressing plan for each AS in this network. Each AS gets their own address block, again
a /20 (typical minimum allocation for a starter ISP). This address block should be assigned to
links and loopbacks on the routers making up each ASN. The allocations are as follows:
AS10 10.10.0.0/20 AS30 10.30.0.0/20

AS20 10.20.0.0/20 AS40 10.40.0.0/20
Again we need to divide up each address block so that we have customer address space, network
infrastructure address space, and some space for loopbacks. Figure 3 below reminds how this
could be done:
10.10.0.0 10.10.14.255 10.10.15.0 10.10.15.223 10.10.15.224 10.10.15.255
Please refer to the accompanying hand out for the address plan which should be used for this
module onwards – it is entitled “Addressing Plan – Modules 6 to 9”. As with Module 1, configure
the addresses on each interface which will be used for this module, and check basic IP connectivity
with your immediately adjacent neighbours.
5. Router Loopback Interface Addressing. We have set aside a /27 for loopbacks even though each
AS has either 3 or 4 routers in it – this leaves more than sufficient room for future expansion. The
loopback address assignments which will be used for this module are below:
Router1 10.10.15.224 Router8 10.30.15.224

Router2 10.10.15.225 Router9 10.30.15.225
Router3 10.10.15.226 Router10 10.30.15.226
Router4 10.20.15.224 Router11 10.40.15.224
Router5 10.20.15.225 Router12 10.40.15.225
Router6 10.20.15.226 Router13 10.40.15.226
Router7 10.20.15.227 Router14 10.40.15.227
6. Configure ISIS on the routers within each AS. In each AS configure ISIS routing. This means
that each router team should configure router ISIS with ISIS ID asY on the router, where Y is the
AS number. And the internal links to each member of the AS must be configured with ip router
ISIS asY. The NET should be 49.0001.x.x.x.00, where x.x.x is built from the loopback IP address
(refer to Module 1 for details).
ISIS should be configured on internal interfaces only. You do not want to set up adjacencies with
devices outside your AS. Make sure that there are no ip router isis commands on external
3
interfaces. A side effect of this is that external link addresses will not appear in the IGP (see the
next section discussion iBGP deployment).
As an example, Router 1, which has two interfaces inside AS 10, would have the following
configuration:
Router1 (config)# router isis as10

Router1 (config-router)# net 49.0001.0100.1001.5224.00
Router1 (config-router)# is-type level-2-only
Router1 (config-router)# metric-style wide level-2
Router1 (config-router)# log-adjacency-changes
Router1 (config-router)# set-overload-bit on-startup wait-for-bgp
!
Router1 (config)# interface fastethernet 0/0
Router1 (config-if)# ip router isis as10
Router1 (config-if)# isis metric 2 level-2
Router1 (config-if)# isis circuit-type level-2-only
!
Router1 (config)# interface serial 1/0
Router1 (config-if)# ip router isis as10
Router1 (config-if)# isis metric 20 level-2
Router1 (config-if)# isis circuit-type level-2-only
Note:
• Different ISPs use different NET addressing schemes. But it is common to use the router
loopback IP address as the system ID in either hex or decimal format. In this module all
routers in an AS are level-2 in and one area (49.0001) only.
7. Passive Interfaces in ISIS. Now mark the interfaces on which you do not want to run ISIS as
passive. For ISIS, marking an interface as passive means that CLNS adjacencies are not solicited
and the IP subnet used on the interface is inserted into ISIS. For this lab we are only going to mark
the Loopback interface as passive. Note that you cannot mark interfaces as passive until you have
ISIS assigned to at least one physical interface on the router (as you did in Step 6). The example
for Router1 is below:
Router1 (config)# router isis as10

Router1 (config-router)# passive-interface Loopback0
Interface Rules:
1. “ip router isis” on an interface means that CLNS adjacencies are sought and the IP
subnet used on the interface is inserted into ISIS.
2. “passive interface” in the ISIS process for an interface on the router means that no
CLNS adjacencies are sought, but the IP subnet used is inserted into ISIS.
3. No ISIS configuration for the interface means that no CLNS adjacencies are sought, and no IP
subnet used on the interface is put into ISIS.
Note:
• ISIS by default will only set up adjacencies and announce the prefixes of the interfaces which
are activated by the “ip router isis” command. This is different behaviour from OSPF
which will attempt to set up adjacencies on interfaces covered by the network statement (and
hence require the use of passive and no passive to control its behaviour).
4
ISP Workshop Lab
8. ISIS on Point-to-Point Ethernet Links. One feature mentioned in the ISIS presentations was the
option to modify ISIS’s behaviour on point-to-point broadcast media links, such as Ethernet, when
there are only two devices on that media. If we declare such a situation to be point to point, then
ISIS does not try and determine a designated router; furthermore, there will be an
improvement/simplification in SPF calculations and memory requirements on the router.
Those router teams who have ISIS configured on an internal Ethernet interface should now
convert ISIS to point-to-point mode, for example:

Router1 (config-interface)# isis network point-to-point
The link is now treated like a point-to-point serial connection. Note that there is no need to
configure point-to-point mode on an Ethernet interface which is not running ISIS.
9. Ping Test. Check the routes via ISIS. Make sure you can see all the networks within your AS, and
see no networks from other ASs. Ping all loopback interfaces within your AS Set. Use the “show
clns neighbor” and “show ip route” commands.
10. Telnet source address. Most ISPs use the router Loopback address for administrative purposes as
well as the anchor point for their network’s iBGP sessions. In this step we will configure telnet so
that it uses the loopback interface as the source address for all telnet packets originated by the
router.
Router3>who
* 2 vty 0 isplab idle 00:00:00 10.10.15.224
11. Save the configuration. Don’t forget to save the configuration to NVRAM!
Checkpoint #1 : call the lab assistant to verify the connectivity.
12. Turning on neighbour authentication for ISIS – Part 1. ISIS supports neighbour authentication;
this is considered more and more important inside ISP networks as attacks on infrastructure
increase and ISPs seek to use all available tools to secure their networks. (While an attack on ISIS
is harder as it runs on the link layer alongside IP rather than on top of IP like OSPF, some ISPs are
still prudent and implement neighbour authentication.)
Each router team will now turn on neighbour authentication for ISIS. The first step is to set up the
keychain to be used – we will use the key “cisco” for this lab:
Router1(config)# key chain lab-key
Router1(config-keychain)# key 1
Router1(config-keychain-key)# key-string cisco
5
13. Turning on neighbour authentication for ISIS – Part 2. Now that the keychain has been
defined, we activate authentication within the ISIS processes. The first step is to enable MD5 for
level-2 IS’s:

Router1(config-router)# authentication mode md5 level-2
And then associate the key-chain we defined earlier with the configured authentication:
Router1(config-router)# authentication key-chain lab-key level-2
Notice now that the ISIS adjacencies do not come up unless the neighbouring router has also
entered the same configuration and key. Notice also how the ISIS adjacencies were reset as the
14. Final check. Use the various “show isis” commands to see the ISIS status of the lab network now.
Check the routing and the routing table. Make sure all the adjacencies have come back up again. If
any adjacency has failed to come up, and you see several log messages saying:
*Mar 1 00:05:17.825: %CLNS-4-AUTH_FAIL: ISIS: LAN IIH authentication failed
you should reasonably expect that either you or your connected neighbour have forgotten to set up
neighbour authentication.
Note: Wherever an ISIS session is configured from now on in the workshop, all Router
Teams MUST use passwords on these ISIS sessions.
15. Configure iBGP peering between routers within an AS. Use the loopback address for the iBGP
peerings. Also, configure the network command to add the address block assigned to each Router
Team for advertisement in BGP.

Router1 (config-router)# distance bgp 200 200 200
Router1 (config-router)# neighbor 10.10.15.225 remote-as 10
Router1 (config-router)# neighbor 10.10.15.225 update-source loopback 0
Router1 (config-router)# neighbor 10.10.15.225 next-hop-self
Router1 (config-router)# neighbor 10.10.15.225 description iBGP Link to R2
Router1 (config-router)# exit
16. Next-hop-self configuration. The previous step introduced the next-hop-self configuration
command. Referring to the BGP presentation, the next-hop-self configuration makes the iBGP
speaking router use the iBGP source address (in this case the loopback) rather than the external
6
ISP Workshop Lab
next-hop address (as per the BGP specification). This is industry best practice and means that ISPs
do not need to carry external next-hops in their IGP.
17. Test internal BGP connectivity. Use the BGP Show commands to ensure you are receiving
everyone's routes from within your AS.
18. Configure passwords on the iBGP sessions. Passwords should now be configured on the iBGP
sessions. Review the presentation why this is necessary. Agree amongst all your team members in
your AS what the password should be on the iBGP session, and then apply it to all the iBGP
peerings on your router. For example, on Router2’s peering with Router3, with “cisco” used as the
password:

Router2 (config-router)# neighbor 10.10.15.226 password cisco
IOS currently resets the iBGP session between you and your neighbouring router whenever an
MD5 password is added. So when passwords are added to BGP sessions on live operational
networks, this work should be done during a maintenance period when customers know to expect
disruptions to service. In the workshop lab, it doesn’t matter so much. (Future IOS releases will
avoid having this rather serious service disruption.)
Watch the router logs – with the BGP session neighbour changes being logged, any mismatch in
the password should be easy to spot. A missing password on one side of the BGP session will
result in the neighbouring router producing these errors:
%TCP-6-BADAUTH: No MD5 digest from 3.3.3.3:179 to 2.2.2.2:11272

whereas a mismatch in the configured passwords will result in these messages:
%TCP-6-BADAUTH: Invalid MD5 digest from 3.3.3.3:11024 to 2.2.2.2:179

Checkpoint #3: Call the lab assistant and demonstrate the password as set on the iBGP session.
Once confirmed by the lab assistant, move on to the next steps.
19. Configure eBGP peering. Use Figure 1 to determine the links between the AS’s. Addressing for
eBGP links between 2 AS’s will use the point-to-point interface addresses, NOT the loopback
addresses (review the BGP presentation if you don’t understand why).

Router1 (config-router)# neighbor 10.10.15.14 description eBGP to Router13
Use the BGP Show commands to ensure you are sending and receiving the BGP advertisements
from your eBGP neighbours.
Q. Why can’t the loopback interfaces be used for the eBGP peerings?
7
A. The IP address of a router’s loopback interface is not known to external BGP peers, so the
external peers will have no way of knowing how to contact each other to establish the peering.
Q. Which BGP show command allows you to see the state of the BGP connection to your
peer?
A. Try show ip bgp neighbor x.x.x.x – this will give detailed information about the state of the
peer. There are subcommands of this one, giving more information about the peering.
Q. Which BGP Show command will allow you to see exactly which networks you are
advertising and receiving from your eBGP peers?
A. Try show ip bgp neighbor x.x.x.x route – this will show which routes you are receiving from
your peer. Likewise, replacing route with advertised-routes will list the networks which are
being announced to your peer. (Note that in general ISP operational practice, there are caveats
here – if you apply route-maps and some BGP policies, these will not be processed by the
advertised-routes command. Use the advertised-routes subcommand with due caution.)
20. Configure passwords on the eBGP session. Passwords should now be configured on the eBGP
sessions between your and your neighbouring ASes. Agree between you and your neighbouring
AS what the password should be on the eBGP session, and then apply it to the eBGP peering. For
example, on Router2’s peering with Router4, with “cisco” used as the password:

As previously for the iBGP session, watch the logs for password mismatches, or missing
passwords. As with the iBGP sessions previously, you will find that the router will reset the eBGP
session as soon as the password is applied.
Note: Wherever a BGP (either iBGP or eBGP) session is configured from now on in the
workshop, all Router Teams MUST use passwords on these BGP sessions.
Checkpoint #4: Call the lab assistant and demonstrate the password as set on the eBGP session.
21. Adding a “customer” route into BGP. As we did in Module 1, we are now going to add a
“customer” route into BGP on each router. We don’t have any “customers” as such connected to
our routers in the lab, so we are going to simulate the connectivity by simply using a Null0
interface. The “customer” address space that each router team will introduce into the iBGP is listed
below – again we will each use a /26, for simplicity’s sake.
R1 10.10.0.0/26 R8 10.30.0.0/26
R2 10.10.0.64/26 R9 10.30.0.64/26
R3 10.10.0.128/26 R10 10.30.0.128/26
R4 10.20.0.0/26 R11 10.40.0.0/26
R5 10.20.0.64/26 R12 10.40.0.64/26
R6 10.20.0.128/26 R13 10.40.0.128/26
R7 10.20.0.192/26 R14 10.40.0.192/26
8
ISP Workshop Lab

the class has done their configuration, each team should see the aggregate from each AS as well as
the fourteen /26s introduced in the previous step. If this is not happening, work with your
neighbours to fix the problem.
Checkpoint #5: Call the lab assistant to verify the connectivity. Use commands such as “show ip route
sum”, “show ip bgp sum”, “show ip bgp”, “show ip route”, and “show ip bgp neigh x.x.x.x route |
advertise”. There should be 4 aggregate prefixes (one for each ISP) and the 14 customer /26’s in the
BGP table.
23. The Importance of Aggregation. Each AS was allocated a /20 address block. It is expected by all
Internet operators that any address space an ISP is using is aggregated as much as possible before
it is announced to the rest of the Internet. Subdividing the address space inside an AS is perfectly
acceptable and obviously very common (as we have done here) – but most operators consider
leaking this subdivided address space out to the Internet at large antisocial and unfriendly.
Q. How do you automatically aggregate via BGP smaller address blocks from within your
network to a larger address block outside your network? Hint: Review the BGP documentation.
A. The “aggregate-address” command is quite often used to achieve this.
We are not doing any filtering or limitation of the announcements of the “customer” address
blocks we have introduced into each ASN. This will be one of the goals of the next modules in the
workshop.
9
Figure 4 – Aggregates for each ASN
24. BGP Update Activity (Optional). Use debug ip bgp update to see BGP update activity after
clearing a BGP session. To stop the debug running, do undebug ip bgp update.
Warning: it might not be such a good idea to run this debug command on a router receiving the
full Internet routing table; using this command in a lab network such as this might show you why!
Review Questions
1. How many origin types exist in BGP?
2. List the origin types. Hint: Review the BGP presentations.
3. How are they used?
4. Why are passwords necessary on both iBGP and eBGP sessions? What do they protect against?
5. Why is aggregation important for the Internet?
10
ISP Workshop Lab
Module 6 – More iBGP, and Basic eBGP Configuration

Objective: Simulate four different interconnected ISP backbones using a combination of OSPF,
internal BGP, and external BGP.
Prerequisites: Module 1
Topology :
1
Friday, August 02, 2013
Lab Notes
The purpose of this module is to introduce the student to external BGP (eBGP). This is the
relationship between different autonomous systems in an “Internet”. The classroom is split into four
distinct networks, and the teams belonging to each network work together as a typical ISP. Each AS
has two links to its neighbouring ASes, and this feature will used throughout a significant portion of
this workshop.
The connectivity shown in the diagrams represents links between AS’s. It is assumed that all the
routers within an AS are physically connected to each other as per Figure 1.
Lab Exercises
1. Connect routers as shown in Figure 1. All routers within an AS must be physically connected and
reachable. The relationship between the ASes is as drawn in Figure 2 and gives a view which can
be related to the “real world”.
Figure 2 – AS relationship
2. Re-configure BGP and OSPF. On each router, remove the BGP and OSPF processes from the
previous modules by using the following two commands:
Router1(config)# no router bgp 10

Router1(config)# no router ospf 41
This will clear the BGP and OSPF configuration ready for the current module.
3. Remove the IP addressing. Before we proceed to even think about configuring the routing
protocols to match the above AS layout, we need to tidy up the addressing from previous
modules. In this step we remove the IP addresses from all physical interfaces, and the loopback
2
ISP Workshop Lab
interface. This effectively takes the lab back to the configuration it had prior to step 10 in Module
1. Don’t forget to remove all the IP addressing.
4. IP Addressing. As we did in step 10 of Module 1, we need to come up with a sensible and

scalable addressing plan for each AS in this network. Each AS gets their own address block, again
a /20 (typical minimum allocation for a starter ISP). This address block should be assigned to
links and loopbacks on the routers making up each ASN. The allocations are as follows:
AS10 10.10.0.0/20 AS30 10.30.0.0/20

AS20 10.20.0.0/20 AS40 10.40.0.0/20
Again we need to divide up each address block so that we have customer address space, network
infrastructure address space, and some space for loopbacks. Figure 3 below reminds how this
could be done:
10.10.0.0 10.10.14.255 10.10.15.0 10.10.15.223 10.10.15.224 10.10.15.255
Please refer to the accompanying hand out for the address plan which should be used for this
module onwards – it is entitled “Addressing Plan – Modules 6 to 9”. As with Module 1, configure
the addresses on each interface which will be used for this module, and check basic IP connectivity
with your immediately adjacent neighbours.
5. Router Loopback Interface Addressing. We have set aside a /27 for loopbacks even though each
AS has either 3 or 4 routers in it – this leaves more than sufficient room for future expansion. The
loopback address assignments which will be used for this module are below:
Router1 10.10.15.224 Router8 10.30.15.224

Router2 10.10.15.225 Router9 10.30.15.225
Router3 10.10.15.226 Router10 10.30.15.226
Router4 10.20.15.224 Router11 10.40.15.224
Router5 10.20.15.225 Router12 10.40.15.225
Router6 10.20.15.226 Router13 10.40.15.226
Router7 10.20.15.227 Router14 10.40.15.227
6. Configure OSPF on the routers within each AS. In each AS configure OSPF routing. This
means setting up the OSPF process, marking internal interfaces as non-passive, and then
configuring each internal interface and the loopback with the OSPF process. All the routers in an
AS will be in the same OSPF area 0 and use the same OSPF process ID.
OSPF should be configured on internal interfaces only. You do not want to set up adjacencies with
devices outside your AS. Make sure that there are no ip ospf commands on external interfaces. A
side effect of this is that external link addresses will not appear in the IGP (see the next section
discussion iBGP deployment).
3
For example, Router 1, with two interfaces connecting to other routers in their AS, would have the
following:
Router1 (config)# router ospf 10

Router1 (config-router)# passive-interface default
Router1 (config-router)# no passive-interface fastethernet 0/0
Router1 (config-router)# no passive-interface serial 1/0
Router1 (config-router)# log-adjacency-changes
!
Router1 (config-interface)# ip ospf 10 area 0
!
Router1 (config-interface)# interface serial 1/0
!
Router1 (config-interface)# interface loopback 0
!
Notes:
• Passive-interface default makes sure that OSPF does not attempt to set up adjacencies on any
interfaces apart from those specified by the no passive-interface commands.
• The number following “router ospf” is a process ID and is used inside the router only (so it can
be any number). But for this lab we recommend the OSPF process ID be the same as the AS
number (which is the convention used by a number of ISPs).
7. OSPF on Point-to-Point Ethernet Links. One feature mentioned in the OSPF for ISPs
presentations was the option to modify OSPF’s behaviour on point-to-point broadcast media links,
such as Ethernet, when there are only two devices on that media. If we declare such a situation to
be point to point, then OSPF does not try and determine a designated or backup designated router;
furthermore, there will be an improvement/simplification in SPF calculations and memory
requirements on the router.
Those router teams which have OSPF configured over an Ethernet interface should now convert
OSPF to point to point mode, for example:

Router1 (config-interface)# ip ospf network point-to-point
The result will be that the DR or BDR entry in the status column of “show ip ospf neighbor” will
disappear, to be replaced with a FULL. The link is now treated like a point-to-point serial
connection.
8. Ping Test. Check the routes via OSPF. Make sure you can see all the networks within your AS.
Ping all loopback interfaces within your AS. Use the “show ip ospf neighbor” and “show ip route”
commands. If you cannot see the other routers in your AS, you will not be able to bring up BGP in
the next steps.
9. Telnet source address. Most ISPs use the router Loopback address for administrative purposes as
well as the anchor point for their network’s iBGP sessions. In this step we will configure telnet so
that it uses the loopback interface as the source address for all telnet packets originated by the
router.
4
ISP Workshop Lab
Router3>who
* 2 vty 0 isplab idle 00:00:00 10.10.15.224
10. Save the configuration. Don’t forget to save the configuration to NVRAM!
11. Intra Area Authentication – Part 1. OSPF supports neighbour authentication. This is quite
important inside ISP networks to prevent the introduction of improperly configured or unintended
equipment.
Each router team will turn on authentication. This first step will enable the area to support
authentication using the area N authentication message-digest command. As soon as this is done,
all adjacencies will be expected to use neighbour authentication.
Router6(config-router)# area 0 authentication message-digest
12. Intra Area Authentication – Part 2. Now that support for authentication in each area has been

Router6(config-if)# ip ospf message-digest-key 1 md5 cisco
!
Router6(config-if)# interface serial 1/1
Router6(config-if)# ip ospf message-digest-key 1 md5 cisco
5
13. Final check. Use the various “show ip ospf” commands to see the OSPF status of the lab network
now. Check the routing and the routing table. If you are missing any adjacencies, work with your
neighbouring routers in your AS to work out why, and what might have gone wrong with the
neighbour authentication.
Note: Wherever an OSPF session is configured from now on in the workshop, all Router
Teams MUST use passwords on these OSPF sessions.
14. Configure iBGP peering between routers within an AS. Use the loopback address for the iBGP
peerings. Also, configure the network command to add the address block assigned to each AS for
advertisement in BGP. Each router team should announce this address block from their routers.

Router1 (config-router)# distance bgp 200 200 200
Router1 (config-router)# exit
15. Next-hop-self configuration. The previous step introduced the next-hop-self configuration
command. Referring to the BGP presentation, the next-hop-self configuration makes the iBGP
speaking router use the iBGP source address (in this case the loopback) rather than the external
next-hop address (as per the BGP specification). This is industry best practice and means that ISPs
do not need to carry external next-hops in their IGP.
16. Test internal BGP connectivity. Use the BGP Show commands to ensure you are receiving
everyone's routes from within your AS.
17. Configure passwords on the iBGP sessions. Passwords should now be configured on the iBGP
sessions. Review the presentation why this is necessary. Agree amongst all your team members in
your AS what the password should be on the iBGP session, and then apply it to all the iBGP
peerings on your router. For example, on Router2’s peering with Router3, with “cisco” used as the
password:

6
ISP Workshop Lab
IOS currently resets the iBGP session between you and your neighbouring router whenever an
MD5 password is added. So when passwords are added to BGP sessions on live operational
networks, this work should be done during a maintenance period when customers know to expect
disruptions to service. In the workshop lab, it doesn’t matter so much. (Future IOS releases will
avoid having this rather serious service disruption.)
Watch the router logs – with the BGP session neighbour changes being logged, any mismatch in
the password should be easy to spot. A missing password on one side of the BGP session will
result in the neighbouring router producing these errors:

whereas a mismatch in the configured passwords will result in these messages:

Checkpoint #3: Call the lab assistant and demonstrate the password as set on the iBGP session.
18. Configure eBGP peering. Use Figure 1 to determine the links between the AS’s. Addressing for
eBGP links between 2 AS’s will use the point-to-point interface addresses, NOT the loopback
addresses (review the BGP presentation if you don’t understand why).

Router1 (config-router)# neighbor 10.10.15.14 description eBGP to Router13
Use the BGP Show commands to ensure you are sending and receiving the BGP advertisements
from your eBGP neighbours.
Q. Why can’t the loopback interfaces be used for the eBGP peerings?
A. The IP address of a router’s loopback interface is not known to external BGP peers, so the
external peers will have no way of knowing how to contact each other to establish the peering.
Q. Which BGP show command allows you to see the state of the BGP connection to your
peer?
A. Try show ip bgp neighbor x.x.x.x – this will give detailed information about the state of the
peer. There are subcommands of this one, giving more information about the peering.
Q. Which BGP Show command will allow you to see exactly which networks you are
advertising and receiving from your eBGP peers?
A. Try show ip bgp neighbor x.x.x.x route – this will show which routes you are receiving from
your peer. Likewise, replacing route with advertised-routes will list the networks which are
7
being announced to your peer. (Note that in general ISP operational practice, there are caveats
here – if you apply route-maps and some BGP policies, these will not be processed by the
advertised-routes command. Use the advertised-routes subcommand with due caution.)
19. Configure passwords on the eBGP session. Passwords should now be configured on the eBGP
sessions between your and your neighbouring ASes. Agree between you and your neighbouring
AS what the password should be on the eBGP session, and then apply it to the eBGP peering. For
example, on Router2’s peering with Router4, with “cisco” used as the password:

As previously for the iBGP session, watch the logs for password mismatches, or missing
passwords. As with the iBGP sessions previously, you will find that the router will reset the eBGP
session as soon as the password is applied.
Note: Wherever a BGP (either iBGP or eBGP) session is configured from now on in the
workshop, all Router Teams MUST use passwords on these BGP sessions.
Checkpoint #4: Call the lab assistant and demonstrate the password as set on the eBGP session.
20. Adding a “customer” route into BGP. As we did in Module 1, we are now going to add a
“customer” route into BGP on each router. We don’t have any “customers” as such connected to
our routers in the lab, so we are going to simulate the connectivity by simply using a Null0
interface. The “customer” address space that each router team will introduce into the iBGP is listed
below – again we will each use a /26, for simplicity’s sake.
R1 10.10.0.0/26 R8 10.30.0.0/26
R2 10.10.0.64/26 R9 10.30.0.64/26
R3 10.10.0.128/26 R10 10.30.0.128/26
R4 10.20.0.0/26 R11 10.40.0.0/26
R5 10.20.0.64/26 R12 10.40.0.64/26
R6 10.20.0.128/26 R13 10.40.0.128/26
R7 10.20.0.192/26 R14 10.40.0.192/26

the class has done their configuration, each team should see the aggregate from each AS as well as
the fourteen /26s introduced in the previous step. If this is not happening, work with your
neighbours to fix the problem.
8
ISP Workshop Lab
Checkpoint #5: Call the lab assistant to verify the connectivity. Use commands such as “show ip route
sum”, “show ip bgp sum”, “show ip bgp”, “show ip route”, and “show ip bgp neigh x.x.x.x route |
advertise”. There should be 4 aggregate prefixes (one for each ISP) and the 14 customer /26’s in the
BGP table.
22. The Importance of Aggregation. Each AS was allocated a /20 address block. It is expected by all
Internet operators that any address space an ISP is using is aggregated as much as possible before
it is announced to the rest of the Internet. Subdividing the address space inside an AS is perfectly
acceptable and obviously very common (as we have done here) – but most operators consider
leaking this subdivided address space out to the Internet at large as antisocial and unfriendly
behaviour.
Q. How do you automatically aggregate via BGP smaller address blocks from within your
network to a larger address block outside your network? Hint: Review the BGP documentation.
A. The “aggregate-address” command is quite often used to achieve this.
We are not doing any filtering or limitation of the announcements of the “customer” address
blocks we have introduced into each ASN. This will be one of the goals of the next modules in the
workshop.
Figure 4 – Aggregates for each ASN
23. BGP Update Activity (Optional). Use debug ip bgp update to see BGP update activity after
clearing a BGP session. To stop the debug running, do undebug ip bgp update.
Warning: it might not be such a good idea to run this debug command on a router receiving the
full Internet routing table; using this command in a lab network such as this might show you why!
9
Review Questions
1. How many origin types exist in BGP?
2. List the origin types. Hint: Review the BGP presentations.
3. How are they used?
4. Why are passwords necessary on both iBGP and eBGP sessions? What do they protect against?
5. Why is aggregation important for the Internet?
10
ISP Workshop Lab
Module 7 – BGP route filtering and advanced features

Objective: Using the network configured in Module 6, use various configuration methods on
BGP peerings to demonstrate neighbour filtering and more advanced IOS features.
Prerequisite: Module 6
Topology:
1
Wednesday, January 29, 2014
Lab Notes
The previous Module provided an introduction to setting up external BGP, but provided no way of
controlling which networks are announced to which AS. The purpose of this module is to introduce
the student to the types of routing policy which are available using BGP.
In steps 2 to 6 we will configure each AS to become a non-transit AS, i.e. the AS won’t allow a
connecting AS’s traffic to traverse it to reach another connecting AS. This will mean disconnectivity
in the classroom – for example, AS3 will no longer be able to see any networks from AS10, etc. This
is a deliberate policy to demonstrate the effectiveness of the BGP filtering being employed.
In step 2 we achieve this by configuring an outgoing route filter to allow only local prefix(es) be sent
to eBGP peers. At the same time, we also make sure that our peers only send us their own prefixes.
We guarantee this by configuring an incoming filter. In general, it is good practice to configure filters
in both directions to protect against misconfiguration at both ends of the peering. In step 4 we use AS-
PATH filters. And in step 6 we make use of BGP communities to achieve the same effect.
Important: each step must be carried out and completed by the whole workshop before the next step
can be started. Do not immediately start the next step without receiving the go-ahead from the
workshop instructors. If you do, it is likely the routing will break, and other router teams may be
unable to understand the results of the configuration they are trying to implement.
Important: retain the configuration used for Module 6.
Lab Exercise
1. Implement BGP policies. Before doing any configurations in this module it is important to note
how to go about implementing BGP policies. Entering prefix lists, as-path filters or route-maps
can be done at the CLI, but they only apply to BGP updates received after the policy configuration
has been entered at the router. This is because BGP sends incremental updates describing changes
in routes announced or withdrawn. To apply the policy to the entire BGP routing table received
from or sent to the peer, the BGP session needs to be “reset”. In older versions of IOS, this literally
meant tearing down the BGP session, and then restoring it. However, as can be imagined, this
causes severe stability issues on the service provider network, and ‘RFC2918: Route Refresh
Capability’ has been added to most modern BGP implementations to allow graceful updates to
BGP sessions when policy changes are required.
To implement policy changes in any of the following worked examples, use the following router
commands, for example to implement new policy inbound and outbound on the peering between
Router 1 and Router 13:
Router1# clear ip bgp 10.10.15.14 out

Router1# clear ip bgp 10.10.15.14 in
Note 1: Do not forget the in and out subcommands in the above clear commands – omitting them
will implement a hard reset of the BGP session. Review the BGP presentation if you don’t
understand why you do not want to ever do a hard reset on a BGP session.
2
ISP Workshop Lab
Note 2: Rather than refreshing the BGP session using the neighbour IP address, it is also possible
to refer to the neighbour by its ASN. Since Router 13 is in AS40, the alternative commands would
be:
Router1# clear ip bgp 40 out

Router1# clear ip bgp 40 in
Checkpoint #1: call the lab assistant to verify the connectivity. Each router team should check
peerings to see the effect of this step. Use the “show ip bgp” command to show the BGP table –
ensure that the external prefixes learned by BGP now have a local next-hop address. Use the “trace”
command to show that network connectivity is unaffected.
Filtering using prefix-lists
2. Configure prefix filter based on network address: This step configures route prefix filtering
based on network address. This is done using prefix-lists, and is one method of controlling
networks which are exchanged in BGP peerings. The aim here is to configure eBGP peerings so
that only networks from neighbouring ASes are exchanged.
Example: Router R13 (peering with R1)
!
ip prefix-list out-peer permit 10.40.0.0/20
ip prefix-list out-peer deny 0.0.0.0/0 le 32
!
ip prefix-list in-peer permit 10.10.0.0/20
ip prefix-list in-peer deny 0.0.0.0/0 le 32
!
router bgp 40
no synchronization
network 10.40.0.0 mask 255.255.240.0
neighbor 10.10.15.13 description eBGP peering with Router1
neighbor 10.10.15.13 prefix-list out-peer out
neighbor 10.10.15.13 prefix-list in-peer in
no auto-summary
!
!
ip prefix-list out-peer permit 10.10.0.0/20
ip prefix-list out-peer deny 0.0.0.0/0 le 32
!
ip prefix-list in-peer permit 10.40.0.0/20
ip prefix-list in-peer deny 0.0.0.0/0 le 32
!
router bgp 10
no synchronization
network 10.10.0.0 mask 255.255.240.0
neighbor 10.10.15.14 description Peering with Router13
neighbor 10.10.15.14 prefix-list out-peer out
neighbor 10.10.15.14 prefix-list in-peer in
3
no auto-summary
!
Note: an IOS prefix-list always has an implicit deny any as the last statement even though it is not
listed in the configuration. Some ISPs add the implicit deny any as they consider it good practice
and a security precaution.
Note: these prefix-lists are only applied to peerings with other ASes. These are called external
peerings (using eBGP). There is usually no need to apply such filters for iBGP peerings.
Checkpoint #2: call the lab assistant to verify the connectivity. Each router team should check
peerings to see the effect of this step. Use the “show ip bgp neigh x.x.x.x advertise|route” commands.
STOP AND WAIT HERE
3. Remove configuration from previous example. This step will demonstrate how to remove the
configuration entered in the previous example. This is essential before we move onto the next step.
Example: Router R1
Router1#conf t
!
! First remove prefix list from BGP peering with R13
!
Router1(config-router)#no neighbor 10.10.15.14 prefix-list out-peer out
Router1(config-router)#no neighbor 10.10.15.14 prefix-list in-peer in
!
! Now remove the prefix-lists themselves
!
Router1(config)#no ip prefix-list out-peer
Router1(config)#no ip prefix-list in-peer
!
! That’s the configuration nice and tidy, the way it should be.
!
Router1(config)#end
!
! Now clear the bgp peering so that the old policy is removed
!
4
ISP Workshop Lab
Router1#clear ip bgp 40 out

Router1#clear ip bgp 40 in
Router1#
AS-PATH filters
4. Configure prefix filter based on AS path attribute: This step configures route prefix filtering
based on AS path. This is done using AS path access-lists, and is another method of controlling
networks which are exchanged in BGP peerings.
ip as-path access-list 2 permit ^$

ip as-path access-list 3 permit ^10$
!
router bgp 40
neighbor 10.40.15.18 filter-list 2 out
neighbor 10.40.15.18 filter-list 3 in
!

!
router bgp 10
neighbor 10.40.15.17 filter-list 2 out
neighbor 10.40.15.17 filter-list 3 in
!
Q. Why does the outbound filter list match the null as-path and not the local AS number in the
above examples?
A. Because the AS-PATH attribute is set after the prefix lists, as-path filters and route-maps. If the
local AS was included in the outbound filter-list configuration, the prefixes would be ignored as
their AS-PATH attribute would not be set at that stage.
To verify that the regular expression works as intended, use the EXEC command “show ip bgp
regexp <regular-expression>” to display all the paths that match the specified regular expression.
Don’t forget that the command clear ip bgp <neighbour address> [in|out] is required to
implement this filter.
5
Notice that the AS Path filters allow all prefixes originated by the neighbouring AS – in the
previous example using prefix filters, only the aggregate from the neighbouring AS got through
the filters.
Checkpoint #3: call the lab assistant to verify the connectivity. Once the lab instructor gives the class
the gohead, remove the attribute and filter list configuration and move on to the next step.
STOP AND WAIT HERE
5. Remove the configuration from the previous example. This step will demonstrate how to
remove the configuration entered in the previous example. This is essential before we move onto
the next step.
Example: Router 2
Router1#conf t
!
! First remove the filter list from BGP peering with R14
!
Router1(config-router)#no neighbor 10.40.15.17 filter-list 2 out
Router1(config-router)#no neighbor 10.40.15.17 filter-list 3 in
!
! Now remove the filter lists themselves
!
Router1(config)#no ip as-path access-list 2
Router1(config)#no ip as-path access-list 3
!
!
Router1(config)#end
!
!
Router1#clear ip bgp 10.40.15.17 in
Router1#clear ip bgp 10.40.15.17 out
Router1#
BGP Communities for filtering (and route-maps)

6. Introduction to BGP communities and route-maps: This step introduces the Router Teams to
using BGP communities for tagging, identifying, and eventually filtering prefixes. We achieve
similar end results to what we achieved in the previous steps using prefix-list and as-path filters.
On all routers, configure BGP to send a community for all prefixes belonging to the local AS
advertised to external BGP peers. The community should be in the form of [AS number]:[Router
number]. For example, Router8 should use community 30:8.
6
ISP Workshop Lab
Example on Router R8:
! Display communities in 16-bit:16-bit format rather than

! as one 32-bit integer.
!
ip bgp-community new-format
!
ip prefix-list out-match permit 10.30.0.0/20 le 26
!
route-map outfilter permit 10
match ip address prefix-list out-match
set community 30:8
!
router bgp 30
neighbor 10.20.15.17 route-map outfilter out
neighbor 10.20.15.17 send-community
!
Note:
1) A community attribute can be seen in the BGP table via show ip bgp <network> command.
2) A community attribute is a 32-bit field. By IETF agreed-upon convention it is divided into two
16-bit fields for easy interpretation. The top 16-bit contains the AS number, while the lower
16-bit represents an integer that has specific meaning to the two ASes involved in the peering.
The exceptions to this are the well-known community attribute strings such as no-export or
local-as.
Q: Why is send-community needed for eBGP peerings?

A: As community values are not passed by default between BGP peers, you need to tell the router
explicitly to do this.
Checkpoint #4: call the lab assistant to verify the connectivity. Each router team should again check
their peerings to see what the effect is this time.
STOP AND WAIT HERE
7. Remove the configuration from the previous example. This step will demonstrate how to
remove the configuration entered in the previous example. This is essential before we move onto
the next step.
Example: Router 9
Router8#conf t
!
! First remove the route-map from the eBGP peering
!
Router8(config-router)#no neighbor x.x.x.x route-map outfilter out
!
! Now remove the route-map itself
!
Router8(config)#no route-map outfilter
!
7
! And now remove the prefix-list used by the route-map

!
Router8(config)#no ip prefix-list out-match
!
!
Router8(config)#end
!
!
Router8#clear ip bgp 10.20.15.17 out
Router9#
BGP Communities
8. Setting BGP Communities. In step 6 the community a network belongs to was generated at the
point where one BGP router spoke to another BGP router. While this situation is valuable in
demonstrating how to set communities, the more common scenario is where an ISP attaches a
community to a network when the network is injected into the BGP routing table.
Each router team should assign a community to the customer network block (the /26) they
originate in BGP. Review the BGP documentation to find out how to do this. Each router should
set a community of format [AS number]:[Router number] exactly as in the previous step.
Example for Router R3:

!
route-map community-tag permit 10
set community 10:3
!
router bgp 10
no synchronization
network 10.10.0.128 mask 255.255.255.192 route-map community-tag
no auto-summary
!
ip route 10.10.0.128 255.255.255.192 null0
Check that the network appears with its community in the BGP routing table.
Q: Why do your external, but not your internal, peers see the community set on the network?
A: See earlier. All peerings require the BGP send-community subcommand for the community
attribute to be sent between BGP peers.
9. Communities on internal BGP peerings. Following on from the previous step, now set up the
internal peerings so that the community attribute for your network is sent to local peers.
Hint: to do this, simply add in the configuration line neighbor x.x.x.x send-community for all iBGP
peerings. Don’t forget to refresh the BGP peering sessions so that the configuration change can be
implemented.
8
ISP Workshop Lab
Checkpoint #5: call the lab assistant and demonstrate how the community has been set for your
network using the “show ip bgp” commands. Also, demonstrate that you can see the communities set
by your internal and external BGP peers.
10. Configure incoming prefix filter based on community attribute. The aim here is to only accept
networks which are received from the neighbouring external BGP peer. (This is similar to what
was being attempted in Steps 2 and 4 with prefix and AS path filtering.) For example, R13 should
only accept the network originated by R1, and should use the knowledge of the community R1 has
attached to the network to achieve this.
Example on Router R13:
ip community-list 3 permit 10:1

!
route-map infilter permit 10
match community 3
!
router bgp 40
neighbor 10.10.15.13 route-map infilter in
!
The community-list number choice is up to each router team – it is not announced in any BGP
peering or used in any other way apart from identifying the community-list (compare with the
access-list number).
11. Set local-preference attribute on received eBGP routes. In this example, local preference will
be set on the routes matching the community filter in Step 10. Retain the route-map used in Step
10 – an additional configuration line will be added to it. You also want to allow the other networks
heard through the filters with the local-preference set to the default value.
Q. Why?
A. Without the second permit directive the route-map implements a default deny and no other
prefixes will be passed through.
Example:

match community 3
set local-preference 120
!
Note that after a new policy is set, BGP session needs to be refreshed so that the new policy can
then be applied. The router does not automatically keep all the updates it ever received from the
peer so this is necessary. This is done by using “clear ip bgp <peer address> in” exec command.
Checkpoint #6: call the lab assistant and demonstrate how the routes originated by your eBGP peers
now have local preference set to 120. Also show how other routes have the default local preference of
100.
STOP AND WAIT HERE
9
BGP Peer-Groups
12. Configure the peer-group feature for iBGP peers. BGP peer-groups help reduce the router
processor load in generating and sending updates to peers which have the same policy. This step
configures BGP peer-groups for the iBGP peers in each AS. Replace the individual configuration
for each iBGP peer with a peer-group configuration, as given in the example below.
Example for Router R9:
router bgp 30
neighbor ibgp-peers peer-group
neighbor ibgp-peers description Peer-Group used for all iBGP peers
neighbor ibgp-peers remote-as 30
neighbor ibgp-peers update-source loopback 0
neighbor ibgp-peers send-community
neighbor ibgp-peers next-hop-self
neighbor ibgp-peers password cisco
!
Note: the old pre-peer-group configuration must be removed before converting from non-peer-
group configuration to using peer-groups.
router bgp 30
!
neighbor 10.30.15.224 peer-group ibgp-peers
!
Q: What are the advantages of using peer-groups?
A: BGP peer groups allow a common configuration to be used for several BGP peers. The most
common application is for iBGP. All internal BGP peers in an ISP network tend to have the same
relationship with each other. Rather than having a substantial configuration per peer, and having to
change every peering when details need to be changed, the configuration can be put in a peer-
group, and only the peer group has to be changed to alter the peering configuration for all iBGP
peers. This substantially reduces the work overhead required in making changes, the router CPU
for processing, and significantly cleans up the configuration to view.
It is strongly recommended that the peer-group subcommand be the “default” way of configuring
all BGP peers. As was stated before, most iBGP peers have the same configuration, so it is of great
benefit to the router, the operations staff, and network engineering staff to simplify the
configurations wherever possible. Besides, a configuration which makes extensive use of peer-
groups is usually much easier to read than one which has distinct configuration per peer, especially
in networks with large numbers of peers.
Note: Wherever BGP is configured in future in the workshop, it is expected that peer-groups
will be used as part of the BGP configuration (and most definitely for any iBGP
configuration).
10
ISP Workshop Lab
13. Summary: This module has introduced some of the basic features available to configure BGP
peerings in Cisco IOS. The reader is encouraged to try further permutations of the configuration
examples given here. Community usage is gaining in popularity as the feature is now recognised to
give considerable advantages in controlling routing policy between different ASes. Route refresh
and peer-groups are also widely used in ISP backbones as they considerably ease administration
and configuration of an operational network.
Review Questions:
1. Why is making use of Route Refresh the best way of implementing new BGP policy?
2. Why do AS-PATH filters provide less granularity then prefix-filters for filtering a BGP
session? And which is preferred in an ISP network, and why?
3. When should the community attribute be set on a BGP prefix?
4. Does IOS send BGP communities by default for iBGP? For eBGP? If not, what must operators
remember to do?
11
ISP Workshop Lab
Module 8 – Multihoming Strategies Lab

Objective: Introduction to routing policy, the manipulation of BGP attributes to control traffic
flow in a multihomed network.
Prerequisite: Module 6 and 7
Topology :
1
Lab Notes
This module demonstrates how an AS can use Local Preference to control outbound routing paths, and
how to use AS path prepend, MEDs (multi-exit discriminators or metrics) or communities to
determine inbound routing paths. All four are very powerful tools for ISPs to control how their
external peering links are used. Refer to the BGP presentations and documentation for more
information about the BGP path selection process and the default values for, and priorities of, the
“local_pref” and “metric” attributes.
Before starting this module, retain the topology and configurations as used in Module 6. This requires
the removal of all the filtering and community configurations examined in Module 7.
Recommendation: Remember, if any configuration on a router is not in use, it should be removed.

Surplus configuration usually gives rise to delayed error detection and debugging of configurations in
cases of routing problems or other network failures.
Lab Exercise
1. Tidy up from Module 7. If the previous module completed was Module 7, the router
configuration needs to be tidied up substantially before this module is attempted. The following
steps show exactly what is required.
Example: Router R1
Router1#conf t
!
! First remove BGP neighbour route-map statement
!
Router1(config-router)#no neighbor 10.10.15.14 route-map infilter in
!
! Now remove community-tag from network statement
!
Router1(config-router)#no network 10.10.0.0 mask 255.255.240.0 route-map
community-tag
Router1(config-router)#network 10.10.0.0 mask 255.255.240.0
!
! Now remove route-maps
!
Router1(config)#no route-map community-tag
Router1(config)#no route-map infilter
!
! Now remove community list
!
Router1(config)#no ip community-list 1
!
!
Router1(config)#end
!
!
Router1#clear ip bgp 10.10.15.14 in
Router1#
2
ISP Workshop Lab
Checkpoint #1: Call your lab instructor and display the following:
i] Output of a “show ip route” and “show ip bgp”

ii] Outputs of the ‘ping’ and ‘trace’ to various destinations within the network
iii] Outputs of the ‘ping’ and ‘trace’ after the primary link fails.
2. Aim of the Module:
The aim of the module is to demonstrate how to achieve a particular traffic flow using four
different methods. These four methods involve applying policies for the modification of outbound
traffic flow, and three ways of applying policies to modify inbound traffic flow. The reader should
review the BGP presentation given prior to this module as a reminder on how to influence path
choice using BGP policies.
The diagram following (Figure 2) displays the desired traffic flows between particular routers and
ASes. Eight traffic flows are to be implemented. The arrows in the figure show the flows that will
be configured. Each arrow originates from a border router in an AS, and terminates on one of the
routers in another AS. This signifies the traffic flows desired for the links between the two
systems. Each of the following steps has a description on how to implement the traffic flow
represented by each arrow. If at any time there is any doubt as to the configuration required,
consult the Cisco CD Documentation, or ask the lab instructors.
3. Assigning Address space within each ASN. To make the different policies in this module work,
we will subdivide each address block allocated to each ASN so that each router is responsible for
announcing a sub-prefix each. This way we will be able to target our traffic engineering based on
what individual routers announce to the rest of the lab network. The assignments for each ASN are
given in the following table:
AS10 AS30
Router1 10.10.0.0/22 Router8 10.30.0.0/22
Router2 10.10.4.0/22 Router9 10.30.4.0/22
Router3 10.10.8.0/22 Router10 10.30.8.0/22
AS20 AS40
Router4 10.20.0.0/22 Router11 10.40.0.0/22
Router5 10.20.4.0/22 Router12 10.40.4.0/22
Router6 10.20.8.0/22 Router13 10.40.8.0/22
Router7 10.20.12.0/22 Router14 10.40.12.0/22
Each router team should add a suitable network statement into their BGP and a static route so that
their allotted /22 is announced into the BGP system. Here is an example for Router5:
router bgp 20
network 10.20.4.0 mask 255.255.252.0
!
ip route 10.20.4.0 255.255.252.0 null0
As we will be doing traceroutes through the network to check that the policies have worked, it’s
also a good idea to configure another loopback interface with one address out of the /22 address
3
block. Each team should choose Loopback1 and give it the first address out of the block. For
example, on Router8:
ip address 10.30.0.1 255.255.255.255
!
Figure 2 – Preferred Paths
4
ISP Workshop Lab
4. Implement the following policies to influence outbound traffic flows using LOCAL_PREF:
This scenario demonstrates how to influence outbound traffic flows for each AS. Local preference
is applied to inbound routing announcements. The address blocks defined in the previous step are
used as the target: they are matched on the eBGP peerings and local preference is set high. While
we set local preference high on the preferred outbound path, it is important that backup paths
should still function – for that reason, we also set local preference low on the other exits from the
ASN. Here are the primary paths/routing that we want to achieve:
AS 10:
- All traffic TO 10.20.8.0/22 (R6) must exit AS 10 via Router R2 only.
AS 20:
AS 30:
AS 40:
Note that we are only trying to define outgoing traffic flow. The return path has no policies
implemented, and the router’s normal path decision process applies.
The example configurations below should be used as guidance for the router configuration for each
router team. Note that for each scenario, one router in each AS will have to set high local
preference and the remaining routers in the AS will set low local preference. The motivation for
doing this is redundancy of configuration. If, for example, Router 7 lost its local-preference
configuration due to some operator error, the low local preference set on the other three routers
will ensure that the traffic policies required will still be implemented. It’s quite common for many
ISPs to have more than one configuration to implement a particular policy – a primary
configuration, and an “inverse” secondary configuration on other routers which could be impacted.
Given there are two scenarios for each AS, each router in the AS will have to create a route-map
covering both scenarios. Note that only one route-map can be used per eBGP session. The example
below shows what Router 7’s configuration could look like.
Sample configurations for Router 7 for the AS 20 scenario above (using LOCAL_PREF).
ip prefix-list R1-prefix permit 10.10.0.0/22

!
route-map set-local-pref permit 10
match ip address prefix-list R1-prefix
!
5

!
!
router bgp 20
neighbor 10.30.15.5 route-map set-local-pref in
!
i] Each router in an AS will be asked to do a ‘trace’ to selected destinations, the trace must
show packets exiting the AS as specified in the exercise given above.
ii] Explain your configuration used to achieve the desired result to the instructor. Display the
output of “show ip bgp”, and “show ip bgp x.x.x.x” for the networks with local preference set
to 200. Show the output of a trace according to instructions.
iii] Wait until the lab instructor gives the goahead to move onto the next step.
STOP AND WAIT HERE
5. Remove configuration from the previous step. Before moving on to the next step it is important
that the configuration from the previous step is removed. This involves removing the route-maps,
prefix-lists and the per-neighbour configuration to set local preference. All router teams should do
this, and then do a soft reset of their eBGP peerings.
6. Implement the following policies to influence inbound traffic flows using MEDs. This step
introduces one of three methods of influencing inbound policies. Here MEDs are used, while the
next steps will introduce the use of BGP communities and AS path prepends. As for the previous
step, read the instructions careful, and discuss in your team, and in your AS, how you are going to
implement the following.
The example in this step achieves exactly the same traffic flow between neighbouring ASes as in
the previous step for the networks in question – remember that local preference is used by an AS to
influence outbound traffic paths, whereas MEDs are used to influence inbound traffic paths. Refer
to Figure 2 for a picture of traffic flow…
AS 10:
- All traffic TO 10.10.0.0/22 (R1) from anywhere in AS 20 must enter AS 10 via the R5 – R3
link. (Hint: this means that R2 must announce 10.10.0.0/22 to AS 20 with a higher metric than
the equivalent announcement from R3.)
AS 20:
6
ISP Workshop Lab
AS 30:
link. (Hint: this means that R10 must announce 10.30.4.0/22 to AS 40 with a higher metric
than the equivalent announcement from R9.)
AS 40:
link. (Hint: this means that R14 must announce 10.40.4.0/22 to AS 10 with a higher metric
than the equivalent announcement from R13.)
- All traffic TO 10.40.8.0/22 (R13) from anywhere in AS 30 must enter AS 40 via the R10 –
R12 link. (Hint: this means that R11 must announce 10.40.8.0/22 to AS 30 with a higher
metric than the equivalent announcement from R12.)
Sample configuration for Router 6 for the AS 20 scenario above (using MED).

!
route-map set-med permit 10
set metric 10
!
set metric 50
!
!
router bgp 20
neighbor 10.20.15.18 route-map set-med out
!
i] Each router in an AS will be asked to do a ‘traceroute’ to selected destinations, the trace

must show packets exiting the AS as specified in the exercise given above.
output of “show ip bgp”, and “show ip bgp x.x.x.x” for the networks with MED set to 50. Show
the output of a trace according to instructions.
7
STOP AND WAIT HERE
7. Remove configuration from the previous step. Before moving on to the next step it is important
that the configuration from the previous step is removed. This involves removing the route-maps,
prefix-lists and the per-neighbour configuration to set MEDs. All router teams should do this, and
then do a route refresh of their eBGP peerings.
8. Implementing policies using BGP communities. This section describes how to use BGP
communities to influence inbound policies. Rather than using MEDs as we did in the previous
scenario (Step 6), we will signal to our neighbouring ASN by using BGP communities. To prepare
for this we need to establish which communities will implement the policies replacing the MEDs.
Setting low MED means that the path would be preferred over one with high MED. This could be
replicated in the peer AS by that AS setting high local preference on the path that would have
heard the low MED, and by setting low local preference on the path that would have heard the
high MED.
9. Choose communities to set high and low priority for outbound traffic. As a recommendation,
configure community <localASN>:150 to set high priority for outbound traffic, and
<localASN>:50 to set low priority for outbound traffic. This means that each AS will have to
configure a community matching route-map to set the appropriate local-preference value.
Example configuration for Router in AS20:

10. Set up the route-map configuration to implement the community policy. We now create a
route map which will set local-preference for each of the communities. If the eBGP neighbour
sends us the community attached to a prefix, we will apply local preference to the prefix
depending on the community value attached.
Example configuration:
route-map customer-comm permit 10

match community 1
!
match community 2
!
!
11. Apply the route-map to eBGP neighbours. With the route-map configured, we now apply it to
all our eBGP neighbours. When they send prefixes with the appropriate communities set, we will
now set the local preference.
Example configuration for Router 8:
8
ISP Workshop Lab
router bgp 30
neighbor 10.20.15.17 description eBGP peering with R6
neighbor 10.20.15.17 route-map customer-comm in
!
While the whole concept looks more complex at first glance, it actually scales a lot better as the
service provider is able to standardise their policy configuration using communities all across their
access network.
12. Implement the following policies to influence inbound traffic flows using BGP communities.
As for the previous step, please read the instructions carefully, and discuss within your team, and
in your AS, how you are going to implement the following.
the previous step for the networks in question. Refer to Figure 2 for a picture of traffic flow…
AS 10:
link. (Hint: this means that R2 must announce 10.10.0.0/22 to AS 20 with AS20’s low priority
community whereas the equivalent announcement from R3 needs AS20’s high priority
community.)
community.)
AS 20:
community.)
community.)
AS 30:
community.)
link. (Hint: this means that R10 must announce 10.30.4.0/22 to AS 40 with AS40’s low
priority community whereas the equivalent announcement from R9 needs AS40’s high priority
community.)
AS 40:
link. (Hint: this means that R14 must announce 10.40.4.0/22 to AS 10 with AS10’s low
9
priority community whereas the equivalent announcement from R13 needs AS10’s high
priority community.)
- All traffic TO 10.40.8.0/22 (R13) from anywhere in AS 30 must enter AS 40 via the R10 –
R12 link. (Hint: this means that R11 must announce 10.40.8.0/22 to AS 30 with AS30’s low
priority community whereas the equivalent announcement from R12 needs AS30’s high
priority community.)
Look at the following examples to see what needs to be done for each router team.
Sample configuration for Router8 for the AS 20 scenario above (using Community).

!
route-map set-comm permit 10
set community 30:50
!
route-map set-comm permit 20
!
router bgp 30
neighbor 10.20.15.17 description eBGP peering with R6
neighbor 10.20.15.17 route-map set-comm out
neighbor 10.20.15.17 route-map customer-comm in
!

output of “show ip bgp”, and “show ip bgp x.x.x.x” for the networks with configured
communities. Show the output of a trace according to instructions.
STOP AND WAIT HERE
13. Remove the configuration used for the previous step. Before moving on to the next step it is
important that the configuration from the previous step is removed. This involves removing the
route-maps, prefix-lists and the per-neighbour configuration to set communities. All router teams
should do this, and then do a soft reset of their eBGP peerings.
14. Implement the following policies to influence inbound traffic flows using the AS path
prepend method. This step introduces the second of two methods of influencing inbound policies.
As for the previous step, read the instructions carefully, and discuss within your team, and within
your AS, how you are going to implement the following.
the previous step for the networks in question. Refer to Figure 2 for a picture of traffic flow…
10
ISP Workshop Lab
AS 10:
- All traffic TO 10.10.8.0/22 (R3) from anywhere in the lab topology must enter AS 10 via the
R14 – R2 link. (Hint: this means that R1 and R3 must announce 10.10.8.0/22 with a longer AS
path than the other networks in AS 10. R2 needs to announce 10.10.8.0/22 with a longer AS
path in its peering with R4.)
path than the other networks in AS 10.)
AS 20:
R8 – R6 link. (Hint: this means that R4, R5 and R7 must announce 10.20.4.0/22 with a longer
AS path than the other networks in AS 20.)
R2 – R4 link. (Hint: this means that R5, R6 and R7 must announce 10.20.8.0/22 with a longer
AS path than the other networks in AS 20.)
AS 30:
path than the other networks in AS 30. R9 needs to announce 10.30.0.0/22 with a longer AS
path in its peering with R11.)
R11 – R9 link. (Hint: this means that R8 and R10 must announce 10.30.4.0/22 with a longer
AS path than the other networks in AS 30. R9 needs to announce 10.30.4.0/22 with a longer
AS path in its peering with R7.)
AS 40:
R1 – R13 link. (Hint: this means that R11, R12 and R14 must announce 10.40.4.0/22 with a
longer AS path than the other networks in AS 40.)
R10 – R12 link. (Hint: this means that R11, R13 and R14 must announce 10.40.8.0/22 with a
longer AS path than the other networks in AS 40.)
AS_PREPEND is commonly used by smaller ISPs who are multihoming to their upstream
providers. It is convention on the Internet to add at least two ASes when using AS_PREPEND.
More usually, three ASes are added, especially if the upstream ISPs have links to each other going
through a third party.
Sample configuration for Router 9 for the AS 30 scenario above (using AS PATH prepend):

!
route-map set-path-as20 permit 10
set as-path prepend 30 30 30
!
11

!
!
!
router bgp 30
neighbor 10.20.15.6 descr eBGP peering with R7
neighbor 10.20.15.6 route-map set-path-as20 out
neighbor 10.40.15.1 descr eBGP peering with R11
neighbor 10.40.15.1 route-map set-path-as40 out
!
Checkpoint #5: Call the lab instructor and display the following:

output of “show ip bgp”, and “show ip bgp x.x.x.x” for the networks with increased AS path
length. Show the output of a trace according to instructions.
iii] How has AS Path prepend changed the BGP table and the Routing decision. Can the
decision be overridden using any other BGP configuration within an AS? Answer: review the
BGP route selection rules from the slide in the presentation section.
15. Summary:
This module has demonstrated several ways of influencing inbound and outbound routing policy.
Q: What is the difference in the resulting effects using the methods in steps 6, 8 and 14?
A: AS PATH prepend affects routing announcements between two ASes, and is visible
everywhere, even outside the two ASes which are making use of the prepend information. MEDs
only apply between multiple peerings between the same AS. If the peer AS is announcing the local
AS onwards, the metric set is that of the peer AS, not the local AS. Communities are generally
only used in peerings between neighbouring ASNs.
Consult the BGP documentation for more information. There are many possible variations on the
examples given in this module. Remember the following points:
• local preference is used for influencing policy within an AS
• MEDs are used to influence policy over multiple links between the local and an immediately
neighbouring AS
• BGP Communities can also be used to influence policy over multiple links between the local
and neighbouring AS
• The AS path prepend is used to influence external policy on a global scale (which includes the
immediately neighbouring AS).
12
ISP Workshop Lab
Review Questions
1. Which is the most effective way of influencing how traffic leaves your network?
2. How useful do you think MEDs are in the real live Internet? Consider the answer to question
one before replying!
13
ISP Workshop Lab
Module 9 – BGP Configuration Essentials Lab

Objective: Using the network configured in Module 6, to introduce more BGP Configuration
Essentials for application in ISP networks
Prerequisite: Modules 6, 7 and 8.
Topology :
1
Friday, June 08, 2012
Lab Notes
This module demonstrates some of the more recent features added to BGP in Cisco IOS. These are
now considered good practice by many of the ISPs in the Internet industry, and are introduced here to
give workshop participants hands on experience of these features prior to deployment on their own
networks.
Before starting this module, retain the topology and router configurations as used at the start of
Module 8. This requires the removal of all the filtering, route-maps, and community configurations
examined in Modules 7 and 8.
Recommendation: Remember, if any configuration on a router is not in use, it should be removed.

Surplus configuration usually gives rise to delayed error detection and debugging of configurations in
cases of routing problems or other network failures.
The links shown in Figure 1 represents connectivity between AS’s. It is assumed that all the routers
within an AS connect to each other.
Lab Exercises
1. Restore Configuration. Restore the lab configuration to what it was at the start of Module 8/end
of Module 7. The lab should be made up of 4 ASes, each AS is running either OSPF or ISIS
internally, iBGP overlaid on that, and eBGP between the AS neighbours. Good configuration
practices learned in Module 7 should be retained (for example, use peer-groups for iBGP, soft-
reconfiguration if you think it will help you, etc).
2. Path MTU discovery. The default MTU for all communications originating from the router is 512
bytes – while this may be sufficient for most light use purposes, ISP networks tend to place larger
stresses on routers. Enabling path MTU discovery on the router will ensure that the router will use
the optimum (i.e. largest) MTU possible for a communication. For example, a router with several
BGP neighbours and exchanging the full Internet routing table with each neighbour will be able to
transfer this routing table almost 3 times faster over Ethernet or serial connections with path MTU
discovery enabled (allowing 1500 byte packets) than with using the default MTU of 576 bytes.
Enable path MTU discovery on your router:
Router(config)#ip tcp path-mtu-discovery

Router(config)#
While there may not be much visible difference in router performance in the workshop lab,
participants are encouraged to add this command to their default router configuration.
Once this configuration has been entered, each BGP peering should be reset, and the new MTU
should now be displayed in the results of show ip bgp neighbor.
3. IP unreachables. When implementing BGP in an ISP network, the classic and recommended way
of inserting a prefix into the BGP table is by configuring a network statement in BGP and a
matching static route to the Null0 interface (the so-called pull-up route). We saw this used in
Modules 1 and 6.
2
ISP Workshop Lab
The benefit delivered to the ISP network by using this method is that any traffic destined for any
IP address covered by that address block will have a final destination, regardless as to whether the
IP address is routed on the network or not. For example, if a customer is using a /25 address range
out of the ISPs /20 address block, and that customer disconnects from the Internet to allow
maintenance on their connection, traffic trying to reach the /25 address block will be “caught” by
the aggregate’s null route. This means the traffic doesn’t traverse the ISP’s backbone before dying
on the aggregation router, but is caught “early” on as it enters the backbone. This is operationally
tidier for many ISPs, and can be less confusing for Internet users as well.
(The static route to Null0 has many uses, and is one of the tools used frequently in helping with
defeating denial of service attacks on service provider and end user networks.)
However, the side effect of doing this is that the router has to send a response that the packet has
reached a destination – this response is that the destination is “unreachable”. Each packet generates
one response – an ICMP unreachable message. For a stream of packets, this can introduce some
burden on the router CPU, so many ISPs configure the Null0 interface to not send ICMP
unreachables – the packets end up at the Null0 interface and are silently discarded. This is much
lighter on the router CPU.
First ask your neighbour to ping any IP address in your address block – the router should respond
with a series of “UUUUU”, indicating a host unreachable.
Now disable the sending of ICMP unreachables on your router’s Null0 interface:
interface Null 0
no ip unreachables
If you now ask your neighbour repeat the ping – there should be no response at all, the router
simply displaying a series of “…..”, indicating no response what so ever.
Try this with your neighbour’s address block too, before and after they have disabled the ICMP
unreachable processing on their router.
4. OSPF & ISIS timers. Many ISPs no longer retain the standard IGP timers that have been the
defaults in IOS for many years. Technology has moved forwards rapidly, and what were
considered acceptable defaults in the early nineties given the processor technology then are now
rather out of date.
OSPF Lab: The IOS default for OSPF is for hellos to be sent every 10 seconds, with the
deadtimer set for 40 seconds (in other words, the router will have had to miss 4 hellos/keepalives
before it considers the neighbour has disappeared). Each router team should now reduce the OSPF
timers from the defaults to new values of 3 seconds for hellos, 15 seconds for the deadtimer. This
configuration has to be applied to each interface where an OSPF adjacency is expected. For
example:

ip ospf hello-interval 3
ip ospf dead-interval 15
ISIS Lab: The IOS default for ISIS is for hellos to be sent every 10 seconds, with the deadtimer
set for 30 seconds (in other words, the router will have had to miss 3 hellos/keepalives before it
3
considers the neighbour has disappeared). Each router team should now reduce the ISIS timers
from the defaults to new values of 3 seconds for hellos, 15 seconds for the deadtimer (so making
the hello-multiplier the value 5). This configuration has to be applied to each interface where an
ISIS adjacency is expected. For example:

isis hello-interval 3
isis hello-multiplier 5
5. BGP timers. For the reasons mentioned in the previous step, many ISPs also no longer retain the
standard IOS BGP timers for their iBGP and eBGP. IOS defaults are 60 second keepalives, and
180 second dead time (in other words, if the router misses 3 keepalives it tears the session down).
For this lab, each time will reduce the BGP timers to 15 second keepalives and 45 second
deadtime.
router bgp 1
timers bgp 15 45
To make the peering use these new timer values, restart the BGP sessions on the router. The two
peers will negotiate timer values (usually the lowest values will win) to be used for the BGP
session. Check with a show ip bgp neighbor command.
What values an ISP should use in their own network really come down to experience as well as a
bit of experimentation. While the IOS defaults will work perfectly well, significant gains in
performance and routing failover can be achieved by reducing the timers. Review the BGP Best
Practices presentation for more advice.
6. Deterministic MED. Another essential BGP configuration considered industry best practice is the
use of deterministic MED. Normally if a BGP speaker hears an external prefix from several
different connections to the same ASN, path selection will be carried out according to the order
that the prefixes are heard in (oldest through to the most recent). For an ISP with multiple
connections to another ASN, this usually proves to be very annoying, as the best path will be
different each time a link is reset. To fix this, we enable deterministic MED, and that ensures that
the prefixes are sorted in a predefined order (according to AS-PATH) and will give us the same
best-path selection result each time. To activate deterministic MED, each router team should now
configure the following global BGP command:
router bgp 1
bgp deterministic-med
7. Shutting down BGP peerings. A useful feature is the ability to shut down eBGP peerings – most
BGP implementations have this functionality now, and IOS has had it for a few years (even though
many operators don’t realise so). If a peering needs to be temporarily disabled, either to help with
troubleshooting, or to suspend a peering, etc, then the shutdown option is substantially better to
use than the old method of simply removing the configuration altogether. The syntax is very
straight forward – neighbor x.x.x.x shutdown. To reactivate the neighbour again, the
reverse is used – no neighbor x.x.x.x shutdown. Be careful with the reactivation command
– forgetting the shutdown option will result in the entire configuration for this neighbour peering
being removed.
4
ISP Workshop Lab
For this step, all router teams are going to shutdown the eBGP peerings they have configured on
their routers. Routers 2 and 9 have two eBGP peerings, the rest have just one. An example
configuration might be:
router bgp 20
neighbor 10.30.15.1 shutdown
!
which shuts down the BGP peering between the local router and its 10.30.15.1 neighbour.
Checkpoint #1: Call the lab instructors and let them know that you have completed the module up to
this point. Once the lab instructors have explained the eBGP multihop section coming next, you will
be asked to carry on with this Module.
STOP AND WAIT HERE
8. Using eBGP multihop. Some ISPs use a concept called eBGP-multihop for their eBGP peering
sessions. While this practise is strongly discouraged unless circumstances dictate otherwise, it is
worth looking at the configuration so that the concepts can be understood.
eBGP multihop basically means that the eBGP router disables the check that the eBGP neighbour
is accessible on a directly connected interface of the local router. This allows the service provider
to establish an eBGP session between the local and a distant router in the neighbouring AS. Some
ISPs do this as they have a policy of only running BGP in their core network devices, and don’t or
won’t support BGP on their aggregation/edge devices.
This step will reconfigure the lab eBGP sessions to use eBGP multihop instead of direct neighbour
peering. The IOS configuration command is neighbor x.x.x.x ebgp-multihop N where N
represents the number of TCP hops the neighbour x.x.x.x is away from the local router. Some ISPs
make N equal to 255 – which means the neighbour can be up to 255 hops away. Other ISPs
actually specify how many hops away the peer is. The former is easy to manage – you don’t need
to worry about internal topology changes of your neighbouring ISP – but it can be highly risky too,
as it is possible that the peering path between your router and your ebgp-multihop neighbour might
change to one that you do not expect1 (or want).
A more complete configuration example would be as follows. The static route for the 1.2.3.4/32
destination is required – BGP cannot use a next-hop which has been learned by BGP (and as ISPs
don’t run IGPs between their networks, a static route is what is required).
router bgp 55
neighbor 1.2.3.4 ebgp-multihop 5
!
ip route 1.2.3.4 255.255.255.255 serial 0
!
1
It is for this reason that many ISPs simply won’t permit or support ebgp-multihop configuration on their networks.
5
Finally, ebgp-multihop is usually run between loopback interfaces on peering routers. It doesn’t
have to be, but it by and large is, and for the same reasons that iBGP is run between loopback
interfaces. Loopbacks rarely change, whereas physical interface addresses can quite often do so.
Now convert all the eBGP sessions so that they use ebgp-multihop between the loopback
interfaces of the peering routers. Use the correct number for the TCP hop count. And remember
the static route to point to the remote router address. An example configuration might be:
router bgp 20
neighbor 10.30.15.224 ebgp-multihop 2
!
ip route 10.30.15.224 255.255.255.255 serial 0/0
!
Check that the eBGP session comes up – if it doesn’t check with your neighbour that they have
also completed this step. The prefixes advertised by your neighbouring AS will have next hop of
the peer address (as before).
Note that the configuration for the neighbour 10.30.15.1 should be simply shut down (as it was in
the previous step!) because we will be removing the ebgp-multihop configuration and reverting to
the direct eBGP configuration at the end of this exercise.
this point. Once the lab instructors have demonstrated the eBGP multihop configurations to the rest of
the class, you will be asked to carry on with this Module.
STOP AND WAIT HERE
9. Remove ebgp-multihop configuration. The ebgp-multihop configuration added in the previous

steps should now be removed. And the prior directly connected point-to-point configuration should
be re-enabled (in other words, do a no neighbor x.x.x.x shutdown on the affected eBGP
peering). So something like the command sequence following:
Router1#conf t
!
! First reactivate directly connected eBGP with Router 13
!
Router1(config-router)#no neighbor 10.10.15.14 shutdown
!
! Now remove Router 13 eBGP peering which used ebgp-multihop
!
Router1(config-router)#no neighbor 10.40.15.226
!
Router1(config-router)#end
Router1#
Once this is complete, there should be no more ebgp-multihop configuration in the network. The
previous steps were there simply to show how it is configured. As mentioned earlier in the module,
and in the BGP presentations, use of ebgp-multihop is strongly discouraged.
6
ISP Workshop Lab
10. Prefix Threshold Warning. In the lab network we are dealing with only a limited number of
prefixes – each router is originating one prefix into the BGP routing system. In the Internet, there
are well over one hundred thousand prefixes, and only routers with large amounts of memory can
handle such large numbers. It’s for this reason that many ISPs implement a so-called cut-off
system on their routers – if they receive more prefixes than they expect from an eBGP peer, then
the router can first warn them and then tear down the BGP session if the warnings are ignored and
the router is in danger of running out of memory. (A downed BGP session is more likely to be
noticed more quickly than an oscillating BGP session caused by a router which has run out of
memory. Similarly, a downed BGP session will have less severe impact than a BGP session which
is receiving thousands of spurious prefixes and causing havoc in the ISP’s routing system.)
The IOS BGP subcommand to set this maximum prefix is this:
neighbor x.x.x.x maximum-prefix n [threshold] [warning-only] [restart m]
where threshold is the percentage of maximum-prefix when the router will start sending
warning messages, warning-only stops the router from tearing the peering down when max-
prefix is reached, and restart m is the time in minutes before the router will attempt to restart
the eBGP session.
For this step, the Router Teams will set a maximum prefix limit on all their eBGP sessions. There
are 4 prefixes visible in the routing system, so it’s suggested that a useful limit to use might be 10.
The configuration to do so is similar to this:
router bgp 20
neighbor 10.30.15.1 max-prefix 10
!
This means that the router will tear down the BGP session when the number of prefixes learned
from Router8 exceeds 20. With existing IOS defaults, the router will warn by log messages that
the limit is getting close when it has received 15 prefixes (or 75% of the threshold).
this point. Once everyone has reached this stage, the lab instructors will introduce more prefixes to
the BGP routing system, as described in the next step.
11. Testing max-prefix. The lab instructors will now introduce more prefixes to the routing system in
the workshop lab network. All Router Teams should watch for the warning log messages
appearing in their router log files, for example like:
%BGP-4-MAXPFX: No. of unicast prefix received from 10.10.15.14 reaches 8, max 10
and when the actual max-prefix limit has been reached, messages such as:
%BGP-3-MAXPFXEXCEED: No. of unicast prefix received from 10.10.15.14: 11 exceed limit 10
and the resultant tear down of the BGP session. Router Teams should experiment with using the
warning-only keyword, as well as the restart timer. The instructors will allow sufficient time for
this, and will demonstrate each option before moving on with this Module.
7
Checkpoint #4: Wait at this point until the Lab Instructors have completed demonstrating the
maximum-prefix function available for the BGP session.
STOP AND WAIT HERE
12. Maximum-prefix summary. Before moving onwards tidy up from the previous steps, but leave a
usable and working maximum-prefix configuration in place. It is recommended that maximum-
prefix should become part of the ISP’s standard BGP configuration. All operators should know
exactly how many prefixes they should expect from each eBGP neighbour and should set an
appropriate limit.
13. Limiting AS-path length. As with limiting the number of prefixes learned from a peer, the router
can also be configured to limit the length of AS-paths learned from peers. Configuring this limit
means that the router will discard all prefixes with an AS-path length longer than the limit.
The syntax for this command is as follows:
bgp maxas-limit n
where n is the maximum AS-path length permissible for any prefix. All Router Teams should
configure this in their BGP configurations. It is suggested that n should be 5 for this workshop –
this is two more than the longest AS-path length ever likely to be seen (there are 4 ASNs!).
14. Effect of limiting AS-path length. On completion of the previous step, the router teams should
now artificially increase the length of AS-paths of prefixes announced in eBGP sessions to their
neighbours. To do this, they should use a route-map with the as-path prepend subcommand on
the eBGP peerings.
Recall that the final exercise in Module 8 demonstrated rudimentary traffic engineering using the
as-path prepend construct in a route-map. Consult your notes from Module 8 to construct a route-
map to increase the as-path length of the prefix you originate into the routing system when you
announce it to your neighbours.
The following configuration snippet is an example of what is expected:
router bgp 20
neighbor 10.30.15.1 route-map increase-as-out out
!
route-map increase-as-out permit 10
match ip address prefix-list myaddress
set as-path prepend 20 20 20 20 20
route-map increase-as-out permit 20
!
ip prefix-list myaddress permit 10.20.0.0/20
!
The route-map only applies the as-path prepend of five times AS 20 to the prefix originated by the
router above. Other prefixes learned by the router through its other BGP sessions are announced
unaltered.
8
ISP Workshop Lab
Q. What happens to the announced prefix? You may need to ask to see the console of your
neighbour’s router.
A. The prefix is discarded. And you should see log messages similar to:
%BGP-6-ASPATH: Long AS path 6435 145 6175 109 16713 3549 790 6667 received from
192.168.1.1: More than configured MAXAS-LIMIT
in the router logs, and that the prefix with this long path is not present in the BGP table. Try
changing the maxas-path limit value and see what happens.
Checkpoint #5: Wait at this point until the Lab Instructors have completed demonstrating the maxas-
path function available for the BGP session.
STOP AND WAIT HERE
15. Maximum AS-path length summary. Before moving onwards tidy up from the previous steps,
removing the route-map which was added to artificially increase the as-path length of the
originated prefixes, but leave a usable and working maximum-prefix configuration in place. It is
recommended that maximum as-path length limit should become part of the ISP’s standard BGP
configuration. The Internet is only around 5 ASNs deep on average, and a maximum length seen
over the last decade, including prepends (and excluding accidents!), has been 27 ASNs. Many ISPs
set a limit around 25 or 30, and this avoids problems should the inevitable accidents happen.
16. AS masquerading. Another useful tool for ISPs, especially those on the acquisition trail, or
indeed any service provider merging ASes into one network, is the local-as BGP subcommand.
This allows a router in one AS to masquerade being in another AS. For example, if one service
provider has bought another, it’s very easy to migrate the network during maintenance periods
from one ASN to another ASN. However, migrating customers and peers can only be done at
times that those peers and customers agree upon – and not many will want to migrate their eBGP
sessions when the service provider wants to. (From personal experience, the author doesn’t know
of many customers who would be happy to get up at 4am to reconfigure their eBGP session with
their ISP!)
The local-as BGP subcommand allows the service provider to migrate the ASN the router is using,
but still present the original AS to his customers and peers routers. The syntax is as follows:
neighbor x.x.x.x local-as N
where N is the ASN that the service provider wants to masquerade as – i.e. the ASN that the router
x.x.x.x is expecting to peer with.
This section is going to introduce this option into the ebgp-sessions in the network. Rather than
doing a wholesale reconfiguration of BGP on all the lab routers, we will introduce “ghost” ASNs
between the existing ASNs, according to the following table:
First AS Inserted AS Second AS

10 100 20
20 200 30
30 300 40
9
40 400 10
This means that Router 2 and Router 3 should add in local-as 100 in their eBGP with Router 4
and Router 5 respectively. Router 4 and Router 5 should replace their peering with AS 10 with a
peering with AS 100. (And similar for other AS peerings – the first AS adds in the local-as option,
the second AS changes the eBGP session from that of the First AS to that of the Inserted AS.) For
example:
On Router 2:
router bgp 10
neighbor 10.10.15.10 local-as 100
!
On Router 4:
router bgp 20
!
Once this work has been completed over the whole workshop network, the eBGP sessions will all
be restored, and AS-paths will show the inserted AS.
Checkpoint #6: Wait at this point until the Lab Instructors have completed demonstrating the local-as
function available for the BGP session.
STOP AND WAIT HERE
17. local-as summary. Before moving onwards tidy up from the previous steps, removing the local-as
configuration which was added, and restore the original eBGP configuration between the 4 ASes.
This feature only needs to be used in circumstances described earlier – but it does exist, and makes
many service provide ASN transition scenarios much easier to plan and implement.
18. Private ASes. Private AS numbers are often used by service providers for multihoming their
customers on to their backbones (for example, as in RFC2270), or for separating out parts of their
internal network (e.g. for a lab or development network), or if they have deployed Confederations
to scale their iBGP mesh. Private ASNs range from 64512 to 65534 – 1023 are available, more
than enough for service provider networks today. As they are intended for private use, their
assignment internal to an ISP network is proprietary to that ISP – there does not need to be any
coordinated assignment as with the public ASNs.
For this step, we are going to convert two of the ASes from public numbers to private numbers.
AS 10 should change their BGP configuration so that they are in AS 64512 and AS 40 should
change their BGP configuration so that they are in AS 65500. This will involve changing the entire
BGP configuration, including the iBGP. The easiest way of doing this to make an electronic copy
of the BGP configuration as it is now, replace the instances of AS 10 with AS 64512 and AS 40
with AS 65500 respectively, remove the existing BGP configuration from the routers in AS 10 and
AS 40, and apply the new configuration. Note that the routers in AS 20 and AS 30 will have to
change all instances of AS 10 and AS 40 to the new ASNs.
10
ISP Workshop Lab
The eBGP peerings between Router 1 and Router 13, and Router 2 and Router 14 should be shut
down for this step – the only transit path between AS 10/64512 and AS 40/65500 will be through
AS 20 and AS 30.
For example, the main parts of the Router 3 BGP configuration changes from:
router bgp 10
!
to:
router bgp 64512

!
The routers in AS 20 and AS 30 which peer with the new private ASes have their configuration
changed similar to the following example for Router 5:
router bgp 20
!
Once this step has been completed by all participants in the workshop, two ASNs in use will be
private, and two will be public. The BGP table anywhere in the lab will show both private and
public ASNs in the AS-paths.
19. Private ASNs and the public Internet. As mentioned in the previous step, private ASNs should
not be used or announced on the Internet. To ensure that private ASNs are not leaked out to the
Internet, configuration which strips private ASNs out of the AS-paths should be added to all eBGP
sessions.
All eBGP sessions in the lab should now add configuration to remove these private ASNs. For
example:
router bgp 20
neighbor 10.30.15.1 remove-private-AS
!
The neighbor x.x.x.x remove-private-AS line above will strip out any private ASNs in
the announcements it makes to the eBGP neighbour. Once everyone in the workshop has
completed this task, observe what has happened to the BGP table – note the absence of private
ASNs in the BGP table now.
11
this point. Once the lab instructors have demonstrated the remove-private-AS configurations to the
rest of the class, you will be asked to carry on with this Module.
STOP AND WAIT HERE
20. Private ASNs and the public Internet (part 2). The remove-private-AS neighbour
subcommand will only remove the private ASNs in certain circumstances. Most especially, if there
is a public ASN in the AS path, the private ASNs will not be removed, as this situation is
considered as a configuration error. This feature sometimes confuses some operators, so it’s worth
demonstrating it in the confines of the workshop network.
The Router Teams operating Routers 1, 2, 13 and 14 should now reactivate the eBGP peering
between their routers. Note that the teams will need to change their peering AS numbers (unless
this was already done as part of the step before last).
Once the two BGP sessions are re-established, look at the AS-paths visible throughout the
network. Note that in some cases, the private ASNs have reappeared – this is because some routers
see the public ASN in amongst the private ASNs, so the remove-private-AS subcommand has
no effect.
21. Summary. This module has introduced some of the more advanced BGP configuration options
available for ISPs. While this may be an introductory Workshop module, these configuration
options are vital fundamental building blocks of any ISP operation requiring to use BGP for their
Internet connectivity.
Review Questions
1. How many private ASNs are there?
2. How do you ensure that private ASNs are always removed from the Internet Routing Table?
12
ISP Workshop Lab
Module 10 – An Internet Exchange Point

Objective: To investigate methods for connecting to an Internet Exchange Point.
Prerequisites: Modules 1, 6 to 8, and the Exchange Points Presentation
Lab Notes
The purpose of this module is to introduce the concept of an Internet Exchange Point, how to peer at
IXPs, and look at some of the recommended configuration practices.
This module concentrates more on IXP peering, so setting up the eBGP sessions, implementing
correct prefix-filters, and so on. It is more suited to where workshop participants are interested in
participating in an IX in their region, and so can get first hand experience at configuring the eBGP
sessions with the other participants. However it misses out some important details regarding
connecting a fully fledged ISP network to an IX.
Module 16 on the other hand looks at a more realistic IXP. It has six ASNs participating at the IX,
with two routers being assigned to each autonomous system. One router peers at the IX, the other
router is internal to the autonomous system (so will have an OSPF and iBGP session with the IX
facing router). It also has three routers operating as three transit provider or Tier1 ISPs. That module is
strongly recommended over this one as it covers correct OSPF and iBGP configuration practices for
connecting the IXP facing router to the rest of the ISP network.
Lab Exercises
1. A Simple IXP. This example is of a very simple IXP. But using this configuration, any participant
in this workshop should be able to go away and set up a working IXP in their own economy.
Technically they are not hard to implement. Politics & business economics are not covered in this
workshop.
Only prefix lists are used to filter BGP announcements. eBGP peers should be in peer-groups, and
route refresh should be used to implement any policy changes as in other modules.
2. Basic Configuration. Each router team should configure their router to fit into the network layout
depicted in Figure 1. Check all connections. Note that all links are by ethernet.
3. Addressing Plan. These address ranges should be used throughout this module. You are welcome
to use your own range within an AS if you desire, just so long as you consult with the teams in
other ASes to ensure there is no overlap.
AS101 10.1.0.0/20 AS108 10.8.0.0/20

AS102 10.2.0.0/20 AS109 10.9.0.0/20
AS103 10.3.0.0/20 AS110 10.10.0.0/20
AS104 10.4.0.0/20 AS111 10.11.0.0/20
AS105 10.5.0.0/20 AS112 10.12.0.0/20
AS106 10.6.0.0/20 AS113 10.13.0.0/20
AS107 10.7.0.0/20 AS114 10.14.0.0/20
1
4. Basic Router Setup. Set up the routers as you would have done in previous modules. That is,
basic security, the BGP outline configuration, IOS Essentials, etc.
AS104 AS105
R4 R5
AS103 AS106
R3 R6
AS102 e0 AS107
R2 R7
AS101 AS108
R1 C2924XL R8
AS109
AS114
R14 R9
AS113 AS110
R13 R10
AS112 AS111
R12 R11
IXP Management
R15
Figure 1 – IXP Configuration
5. Management Router and IXP LAN. The lab instructor will have connected another router to the
exchange point – this is Router 15 in the figure. Each router team should set up their router to
synchronise time off that router using NTP. The password on the NTP session is “cisco” as in
previous Modules. The address range used for the IXP LAN is 172.16.0.0/24 – the management
router in this module has an IP address of 172.16.0.254. Each of the ASes is assigned a block of 3
addresses to use on the exchange point LAN. So, for example, AS101 has 172.16.0.1, 172.16.0.2
and 172.16.0.3. AS102 has 172.16.0.4, 172.16.0.5 and 172.16.0.6. And so on.
Q. Why do you think three addresses have been assigned to each participant at the IXP?
Checkpoint #1: When you have properly configured your router, and the other routers at the IXP are
reachable (i.e. you can ping the other routers), please let the instructor know.
2
ISP Workshop Lab
6. Configure the ethernet of each router at the IXP. The ethernet interfaces connected to the IXP
should be configured appropriately for a public connection. Review the IOS Essentials materials
and the IXP presentation. The configuration for Router 14 might be:
interface fastethernet 0
description Exchange Point LAN
ip address 172.16.0.40 255.255.255.0
no ip directed-broadcast
no ip proxy-arp
no ip redirects
!
If you are unclear as to what any of the configuration lines do, please ask the lab instructor.
7. Configuring BGP on the routers. Next, eBGP needs to be set up on the routers. Create a peer-
group and apply that peer-group to each eBGP neighbour. A sample configuration for Router13
might be:
ip prefix-list myprefixes permit 10.13.0.0/20

ip prefix-list peer101 permit 10.1.0.0/20
..
!
router bgp 113
no synchronization
network 10.13.0.0 mask 255.255.240.0
neighbor ixp-peers peer-group
neighbor ixp-peers remove-private-AS
neighbor ixp-peers prefix-list myprefixes out
neighbor ixp-peers route-map set-local-pref in
neighbor <router1> remote-as 101
neighbor <router1> description Peering with AS101
neighbor <router1> peer-group ixp-peers
neighbor <router1> prefix-list peer101 in
..
no auto-summary
!
!
The configurations for the other routers will be similar to this one. All router teams will have done
sufficient BGP configuration throughout this workshop to extrapolate from the above examples. If
in any doubt, ask the lab demonstrator for assistance.
Note the prefix-lists: there is a per-peer inbound prefix-list. Some service providers only filter
ASes – that has inherent dangers, and does not prevent against inbound leaking of prefixes
incorrectly originated by the peer AS. But only filtering on prefixes doesn’t scale, especially in
larger IXPs with large participating service providers as they are frequently adding to the prefixes
they announce. The Internet Routing Registry is usually used to solve this problem.
3
8. Connectivity Test. Check connectivity throughout the IXP network. Each router team should be
able to see all the other routers at the IXP. When you are satisfied that BGP is working correctly,
try running traceroutes to check the paths being followed.
Checkpoint #2: Once the BGP configuration has been completed, check the routing table and ensure
that you have complete reachability over the entire network. If there are any problems, work with the
other router teams to resolve those.
9. Set up passwords on the eBGP sessions. Negotiate with each ASN a password which you can
use on your BGP session with them. And then agree to cut the eBGP session over to using
passwords such that the eBGP session does not fall over due to password mismatches (as in
Module 6).
router bgp 113

neighbor <router14> password peer114
!
10. Set up eBGP session with Router 15. The lab instructors will have now configured Router 15 to
be a route collector. This is a router which collects all the routes available at the IX. It serves
purpose other than to be an information repository showing how many routes are available at the
IX – quite often the IXP management will operate such a router, connected to a Looking Glass
web interface, to increase the marketing value of the IX. The more peers who get attracted by the
routes available at the IX, the greater the value proposition the IX is to the rest of the members. It’s
in everyone’s interest to peer with the router collector:
router bgp 113

...
neighbor 172.16.0.254 description eBGP with the IX Route Collector
neighbor 172.16.0.254 password cisco
neighbor 172.16.0.254 prefix-list deny-all in
neighbor 172.16.0.254 prefix-list myprefixes out
...
!
ip prefix-list deny-all deny 0.0.0.0/0 le 32
Notice that the route collector is running in a private AS – there isn’t really any need for it to use a
public AS as the Collector does not need to be directly visible outside of the IXP.
Note also the inbound prefix filter blocking all prefixes on the eBGP session with the Route
Collector. The Collector will not advertise any prefixes, by design. However, ISPs should never
trust any other AS or its operator, so the inbound prefix filter is provided for safety, just in case of
problems at the Route Collector.
12. Completed! The IXP is now complete, up and running. The lab instructors will log into the route
collector and show the prefixes visible. All 14 announcements should be clearly seen in the output
of sh ip bgp on the route collector.
4
ISP Workshop Lab
Checkpoint #3: Compare your BGP routing table with that you see on the route collector. If you have
missing prefixes, or some other problems, ask the lab demonstrators.
STOP AND WAIT HERE
13. Route Server. Some Internet Exchange Points operate a device called a route server. This is
basically a route collector but with the added feature that it announces the prefixes it has learned to
all the ISPs who are connected to it. The lab instructors will now modify the route collector’s
configuration so that it announces prefixes to the ASNs participating in the IXP.
14. Participating with the Route Server. Each ISP should now modify their BGP peering with the
Route Collector so that it can receive prefixes from that router. This means modifying or simply
removing the inbound prefix filter, for example as below:
router bgp 113

...
neighbor 172.16.0.254 description eBGP with the IX Route Collector
neighbor 172.16.0.254 prefix-list route-server in
...
!
ip prefix-list route-server permit 0.0.0.0/0 le 32
...
15. Bi-lateral peering sessions. Each team will now see that they have BGP prefixes learned by direct
or bi-lateral peerings as well as through the route-server. The prefixes learned via the route-server
have AS65534 in the AS-PATH – the AS is longer, so the bi-lateral peering is preferred. Each
router team will now remove all their bi-lateral peerings, leaving only the peering with the route-
server in place.
router bgp 113

no neighbor <router1>
no neighbor <router2>
. . .
Once this is done, the BGP table will only contain prefixes learned via the route-server.
16. Summary. Route servers are more normally used to assist with the scaling of larger IXPs.
Managing a large number of eBGP sessions can be quite challenging, especially for ISPs who have
routers which are less able to handle large numbers of eBGP sessions. To reduce the administrative
overhead, they learn most if not all their peer prefixes via the route-server instead. In the industry
today it is normal for ISPs at an Exchange Point to set up bi-lateral peerings with ISPs who do not
participate in the route-server, and set up peerings with the route-server for those who do
participate. Not ever ISP chooses to participate in the route-server, as was discussed in the
presentation.
Checkpoint #3: Compare your BGP routing table with that you see on the route collector. If you have
missing prefixes, or some other problems, ask the lab demonstrators.
5
ISP Workshop Lab
Module 11 – Advanced Router Configuration

Objective: Create a basic physical lab interconnection with two autonomous systems. Each AS
should use ISIS, iBGP and eBGP appropriately to construct a working network.
Prerequisites: Basic ISP Workshop (at least Modules 1 and 6 to 8)
The following will be the common topology used.
1
Tuesday, July 29, 2014
Lab Notes
The purpose of this module is to construct the workshop lab and serve as a reminder of the basic
principles of building a network, introducing an IGP, properly function iBGP, and the basics of eBGP:
verified.
should be set up. Note that Router15 is the Workshop Instructor’s router and it will be used at
various instances throughout the workshop.
then external BGP.

Lab Exercise
The following list is typical for what needs to be done to bring up the lab configuration:
Router3, etc. Documentation and labs will also refer to Router1 as R1.
hostname Router1
2. Set Domain name and turn Off Domain Name Lookups. Cisco routers will always try to look
up the DNS for any name typed on the command line. You can see this when doing a trace on a
router with no DNS server or a DNS server with no in-addr.arpa entries for the IP addresses.
Unless the Workshop Instructor specifically tells you that there is a nameserver configured for the
lab, we will turn this lookup off for the labs to speed up traceroutes. We will set a domain-name
though, as this is required to set up SSH support later in the lab.
no ip domain-lookup
2
ISP Workshop Lab
line con 0
line vty 0 4
no ip source-route
username isplab secret lab-PW

enable secret lab-PW
service password-encryption
There is the temptation to simply have a username of cisco and password of cisco as a lazy
solution to the username/password problem. Under no circumstances must any service provider
operator ever use easily guessable passwords as these on their live operational network1.
6. Enabling login access for other teams. In order to let other teams telnet into your router, you
need to configure a password for all virtual terminal lines.
aaa new-model
aaa authentication login default local
aaa authentication enable default enable
1
3
router by default will display system logs on the router console. We will retain this functionality
for the workshop, but it is undesirable for Internet operational routers, as the console is a 9600
baud connection, and can place a high processor interrupt load at the time of busy traffic on the
network. However, the router logs can also be recorded into a buffer on the router – this takes no
interrupt load and it also enables to operator to check the history of what events happened on the
router. In a future module, the lab will configuration the router to send the log messages to a
SYSLOG server.
which records all logs in a 8192byte buffer set aside on the router. Note that outside this workshop
environment, console logging should normally be disabled as such:
no logging console
8. CIDRise the router. Make sure the router is configured for CIDR. These two commands are now
default in 12.0S and from 12.3 and more recent releases, but it is good practice to check just in
case:
ip subnet-zero
ip classless
9. Set up timestamps for all logs on the router. 12.0S has made basic timestamping on the logs the
default but ISPs should enable the complete detail on their logs as follows:

Refer to IOS Essentials or the router’s on-line help system if you have forgotten what these
options mean.
10. Set up a login banner. Next, set up a login banner. Use an appropriate greeting – consult IOS
Essentials document for appropriate and inappropriate greetings. If you use an inappropriate
greeting, expect the lab instructors to ask you to change it. For example
banner login ^
BGP Multihoming Workshop Lab
^
11. Using SSH for router access. If the router software images have SecureShell support available in
them, this step will enable SSH support for access to and from the routers. You can recognise an
image which has SSH in it as it will have either “k4” or “k9” in the name, signifying 3DES crypto
support; for example, c2801-ik9s-mz.124-8 is a crypto IP PLUS image for the 2801 series routers.
which will generate an RSA crypto key for the router. When the router prompts for key size,
ensure that it is at least 768 bits so that SSH version 2 can be used. This key will be automatically
4
ISP Workshop Lab
stored in a file in NVRAM – this file is not readable by any user on the router. Next, explicitly set
SSH to version 2:
ip ssh version 2
SSH is now available for use on the router.
Note: Don’t use Secure Shell version 1 any more – it is no longer considered secure.
12. Set SSH source interface. Any network device, by default, will use the egress interface as the
source IP address for outgoing traffic originated by the router. But most connectivity and
monitoring of routers is targeted on the loopback interface. So we will now change the source
address for SSH traffic to be that of the loopback interface address.
ip ssh source-interface loopback 0
13. Tidy up the vty and console interface configuration. In the real world we’d now add access-lists
to the vty ports on the router. However, this lab is not connected to any external network or the
Internet so these will not be required. However, we need to make some changes to the defaults.
Basically, only ssh will be the supported mechanism to connect to the routers, and only ssh will be
the permitted mechanism to connect from one router to the next. (Most ISPs completely disable
telnet support on their routers – industry best practice considers telnet an obsolete and archaic
protocol, ill-suited for use for management of public network infrastructure equipment.)
line vty 0 4
transport input ssh
transport output ssh
14. Create a loopback interface. Loopback interfaces will be used in this workshop for many things.
They are an essential and fundamental requirement for any ISP backbone:
description Loopback Interface for RouterXX
15. Disable pad, finger and bootp servers. The pad, finger and bootp servers are running by default
in IOS. These should be disabled on any Internet router. Finger is a security risk, bootp and pad are
simply unnecessary.
no service pad
no ip finger
no ip bootp
16. Remove unneeded SNMP configuration. IOS versions prior to 12.0S install a default SNMP
configuration when the router first starts with an unconfigured NVRAM. As we will not be using
SNMP to access the routers in the workshop, check if the SNMP configuration is there and remove
it if it is. (Unless configured correctly SNMP is a security risk in the Internet.) Example:
no snmp-server community public
17. Disable built-in http server. IOS now comes with a built-in http server which is enabled by
default (assists with simple installation for non-technical users). This server is completely useless
5
for ISPs, and being activated by default is a serious security risk. Disable it before the router
receives any IP address configuration:
no ip http server
no ip https server
18. Enable CEF. While this is now enabled by default on most Cisco IOS router platforms (from IOS
12.4 onwards), it is still worth including the command in your configuration template.
ip cef
Entering the sh cef interface command will show the status of CEF on the router interfaces. And
the command sh ip cef will show the forwarding table – currently empty.
ip tcp path-mtu-discovery
Modules 1 and 6 and will see it again later on in this module.
The side effect from doing this is that the router has to send a response that the packet has reached
a destination – this response is that the destination is “unreachable”. Each packet generates one
response – an ICMP unreachable message. For a stream of packets, this can introduce some burden
on the router CPU, so many ISPs configure the Null0 interface to not send ICMP unreachables –
6
ISP Workshop Lab
the packets end up at the Null0 interface and are silently discarded. This is much lighter on the
router CPU.
interface Null 0
no ip unreachables
21. Saving the configuration. With the basic configuration in place, save the configuration by using
“write memory”. Then log off the router by typing exit, and then log back in again. Notice how the
login sequence has changed, prompting for a “username” and “password” from the user. Don’t
forget to frequently save the configuration to NVRAM after each configuration change.
IMPORTANT NOTE: Each router team is strongly recommended to make a copy of the basic
router configuration at this stage. It will be assumed throughout this workshop that the above
configuration will ALWAYS be present on the router. If it is not, each router team will be
requested to restore it as a matter of urgency.
router either on the worksheet on the end of this hand out, or own your own laptop, or on the
classroom tftp server if it is available. It will be required again several times throughout this
workshop.
22. IP Addresses. Each AS is assigned a block of IP addresses.
AS10 10.10.0.0/19 AS20 10.20.0.0/19
Decide among your team what the addressing plan for you AS should be.
Hint One: point to point links only require /30 blocks.
Hint Two: loopbacks only require a /32 host address.
Hint Three: number your backbone sequentially, from either the start or finish of the range.
Suggestion:
10.10.0.0/24 10.10.1.0 10.10.31.255
Loopbacks Infrastructure addresses Customer assigned addresses
Note: When the IP addresses are assigned, they MUST be annotated on the WHITE-BOARD at
the front of the workshop room. A large network map will have been drawn on the white-board –
all the IP address assignments need to be annotated there so that other Router Teams can document
wrong. Don’t ignore the problem – it may not go away.
7
24. Create vty filters. Set up filters on the vty interfaces restricting vty access to your router to those
addresses you would like to permit. For the purpose of this lab, even though it is not connected to
the Internet, set up filters so that only the address space in the lab has access. The following
example shows an extended access-list (standard and numbered access-lists can be used too):
ip access-list extended ipv4-vty

remark VTY filters
permit ip 10.10.0.0 0.0.31.255 any
permit ip 10.20.0.0 0.0.31.255 any
deny ip any any log
!
line vty 0 4
access-class ipv4-vty in
25. ISIS within the same AS. Each router Team should enable ISIS on their router. All the routers in
one AS will be in ISIS level 2. The NET should be 49.0001.x.x.x.x.00, where x.x.x.x is the
loopback IP address. For this lab, serial interfaces need to have metric 20, fastethernet interfaces
need to have metric 2. Remember to use wide metrics, and don’t forget to mark the loopback
interface as a passive interface – for example:
router isis as20

net 49.0001.0100.2000.0001.00
is-type level-2-only
metric-style wide level-2
log-adjacency-changes all
passive-interface Loopback0
!
ip router isis as20
isis circuit-type level-2 only
isis metric 20 level-2
!
26. Setting ISIS default metric for all interfaces. Many network operators recommending setting the
default ISIS metric away from the IOS value of 10 to something much larger, for example,
100000. (See the Advanced ISIS presentation.) This avoids situations where a new router would be
introduced into the network and becomes the best transit path before the operators has fully
configured all the links to it from other routers. Here is a configuration example:
router isis as20

metric 100000
!
27. DMZ between AS10 and AS20. ISIS must NOT run on the demarcation links between AS10 and
AS20. So Routers 6, 7, 8 and 9 must configure the serial links between each other to be passive.
This is a very important point, and a mistake frequently made by many ISPs. Also, do not put any
ip isis configuration on external facing interfaces – again see iBGP discussion at step 35.
Router 6 should have two adjacencies only – with Router 4 and 5. Router 7 should have one
adjacency only – with Router 9. And so on.
28. Point-to-Point Ethernet Links. Because Ethernet is more and more commonly used for point to
point links within the PoP and on the WAN, we want to try and make the establishment and
failover of ISIS just a little more efficient on these links. ISIS can be configured in point to point
mode on these sorts of connections. This mode prevents the election of a DIS, prevents flooding
8
ISP Workshop Lab
on the LAN (pointless as there is only one other device), and speeds up the SPF computations. To
configure point to point mode, simply go to the Ethernet interface in question and configure:

isis network point-to-point
!
29. Intra Area Authentication – Part 1. ISIS supports neighbour authentication. Even though ISIS
runs alongside IP on the wire, some ISPs still consider neighbour authentication to be necessary.
Authentication helps prevent the introduction of improperly configured or unintended equipment.
This first step will be to set up the authentication key chains:
key chain lab-key

key 1
key-string cisco
This sets up a key chain called lab-key with the key string “cisco”. Obviously on a production
network a key other than “cisco” should be used!
30. Intra Area Authentication – Part 2. Now that the key chain has been set up, the final step is to
actually configure the authentication on the interface. MD5 encryption should be used rather than
exchanging keys in plain text – to do this, use the md5 sub-interface command.
An example configuration might be:
interface fastethernet0/0
isis circuit-type level-2 only
isis authentication mode md5 level-2
isis authentication key-chain lab-key level-2
Notice now that the ISIS adjacencies do not come up unless the neighbouring router has also
entered the same configuration and key. Notice also how the ISIS adjacencies were reset as the
31. ISIS metrics and dependencies on BGP. One very important feature for an ISP network is to
ensure that a router starting up after maintenance does not advertise itself as a best transit path
through the backbone until iBGP is up and operational. This protects the network from blackholing
transit traffic, a common customer complaint during ISP maintenance.
The way to ensure this is for when ISIS to start, it advertises all networks known to it with the
maximum possible metric – this makes them highly undesirable. Once iBGP is up and running (at
least one adjacency is established and converged), the ISIS metric is then returned to normal.
router isis as20

set-overload-bit on-startup wait-for-bgp
32. Ping Test #2. Ping all loopback interfaces in your AS. They should all respond. This will ensure
the ISIS IGP is connected End-to-End. If there are problems, use the following commands to help
determine the problem:
9
show clns neighbor : see a list of CLNS-IS neighbors that the router sees
show clns interface : see if ISIS is configured and see the IS type
show isis database : see ISIS link state database that the router has learned
classroom tftp server if it is available.
STOP AND WAIT HERE
33. BGP distance. Before we set up iBGP with our neighbours in our AS, we need to do some basic
preparation on the router. The IOS defaults are not optimised for Service Provider networks, so
before we bring up BGP sessions, we should set the defaults that we require.
for ISIS is 115. This means that there is a potential for a prefix learned by eBGP to override the
identical prefix carried by ISIS. Recall from the Routing presentation that there is a distinct
separation between BGP and ISIS processes – prefixes present in ISIS will never be found in BGP,
and vice-versa. To protect against accidents2, the eBGP distance is set to 200 also. The command
to do this is the bgp distance subcommand, syntax is:
Note: This should be included in all future BGP configurations in this workshop. Set the BGP
protocol distance so that BGP is always less preferred than any IGP. So:
router bgp 10
distance bgp 200 200 200
34. Passwords on BGP sessions. It is now considered very good practice to use passwords on the
BGP sessions on the router. When BGP is set up in the next step, don’t forget to include a
password on the BGP peering.
The password used for this module will be cisco – obviously on a real operational network
operators will use a password which follows their normal password rules, and not something
which is easily guessable. An example configuration might be:
router bgp 10
Note: Passwords should be included in all future BGP configurations in this workshop.
35. Configuring next-hop-self on iBGP Neighbours. So that BGP has a valid next-hop for external
destinations, we introduce the next-hop-self BGP configuration. This changes the iBGP default
by replacing the next-hop address for external sites from that of the external neighbour address to
2
10
ISP Workshop Lab
the loopback address of the local router. The local router knows how to get to the external
destinations because it is connected to the LAN that leads there – the rest of the network internal to
the AS is told simply to go via this router. Note that because we do this, we no longer need to
quote the external point to point link in our ISIS (or OSPF) configuration – see steps 24 & 26
earlier. For example:
router bgp 10
neighbor 1.2.3.4 next-hop-self
Note that the use of next-hop-self on all iBGP sessions is considered industry best practice,
and its use from now on in the workshop is strongly recommended.
36. Configuring iBGP Neighbours. Configure iBGP peers within each autonomous system. Use a
full iBGP mesh. Don’t forget that iBGP peering is configured to be between the loopback
interfaces on the routers. Also, it is good practice to use a peer-group. For example:
router bgp 10
neighbor ibgp-peers description iBGP v4 peergroup
..etc..
connection to troubleshoot the problem. Note: get into the habit of using peer-groups and
configuring them fully, including the “send-community” directive (Cisco IOS by default does not
send BGP communities to BGP neighbours). This workshop makes extensive use of communities,
and making them part of your configuration is good practice.
Note: Router6 should also include the network connecting to Router15 in the iBGP configuration.
This is so that the network connected to Router15 can be accessed – it has the DNS server and
NTP server located on it.
37. Add Prefixes to BGP. Each Router Team will advertise the CIDR block assigned to them via
BGP. AS10 would advertise 10.10.0.0/19 and AS20 would advertise 10.20.0.0/19:
router bgp 10
no synchronization
no auto-summary
network 10.10.0.0 mask 255.255.224.0
!
ip route 10.10.0.0 255.255.224.0 null0
Don’t forget the static route to Null0. This ensures that the prefix has an entry in the routing table,
and therefore will appear in the BGP table. Also, don’t forget to disable synchronisation and auto-
summarisation – these are also mandatory requirements for ISP routers connecting to the Internet.
11
(Note that a distance of 250 could be applied to the static route to ensure that routing protocols
announcing this exact prefix will override the static (if this is required/desired).)
38. Enable new format of BGP communities. It is also worth getting into the habit of changing the
BGP community format from the default 32-bit integer to colon separated 16-bit integers, as used
in RFC1998. Example:
39. Configure eBGP peering. Now that iBGP is functioning, it is time to configure eBGP. External
BGP will be set up between AS10 and AS20, specifically between Routers 6 and 8, and Routers 7
and 9 only. The remaining lab teams should monitor the BGP table they see on their routers.
Firstly, agree on what IP addresses should be used for the point-to-point links between the ASes.
Put the /30 networks used for the DMZ links into OSPF (network statement and passive interface).
Then configure eBGP between the router pairs, for example:
router bgp 10
neighbor 10.20.2.2 description eBGP with RouterXX
Use the BGP show commands to ensure that you are receiving prefixes from your neighbouring
AS.
40. eBGP filters. The final step of setting up the eBGP sessions is to apply filters on the BGP
sessions. The teams operating Routers 6, 7, 8 and 9 should now set up filters to allow their
neighbour AS prefixes into their network; and set up filters to allow their own AS prefixes out of
their network. For example:
router bgp 10
neighbor 10.20.2.2 prefix-list AS20peer-in in
neighbor 10.20.2.2 prefix-list AS20peer-out out
!
ip prefix-list AS20peer-in permit 10.20.0.0/19
ip prefix-list AS20peer-out permit 10.10.0.0/19
!
This only allows 10.20.0.0/19 in from an AS20 eBGP neighbour, and allows 10.10.0.0/19 out to an
AS20 eBGP neighbour. Don’t forget that you will also need to allow the address of the NTP
timeserver through the BGP filters as well.
41. BGP Communities on eBGP sessions. Network operators only send communities to their eBGP
peers if they have a policy requirement to do so. Cisco IOS also does not send communities on the
eBGP sessions by default, so we will now add this to eBGP peerings. The teams operating Routers
6, 7, 8 and 9 should add the “send-community” directive to their eBGP peerings. For example:
router bgp 10
!
12
ISP Workshop Lab
42. Check the network paths and the routing table. Run traceroutes between your router and other
routers in the classroom. Ensure that all routers are reachable. If any are not, work with the other
router teams to establish what might be wrong. Make sure that you can see Router15. The lab
instructor will have written the addresses and network up on the whiteboard. (The network is
192.168.1.0/24, the address of Router6 on that LAN is 192.168.1.254, and the address of Router15
is 192.168.1.1.)
43. Saving the configuration. For software releases from 12.0 onwards, the commands to save the
configuration are of the format copy <source> <destination> where the source and destinations
can be any of the following options: ftp, lex, null, nvram, rcp, running-config, startup-config,
system, tftp. To save the configuration to the TFTP server, use the “copy system:/running-config
tftp:” command sequence. If the TFTP server is unreachable, “.”s followed by an error message
will be displayed rather than “!”s. (Note that the “write net” command of earlier releases is still
supported but may be removed at a future release.)
Router1#copy system:running-config tftp:

!!
Router1#
44. Summary. This module has covered most of the fundamental configuration topics required to
construct an ISP network. It has covered basic router configuration, configuration Best Current
Practices, ISIS configuration, iBGP configuration, and finally simple eBGP configuration. No
routing policy has been implemented. Each Router team is strongly recommended to make a
copy of their configuration, as most of the configuration concepts will be required
throughout the remainder of the workshop.
13
ISP Workshop Lab
Module 11 – Advanced Router Configuration

Objective: Create a basic physical lab interconnection with two autonomous systems. Each AS
should use OSPF, iBGP and eBGP appropriately to construct a working network.
Prerequisites: Basic ISP Workshop (at least Modules 1 and 6 to 8)
1
Lab Notes
The purpose of this module is to construct the workshop lab and serve as a reminder of the basic
principles of building a network, introducing an IGP, properly function iBGP, and the basics of eBGP:
verified.
should be set up. Note that Router15 is the Workshop Instructor’s router and it will be used at
various instances throughout the workshop.
protocols. And start with the IGP (OSPF is chosen for this workshop). There is no purpose to
building BGP while the chosen IGP (in this case OSPF) is not functioning properly. BGP relies on
OSPF to find its neighbours and next hops, and an improperly or non-functioning OSPF will result
in much time wasted attempting to debug routing problems.
then external BGP.

Lab Exercise
Router3, etc. Documentation and labs will also refer to Router1 as R1.
hostname Router1
2. Set Domain name and turn Off Domain Name Lookups. Cisco routers will always try to look
up the DNS for any name typed on the command line. You can see this when doing a trace on a
router with no DNS server or a DNS server with no in-addr.arpa entries for the IP addresses.
Unless the Workshop Instructor specifically tells you that there is a nameserver configured for the
lab, we will turn this lookup off for the labs to speed up traceroutes. We will set a domain-name
though, as this is required to set up SSH support later in the lab.
no ip domain-lookup
2
ISP Workshop Lab
line con 0
line vty 0 4
no ip source-route
username isplab secret lab-PW

enable secret lab-PW
service password-encryption
There is the temptation to simply have a username of cisco and password of cisco as a lazy
solution to the username/password problem. Under no circumstances must any service provider
operator ever use easily guessable passwords as these on their live operational network1.
6. Enabling login access for other teams. In order to let other teams telnet into your router, you
need to configure a password for all virtual terminal lines.
aaa new-model
aaa authentication login default local
aaa authentication enable default enable
1
3
router by default will display system logs on the router console. We will retain this functionality
for the workshop, but it is undesirable for Internet operational routers, as the console is a 9600
baud connection, and can place a high processor interrupt load at the time of busy traffic on the
network. However, the router logs can also be recorded into a buffer on the router – this takes no
interrupt load and it also enables to operator to check the history of what events happened on the
router. In a future module, the lab will configuration the router to send the log messages to a
SYSLOG server.
which records all logs in a 8192byte buffer set aside on the router. Note that outside this workshop
environment, console logging should normally be disabled as such:
no logging console
8. CIDRise the router. Make sure the router is configured for CIDR. These two commands are now
default in 12.0S and from 12.3 and more recent releases, but it is good practice to check just in
case:
ip subnet-zero
ip classless
9. Set up timestamps for all logs on the router. 12.0S has made basic timestamping on the logs the
default but ISPs should enable the complete detail on their logs as follows:

Refer to IOS Essentials or the router’s on-line help system if you have forgotten what these
options mean.
10. Set up a login banner. Next, set up a login banner. Use an appropriate greeting – consult IOS
Essentials document for appropriate and inappropriate greetings. If you use an inappropriate
greeting, expect the lab instructors to ask you to change it. For example
banner login ^
BGP Multihoming Workshop Lab
^
11. Using SSH for router access. If the router software images have SecureShell support available in
them, this step will enable SSH support for access to and from the routers. You can recognise an
image which has SSH in it as it will have either “k4” or “k9” in the name, signifying 3DES crypto
support; for example, c2801-ik9s-mz.124-8 is a crypto IP PLUS image for the 2801 series routers.
which will generate an RSA crypto key for the router. When the router prompts for key size,
ensure that it is at least 768 bits so that SSH version 2 can be used. This key will be automatically
4
ISP Workshop Lab
stored in a file in NVRAM – this file is not readable by any user on the router. Next, explicitly set
SSH to version 2:
ip ssh version 2
SSH is now available for use on the router.
Note: Don’t use Secure Shell version 1 any more – it is no longer considered secure.
12. Set SSH source interface. Any network device, by default, will use the egress interface as the
source IP address for outgoing traffic originated by the router. But most connectivity and
monitoring of routers is targeted on the loopback interface. So we will now change the source
address for SSH traffic to be that of the loopback interface address.
ip ssh source-interface loopback 0
13. Tidy up the vty and console interface configuration. In the real world we’d now add access-lists
to the vty ports on the router. However, this lab is not connected to any external network or the
Internet so these will not be required. However, we need to make some changes to the defaults.
Basically, only ssh will be the supported mechanism to connect to the routers, and only ssh will be
the permitted mechanism to connect from one router to the next. (Most ISPs completely disable
telnet support on their routers – industry best practice considers telnet an obsolete and archaic
protocol, ill-suited for use for management of public network infrastructure equipment.)
line vty 0 4
transport input ssh
transport output ssh
14. Create a loopback interface. Loopback interfaces will be used in this workshop for many things.
They are an essential and fundamental requirement for any ISP backbone:
description Loopback Interface for RouterXX
15. Disable pad, finger and bootp servers. The pad, finger and bootp servers are running by default
in IOS. These should be disabled on any Internet router. Finger is a security risk, bootp and pad are
simply unnecessary.
no service pad
no ip finger
no ip bootp
16. Remove unneeded SNMP configuration. IOS versions prior to 12.0S install a default SNMP
configuration when the router first starts with an unconfigured NVRAM. As we will not be using
SNMP to access the routers in the workshop, check if the SNMP configuration is there and remove
it if it is. (Unless configured correctly SNMP is a security risk in the Internet.) Example:
no snmp-server community public
17. Disable built-in http server. IOS now comes with a built-in http server which is enabled by
default (assists with simple installation for non-technical users). This server is completely useless
5
for ISPs, and being activated by default is a serious security risk. Disable it before the router
receives any IP address configuration:
no ip http server
no ip https server
18. Enable CEF. While this is now enabled by default on most Cisco IOS router platforms (from IOS
12.4 onwards), it is still worth including the command in your configuration template.
ip cef
Entering the sh cef interface command will show the status of CEF on the router interfaces. And
the command sh ip cef will show the forwarding table – currently empty.
ip tcp path-mtu-discovery
Modules 1 and 6 and will see it again later on in this module.
The side effect from doing this is that the router has to send a response that the packet has reached
a destination – this response is that the destination is “unreachable”. Each packet generates one
response – an ICMP unreachable message. For a stream of packets, this can introduce some burden
on the router CPU, so many ISPs configure the Null0 interface to not send ICMP unreachables –
6
ISP Workshop Lab
the packets end up at the Null0 interface and are silently discarded. This is much lighter on the
router CPU.
interface Null 0
no ip unreachables
21. Saving the configuration. With the basic configuration in place, save the configuration by using
“write memory”. Then log off the router by typing exit, and then log back in again. Notice how the
login sequence has changed, prompting for a “username” and “password” from the user. Don’t
forget to frequently save the configuration to NVRAM after each configuration change.
IMPORTANT NOTE: Each router team is strongly recommended to make a copy of the basic
router configuration at this stage. It will be assumed throughout this workshop that the above
configuration will ALWAYS be present on the router. If it is not, each router team will be
requested to restore it as a matter of urgency.
classroom tftp server if it is available. It will be required again several times throughout this
workshop.
22. IP Addresses. Each AS is assigned a block of IP addresses.
AS10 10.10.0.0/19 AS20 10.20.0.0/19
Decide among your team what the addressing plan for you AS should be.
Hint One: point-to-point links only require /30 blocks.
Hint Two: loopbacks only require a /32 host address.
Hint Three: number your backbone sequentially, from either the start or finish of the range.
Suggestion:
10.10.0.0/24 10.10.1.0 10.10.31.255
Note: When the IP addresses are assigned, they MUST be annotated on the WHITE-BOARD at
the front of the workshop room. A large network map will have been drawn on the white-board –
all the IP address assignments need to be annotated there so that other Router Teams can document
wrong. Don’t ignore the problem – it may not go away.
7
24. Create vty filters. Set up filters on the vty interfaces restricting vty access to your router to those
addresses you would like to permit. For the purpose of this lab, even though it is not connected to
the Internet, set up filters so that only the address space in the lab has access.
ip access-list extended ipv4-vty

remark VTY filters
permit ip 10.10.0.0 0.0.31.255 any
permit ip 10.20.0.0 0.0.31.255 any
deny ip any any log
!
line vty 0 4
access-class ipv4-vty in
25. OSPF within the same AS. Each router Team should enable OSPF on their router. The OSPF
process identifier should be the same as your AS number. All the routers in one AS will be in
OSPF Area 0. Make sure that you run OSPF on every physical interface, but mark those where you
do not expect to see adjacencies as passive – this is most easily done by marking all interfaces
passive by default, and activating those where you expect to see an adjacency. Do not run OSPF
on external facing interfaces (those connecting to other autonomous networks) – see discussion for
BGP at step 34. Don’t forget to mark the loopback interface as a passive interface – for example:
router ospf 41
passive-interface default
no passive-interface serial 0/0
log-adjacency-changes
!
ip ospf 41 area 0
!
ip ospf 41 area 0
!
Note: For releases prior to 12.2, the log-adjacency-changes directive requires ospf to be specified
first.
Note: If using an IOS prior to 12.4, then the above configuration syntax is not supported and
router teams will have to use the older network statement configuration – see the OSPF
presentation for more details.
26. DMZ between AS10 and AS20. OSPF must NOT run on the demarcation links between AS10
and AS20. So Routers 6, 7, 8 and 9 must configure the serial links between each other to be
passive. This is a very important point, and a mistake frequently made by many ISPs. Also, do not
put an ip ospf command on or associate a network statement with external facing interfaces –
again see iBGP discussion at step 34. Router 6 should have two adjacencies only – with Router 4
and 5. Router 7 should have one adjacency only – with Router 9. And so on.
27. Point-to-Point Ethernet Links. Because Ethernet is more and more commonly used for point to
point links within the PoP and on the WAN, we want to try and make the establishment and
failover of OSPF just a little more efficient on these links. OSPF can be configured in point to
point mode on these sorts of connections. This mode prevents the election of a designated and
backup designated router, prevents flooding on the LAN (pointless as there is only one other
8
ISP Workshop Lab
device), and speeds up the SPF computations. To configure point to point mode, simply go to the
Ethernet interface in question and configure:

ip ospf network point-to-point
!
28. Intra Area Authentication – Part 1. OSPF supports router authentication within areas. This is
very important inside ISP networks to prevent the introduction of improperly configured or
unintended equipment.
This first step will enable authentication for the OSPF area the router is in. To do this, the area N
authentication message-digest command is used. An example configuration might be:
router ospf 41
!
29. Intra Area Authentication – Part 2. Now that support for authentication in the area has been
An example configuration might be:
router ospf 41
!
!
!
!
9
30. OSPF metrics and dependencies on BGP. One very important feature for an ISP network is to
ensure that a router starting up after maintenance does not advertise itself as a best transit path
through the backbone until iBGP is up and operational. This protects the network from blackholing
transit traffic, a common customer complaint during ISP maintenance.
The way to ensure this is for when OSPF to start, it advertises all networks known to it with the
maximum possible metric – this makes them highly undesirable. Once iBGP is up and running (at
least one adjacency is established and converged), the OSPF metric is then returned to normal.
router ospf 20
max-metric router-lsa on-startup wait-for-bgp
31. Ping Test #2. Ping all loopback interfaces in your AS. They should all respond. This will ensure
the OSPF IGP is connected End-to-End. If there are problems, use the following commands to
help determine the problem:

show ip ospf : see general OSPF information
show ip ospf interface : Check if OSPF is enabled on all intended interface
show ip ospf neighbor : see a list of OSPF neighbours that the router sees
STOP AND WAIT HERE
32. BGP distance. Before we set up iBGP with our neighbours in our AS, we need to do some basic
preparation on the router. The IOS defaults are not optimised for Service Provider networks, so
before we bring up BGP sessions, we should set the defaults that we require.
for OSPF is 110. This means that there is a potential for a prefix learned by eBGP to override the
identical prefix carried by OSPF. Recall from the Routing presentation that there is a distinct
separation between BGP and OSPF processes – prefixes present in OSPF will never be found in
BGP, and vice-versa. To protect against accidents2, the eBGP distance is set to 200 also. The
command to do this is the bgp distance subcommand, syntax is:
Note: This should be included in all future BGP configurations in this workshop. Set the BGP
protocol distance so that BGP is always less preferred than any IGP. So:
router bgp 10
2
10
ISP Workshop Lab
33. Passwords on BGP sessions. It is now considered very good practice to use passwords on the
BGP sessions on the router. When BGP is set up in the next step, don’t forget to include a
password on the BGP peering.
The password used for this module will be cisco – obviously on a real operational network
operators will use a password which follows their normal password rules, and not something
which is easily guessable. An example configuration might be:
router bgp 10
Note: Passwords should be included in all future BGP configurations in this workshop.
34. Configuring next-hop-self on iBGP Neighbours. So that BGP has a valid next-hop for external
destinations, we introduce the next-hop-self BGP configuration. This changes the iBGP default
by replacing the next-hop address for external sites from that of the external neighbour address to
the loopback address of the local router. The local router knows how to get to the external
destinations because it is connected to the LAN that leads there – the rest of the network internal to
the AS is told simply to go via this router. Note that because we do this, we no longer need to
quote the external point to point link in our OSPF (or ISIS) configuration – see steps 24 & 26
earlier. For example:
router bgp 10
neighbor 1.2.3.4 next-hop-self
Note that the use of next-hop-self on all iBGP sessions is considered industry best practice,
and its use from now on in the workshop is strongly recommended.
35. Configuring iBGP Neighbours. Configure iBGP peers within each autonomous system. Use a
full iBGP mesh. Don’t forget that iBGP peering is configured to be between the loopback
interfaces on the routers. Also, it is good practice to use a peer-group. For example:
router bgp 10
neighbor ibgp-peers description iBGP v4 peergroup
..etc..
connection to troubleshoot the problem. Note: get into the habit of using peer-groups and
configuring them fully, including the “send-community” directive (Cisco IOS by default does not
send BGP communities to BGP neighbours). This workshop makes extensive use of communities,
and making them part of your configuration is good practice.
11
Note: Router6 should also include the network connecting to Router15 in the iBGP configuration.
This is so that the network connected to Router15 can be accessed – it has the DNS server and
NTP server located on it.
36. Add Prefixes to BGP. Each Router Team will advertise the CIDR block assigned to them via
BGP. AS10 would advertise 10.10.0.0/19 and AS20 would advertise 10.20.0.0/19:
router bgp 10
no synchronization
no auto-summary
network 10.10.0.0 mask 255.255.224.0
!
ip route 10.10.0.0 255.255.224.0 null0
Don’t forget the static route to Null0. This ensures that the prefix has an entry in the routing table,
and therefore will appear in the BGP table. Also, don’t forget to disable synchronisation and auto-
summarisation – these are also mandatory requirements for ISP routers connecting to the Internet.
(Note that a distance of 250 could be applied to the static route to ensure that routing protocols
announcing this exact prefix will override the static (if this is required/desired).)
37. Enable new format of BGP communities. It is also worth getting into the habit of changing the
BGP community format from the default 32-bit integer to colon separated 16-bit integers, as used
in RFC1998. Example:
38. Configure eBGP peering. Now that iBGP is functioning, it is time to configure eBGP. External
BGP will be set up between AS10 and AS20, specifically between Routers 6 and 8, and Routers 7
and 9 only. The remaining lab teams should monitor the BGP table they see on their routers.
Firstly, agree on what IP addresses should be used for the point to point links between the ASes.
Put the /30 networks used for the DMZ links into OSPF (network statement and passive interface).
Then configure eBGP between the router pairs, for example:
router bgp 10
neighbor 10.20.2.2 description eBGP with RouterXX
Use the BGP show commands to ensure that you are receiving prefixes from your neighbouring
AS.
39. eBGP filters. The final step of setting up the eBGP sessions is to apply filters on the BGP
sessions. The teams operating Routers 6, 7, 8 and 9 should now set up filters to allow their
neighbour AS prefixes into their network; and set up filters to allow their own AS prefixes out of
their network. For example:
router bgp 10
neighbor 10.20.2.2 prefix-list AS20peer-in in
12
ISP Workshop Lab
neighbor 10.20.2.2 prefix-list AS20peer-out out

!
ip prefix-list AS20peer-in permit 10.20.0.0/19
ip prefix-list AS20peer-out permit 10.10.0.0/19
!
This only allows 10.20.0.0/19 in from an AS20 eBGP neighbour, and allows 10.10.0.0/19 out to an
AS20 eBGP neighbour. Don’t forget that you will also need to allow the address of the NTP
timeserver through the BGP filters as well.
40. BGP Communities on eBGP sessions. Network operators only send communities to their eBGP
peers if they have a policy requirement to do so. Cisco IOS also does not send communities on the
eBGP sessions by default, so we will now add this to eBGP peerings. The teams operating Routers
6, 7, 8 and 9 should add the “send-community” directive to their eBGP peerings. For example:
router bgp 10
!
41. Check the network paths and the routing table. Run traceroutes between your router and other
routers in the classroom. Ensure that all routers are reachable. If any are not, work with the other
router teams to establish what might be wrong. Make sure that you can see Router15. The lab
instructor will have written the addresses and network up on the whiteboard. (The network is
192.168.1.0/24, the address of Router6 on that LAN is 192.168.1.254, and the address of Router15
is 192.168.1.1.)
42. Saving the configuration. For software releases from 12.0 onwards, the commands to save the
configuration are of the format copy <source> <destination> where the source and destinations
can be any of the following options: ftp, lex, null, nvram, rcp, running-config, startup-config,
system, tftp. To save the configuration to the TFTP server, use the “copy system:/running-config
tftp:” command sequence. If the TFTP server is unreachable, “.”s followed by an error message
will be displayed rather than “!”s. (Note that the “write net” command of earlier releases is still
supported but may be removed at a future release.)
Router1#copy system:running-config tftp:

!!
Router1#
43. Summary. This module has covered most of the fundamental configuration topics required to
construct an ISP network. It has covered basic router configuration, configuration Best Current
Practices, OSPF configuration, iBGP configuration, and finally simple eBGP configuration. No
routing policy has been implemented. Each Router team is strongly recommended to make a
copy of their configuration, as most of the configuration concepts will be required
throughout the remainder of the workshop.
13
14
ISP Workshop Lab
Module 12 – Multihoming to the Same ISP

Objective: To investigate various methods for multihoming onto the same upstream’s backbone
Prerequisites: Module 11 and Multihoming Presentation
Figure 1 – ISP Lab Configuration
1
Wednesday, July 23, 2014
Lab Notes
The purpose of this module is to demonstrate multihoming in the situation where the customer has
more than one connection to their upstream service provider. There are several situations where this is
applicable:
! Enterprise customer requires more than one connection to the service provider to provide
resiliency, and/or loadsharing.
! Enterprise customer has multiple sites which require connection to the Internet.
! Start-up ISP requires more than a single link to the Internet, but has little requirement to connect to
more than one upstream ISP
It is important that you review the multihoming presentation before you start with this module. Only
configuration examples will be given – it will be left to the workshop participant to use the
presentation notes to help them configure their routers correctly.
The accepted way to multihome to a single upstream ISP on the Internet today is to use a private AS
number. The IANA defines the range 64512 to 65534 as being private ASes, in the same way that
RFC1918 defines private address space. These ASes should never be visible on the Internet.
Finally, to ensure an understandable and easy to follow configuration, as well as good practice, a few
assumptions about configuring BGP will be made. These are:
! Use prefix-lists to filter prefixes
! Use as-path access-lists to filter ASes
! Use route-maps to implement more detailed policy
There are rarely any exceptions to this. Prefix lists are very efficient access-lists and they make the
implementation of prefix filtering on AS borders (and elsewhere) very easy. Please review the BGP
presentation materials if there is any uncertainty as to how prefix lists work.
Lab Exercise
1. Basic Configuration. Before starting to configure anything for this module, each router team
should clean up the old configuration on the router they are using. The easiest way is probably to
do a “write erase” and start afresh. The alternative is to remove any interface IP addressing,
OSPF/ISIS and all BGP configuration. The latter does need care – remember that all unused
configuration must at all times be removed from the router.
other ASes to ensure there is no overlap. In the every day Internet such address assignment is
carried out by the Regional Internet Registry.
AS65534 10.10.0.0/20 AS65531 10.40.0.0/20

AS65533 10.20.0.0/20 AS10 10.50.0.0/20
AS65532 10.30.0.0/20
When constructing an addressing plan, don’t forget to use a small block for loopback interfaces
and another small block for point-to-point links. Also, agree between yourselves and your
neighbouring ASes which addresses will be used for the point-to-point links between you.
2
ISP Workshop Lab
Remember, it is Internet convention that addresses from the upstream ISP’s address block are used
for point to point link addresses to their customers.
Suggestion:
10.10.0.0/24 10.10.1.0 10.10.15.255
3. Routing Protocols. The chosen IGP (OSPF using area 0 only, or ISIS using L2 only) and iBGP
should now be configured between the routers in each AS. Any interfaces which are not going to
run OSPF/ISIS MUST be marked as passive in the configuration. And don’t forget to use BGP
peer groups for iBGP peers.
Checkpoint #1: When you have properly configured your router, and the other routers in the AS are
reachable (i.e. you can ping the other routers, and see BGP and OSPF/ISIS prefixes in the routing
table), please let the instructor know.
Scenario One – Primary link and backup link
This first scenario is more commonly employed where the customer has two circuits to their upstream:
a large circuit which is used for all the inbound and outbound traffic, and an inexpensive circuit they
use almost exclusively for backup purposes.
In this case the whole address block is announced out of both links. However, the announcement
going out the backup link is “weighted” using MEDs so that it is at a lower priority. Likewise, the
incoming default route announcement from the ISP is “weighted” using local-preference. (Hint:
remember the purpose of MEDs and local-preference? If in doubt, review the BGP presentation
material.)
4. Prepare to enable eBGP between AS10 and AS65534. Both AS65534 and AS10 should now be
running iBGP within their ASes. To announce AS65534’s prefix to AS1 we will take the /20
address block and announce it on both BGP peerings between the ASNs. AS10 will not announce
any specific prefixes to AS65534 – it will simply announce a default route. There is no need for
any more routing information to be injected into the customer site.
Note: some customers request/demand a full Internet routing table to be announced to their
network – if after education they still insist, give it to them, but creative ISPs often charge for this
service. Plus the customer needs to be aware they will need a router with at least 128Mbytes of
free memory given today’s size of the Internet routing table.
5. Prepare to enable eBGP between the other private ASes and AS10. The process for enabling
eBGP in this case is the same as for between AS10 and AS65534.
6. Create AS65534 prefix lists. First, create the prefix lists on the routers in AS65534. Both Router1
and Router3 will announce the /20. Both will accept the default route. Example for Router1:
3
ip prefix-list myblock permit 10.10.0.0/20

ip prefix-list default permit 0.0.0.0/0
7. Create AS10 prefix-lists. The routers in AS10 are the customer aggregation routers and should
only accept those prefixes which the customer is entitled to announce. So prefix lists need to be
installed on Routers 7 and 8 to do this. The example given is for Router 7:
ip prefix-list Customer permit 10.10.0.0/20

8. Configure the main link. Configure the main link between the private AS and the ISP. For
AS65534, the link between Router1 and Router7 in AS10 is the main link – the link between
Router3 and Router8 is the backup (marked in blue). Example configuration for Router1:
router bgp 65534

network 10.10.0.0 mask 255.255.240.0
neighbor <router7> description Link to Router8 in AS10
neighbor <router7> prefix-list myblock out
neighbor <router7> prefix-list default in
!
ip route 10.10.0.0 255.255.240.0 null0
Configure the other ASNs’ primary link – they are marked in black on the diagram in Figure 1.
9. Configure the backup link. Configure the backup links between the private ASNs and the ISP.
Set the metric on outbound announcements to 20, and set local preference on inbound
announcements to 90. Remember that lowest metric and highest local-preference win during the
BGP path selection process. To do this, use a route-map on the peering – you will require an
inbound and outbound route-map. Example configuration for Router4:

!
match ip address prefix-list myblock
set metric 20
!
match ip address prefix-list default
!
router bgp 65533
network 10.20.0.0 mask 255.255.240.0
neighbor <router11> route-map outfilter out
neighbor <router11> route-map infilter in
!
ip route 100.10.0.0 255.255.240.0 null0
4
ISP Workshop Lab
10. Configure eBGP with private ASNs in AS10. AS10 is going to originate the default route in the
peering with AS65534. The BGP command “default-originate” is used to do this. Example
configuration for Router 7:
router bgp 10
neighbor <router1> description Dualhomed Customer
neighbor <router1> default-originate
neighbor <router1> prefix-list Customer in
neighbor <router1> prefix-list default out
!
11. Connectivity Test. Check connectivity throughout the lab network. Each router team should be
able to see all other routers in the room. When you are satisfied that BGP is working correctly, try
running traceroutes to ensure that the primary paths are being followed. When you are satisfied
this is the case, check that the backup functions (do this by disconnecting the cable between the
two routers on the primary path) – you will see that the backup path is now used. For example:
Router8>trace 10.10.0.5

Tracing the route to 10.10.0.5
1 fa0-1.router7 4 msec 4 msec 0 msec

3 ser0-0.router5 [AS 65534] 4 msec * 4 msec
With the cable between Router 1 and Router 7 disconnected, the trace becomes:

1 fa0-0.router3 [AS 1] 4 msec 5 msec 4 msec

Router8>
STOP AND WAIT HERE
Scenario Two – Loadsharing (Method One)

Most dualhomed sites want to implement some kind of loadsharing on the circuits they have to their
upstream provider. The example here discusses only two circuits, but the techniques work equally well
for a greater number.
In this case, the whole address block is announced out of both links. Also, the address block is split
into two pieces, with one subprefix being announced out of one link, and the other being announced
out of the other link. The result of this is that traffic for the first /21 comes in one path, and traffic for
5
the second /21 comes in the other path. If either path fails, the advertisement of the /21 address block
(aggregate) ensures continued connectivity.
12. Clean up the private ASes. Remove the configuration which set the weighting for the previous
example – specifically the route-maps. They must be removed from the BGP configuration, and
from the main configuration.
13. Configure the address block and subprefixes in the private ASes. Modify the router
configuration so that the /20 address block and two /21 subprefixes are present in the BGP table.
Also set up prefix lists to cater for these blocks. For example:
ip prefix-list path1 permit 10.10.0.0/20

!
!
!
router bgp 65534
network 10.10.0.0 mask 255.255.240.0
network 10.10.0.0 mask 255.255.248.0
!
ip route 10.10.0.0 255.255.240.0 null0
ip route 10.10.8.0 255.255.248.0 null0
14. Configure BGP in the private ASes. Configure BGP on the border routers in the private ASes so
that the prefix and one sub prefix is announced to the direct peer. For example, Router1 could
announce path1 as above, whereas Router3 could announce an equivalent path2. For example, for
AS65533:
router bgp 65533

neighbor <router7> prefix-list path1 out
!
15. Check filters in AS10. Ensure that Routers7 and 8 have appropriate filters to allow the /20 and /21
prefixes in from the private AS customers. Here is an example:
router bgp 10
neighbor <router1> prefix-list Customer in
...
!
ip prefix-list Customer permit 10.10.0.0/20 le 21
!
16. Connectivity targets. So that connectivity via each /21 can be tested, Routers 5, 6, 9 and 10
should set up a second loopback interface with an IP address from their ASN’s respective second
/21 block. Note that the /32 address should be announced by the IGP so that other routers in the
ASN know how to get to the destination. The following configuration snippet shows a possible
configuration for Router 10 using OSPF:
6
ISP Workshop Lab
ip address 10.30.15.10 255.255.255.255
ip ospf 65533 area 0 ! ONLY for IOS 12.4 and later
!
router ospf 65533
network 10.30.15.10 0.0.0.0 area 0 ! ONLY for IOS <12.4
!
The following configuration snippet shows a possible configuration for Router 10 using ISIS:
ip address 10.30.15.10 255.255.255.255
!
router isis workshop
passive-interface loopback1
!
17. Connectivity test. Check connectivity throughout the lab network. Each router team should be
running traceroutes to Routers 5, 6, 9 and 10. Check the path being followed for the first and
second /21s for each ASN. For example:




Router8>
Also check that backup via the alternative path still functions (do this by disconnecting the cable
between the two routers on the primary path) – you will see that the backup path is now used.
STOP AND WAIT HERE
Scenario Three – Loadsharing (Method Two)

The third scenario is a variation on the second scenario and provides another example.
7
As before the whole address block is announced out of both links. In fact, one of the key features of
multihoming, and providing redundancy, is that the ISP’s address blocks are always announced out of
each external link. The key to loadbalancing is how those external announcements are made. In this
example, one /21 is taken out of the /20 address block and announced on one link between the
customer and the ISP as well as the /20. The other link sees just the standard announcement of the /20.
18. Clean up the private ASes. Remove the configuration which subdivided the address space for the
previous example. Remember it is always very important to remove any configuration which isn’t
being used from the router.
19. Configure the address block and subprefixes in the private ASes. Modify the configuration of
the routers in the private ASNs so that the /20 address block and one /21 subprefix are present in
the BGP table. Also set up prefix lists to cater for these blocks. For example:
ip prefix-list aggregate permit 10.10.0.0/20

!
ip prefix-list subblock permit 10.10.0.0/20
ip prefix-list subblock permit 10.10.0.0/21
!
!
router bgp 65534
network 120.10.0.0 mask 255.255.240.0
network 120.10.0.0 mask 255.255.248.0
!
ip route 120.10.0.0 255.255.240.0 null0
ip route 120.10.0.0 255.255.248.0 null0
20. Configure BGP in the private ASes. Configure BGP on the border routers in the private ASes so
that the prefix and one sub prefix is announced to the direct peer on one link, and just the
aggregate is announced on the other link. The router teams in each private ASN should discuss
amongst themselves who will leak the /21 prefix. For example, Router1 could announce subblock
as above, whereas Router3 could announce just the aggregate. A configuration example for
AS65533 might look like:
router bgp 65533

neighbor <router11> prefix-list subblock out
!
running traceroutes to check the path being followed. Also check that backup via the alternative
path still functions (do this by disconnecting the cable between the two routers on the primary
path) – you will see that the backup path is now used.
8
ISP Workshop Lab
STOP AND WAIT HERE
Scenario Four – RFC2270
The final scenario describes RFC2270 style multihoming and represents an efficiency saving over the
configuration concepts described in the previous three steps.
22. Routers in Private ASNs. ISPs who deploy RFC2270 style multihoming use only ONE ASN for
their multihoming customers. Each private AS should now replace the private ASN in their BGP
sessions with AS64512 (as in Figure 2). Do not change your BGP peers, just change the ASN
referred to. For example, all the routers in AS65534 will renumber their BGP to use AS64512.
This means removing BGP totally, and then adding it back in (there is no way to easily migrate
BGP ASNs in IOS). The best way of doing this is to copy the BGP configuration on to another
terminal window, edit the 65534 to replace it with 64512, do no router bgp 65534 on the
router, and the copy the modified BGP configuration back onto the router. For example:
Before Modification:
router bgp 65534

neighbor ibgp peer-group
neighbor ibgp remote-as 65534
neighbor ibgp update-source loopback0
neighbor ibgp send-community
neighbor ibgp next-hop-self
neighbor ibgp password cisco
neighbor 10.10.0.3 peer-group ibgp
neighbor 10.10.0.3 description R3
!
After Modification:
router bgp 64512

neighbor ibgp peer-group
neighbor ibgp remote-as 64512
neighbor ibgp update-source loopback0
neighbor ibgp send-community
neighbor ibgp next-hop-self
neighbor ibgp password cisco
!
9
Figure 2 - RFC2270 configuration
23. Routers in AS10. The teams operating Router7 and Router8 should make their BGP configuration
more efficient. Because each neighbour is now in the same ASN, each team can replace the
individual BGP configurations for each neighbouring ASN in the previous steps with a generic
peer-group based configuration. Create a peer-group for the multihoming private ASNs – for
example:
router bgp 10
neighbor rfc2270 peer-group
neighbor rfc2270 remote-as 64512
neighbor rfc2270 default-originate
neighbor rfc2270 send-community
neighbor rfc2270 prefix-list default out
neighbor rfc2270 password cisco
10
ISP Workshop Lab
And then apply the peer-group to each eBGP customer, for example:
router bgp 10
neighbor <router1> peer-group rfc2270
neighbor <router1> prefix-list r1 in
!
24. Final check. This step introduced a more efficient way of configuring multiple multihomed
customers, as described in RFC2270. While there is no requirement to configure customers in this
style, the transit ISP has a more efficient and therefore scalable configuration. Check that the
routing and connectivity is exactly as it was at the end of the previous scenario.
25. Summary. This module has covered the major situations where a customer requires to
multihomed onto the service provider backbone. It has demonstrated how to implement this
multihoming using prefix-lists, MEDs and local-preference where appropriate.
11
ISP Workshop Lab
Module 13 – Multihoming to Different ISPs

Objective: To investigate various methods for multihoming onto two different upstream ISPs.
Prerequisites: Module 12 and Multihoming Presentation
Figure 1 – ISP Lab Multihoming Configuration
1
Lab Notes
The purpose of this module is to demonstrate multihoming in the situation where the customer AS has
one connection to more than one upstream service provider. There are at least two situations where
this is applicable:
! Enterprise or Service Provider customer requires more than one connection to the Internet to
provide resiliency, and/or loadsharing.
! Enterprise or Service Provider customer requires more than one Internet connection to give their
network service provider redundancy.
It is important that you review the multihoming presentation before you start with this module. Only
configuration examples will be given – it will be left to the workshop participant to use the
presentation notes to help them configure their routers correctly.
To ensure an understandable and easy to follow configuration, as well as good practice, a few
assumptions about configuring BGP will be made. These are:
! Use prefix-lists to filter prefixes
! Use as-path access-lists to filter ASes
! Use route-maps to implement policy
There are rarely any exceptions to this. Prefix lists are very efficient access-lists and make
implementation of prefix filtering on AS borders (and elsewhere) very easy. Please review the BGP
presentation materials if there is any uncertainty as to how prefix lists work.
Lab Exercise
depicted in Figure 1. Check all connections. Remember what was covered in Module 11!
other ASes to ensure there is no overlap. In the every day Internet, such address assignment is
carried out by the Regional Internet Registry.
AS10 10.10.0.0/20 AS40 10.40.0.0/20

AS20 10.20.0.0/20 AS100 10.100.0.0/20
AS30 10.30.0.0/20 AS200 10.200.0.0/20
3. Routing Protocols. The IGP (either OSPF or ISIS) and iBGP should now be configured between
the routers for each AS. External interfaces must not be configured to run an IGP. And don’t forget
to use BGP peer-groups for iBGP peers.
reachable (i.e. you can ping the other routers, and see BGP and ISIS/OSPF prefixes in the routing
table), please let the instructor know.
STOP AND WAIT HERE
2
ISP Workshop Lab
Scenario One – Primary link and backup link
This first scenario is more commonly employed where the customer has a large circuit to their
upstream and an inexpensive circuit they use almost exclusively for backup purposes to another ISP.
(The case maybe that the primary ISP has good connection to the Internet, and the backup ISP is used
only as a last resort – for technical, commercial, or political reasons.)
In this case the whole address block is announced out of both links. However, the announcement
going out the backup link has its AS path length increased so that it is at a lower priority. Likewise,
the incoming default route announcement from the ISP is “weighted” using local-preference. (Hint:
remember the purpose of changing the AS Path length? If in doubt, review the BGP presentation
material.)
4. Enable eBGP between the transit ASes. AS100 and AS200 should now enable their eBGP link
to each other. All router teams in these ASes must ensure that they can see all the prefixes of
AS100 and AS200. If they are not there, work with your team members to ensure they appear.
Don’t forget the static pull-up route when injecting prefixes into BGP! Also, at this stage there is
no need to install prefix filters between these ASes – if you would like to, don’t forget that you
need to allow through the network blocks of the ASes you are providing transit to.
5. Prepare to enable eBGP between AS10 and its two upstreams. AS10 should currently be
running iBGP within its own network. To announce AS10’s prefix to AS100 and AS200 we will
take the /20 address block and announce it on both peerings between the ASNs. AS100 and AS200
will not announce any prefixes to AS10 – they will simply announce a default route. There is no
need for any more specific routing information to be injected into the customer site at this stage in
the lab.
6. Prepare to enable eBGP between the other ASes and each of their two upstreams. The
configuration steps to enable BGP between AS20, AS30, AS40 and their two upstreams are the
same as those being described for AS10.
7. Create AS10 prefix lists. First, create the prefix lists on the routers in AS10. Both Router1 and
Router3 will announce the aggregate. Both will accept the default route. Example for Router1:

8. Create AS100 and AS200 prefix-lists. The routers in AS100 and AS200 should only accept those
prefixes which the customer is entitled to announce. So a prefix list needs to be installed on both
Router7 and Router8 to do this. For example:
ip prefix-list Customer permit 10.10.0.0/20

9. Configure the main link. Configure the main link between the customer AS and the ISP. For
AS10, the link between Router1 and Router7 in AS100 is the main link (coloured in black) – the
link between Router2 and Router8 in AS200 is the backup (coloured in blue). An example
configuration for Router1 might be:
router bgp 10
3
network 10.10.0.0 mask 255.255.240.0

!
ip route 10.10.0.0 255.255.240.0 null0
10. Configure the backup link. Configure the backup link between the customer AS and the ISP.
Increase the AS Path Length on outbound announcements to 3, and set local preference on inbound
announcements to 80. Remember that the shortest AS Path Length and highest local-preference
win during the BGP path selection process. To do this, use a route-map on the peering – you will
require an inbound and outbound route-map. Example configuration for Router13:

!
match ip address prefix-list myblock
!
!
router bgp 40
network 10.40.0.0 mask 255.255.240.0
!
ip route 10.40.0.0 255.255.240.0 null0
11. Configure primary and back up links for the other 3 ASNs. The teams in the other three
customer ASes should use the above steps as a guideline to configure their primary and backup
paths with AS100 and AS200. Refer to the diagram in Figure 1. The black coloured links are the
primary paths; the blue coloured links are the backup paths.
12. Configure eBGP in AS100 and AS200 with AS10. AS100 and AS200 are going to originate the
default route in the peering with AS10. The BGP command default-originate is used to do
this. Example configuration for Router 7:
router bgp 100

neighbor <router1> description Multihomed Customer
neighbor <router1> prefix-list AS10 in
!
Once the teams operating Router7 and Router8 have done the configuration for AS10, they should
also complete the configuration for AS20, AS30 and AS40 customers.
4
ISP Workshop Lab
running traceroutes to ensure that the primary paths are being followed. When you are satisfied
this is the case, check that the backup functions (do this by disconnecting the cable between the
two routers on the primary path) – you will see that the backup path is now used.
STOP AND WAIT HERE
Scenario Two – Loadsharing

Most multihomed sites want to implement some kind of loadsharing on the circuits they have to their
upstream provider. The example here discusses only two circuits, but the techniques work equally well
for a greater number.
To do this, the whole address block is announced out of both links. Also, the address block is split into
two pieces, with one subprefix being announced out of one link, and the other being announced out of
the other link. The result of this is that traffic for the first /21 comes in one path, and traffic for the
second /21 comes in the other path. If either path fails, the advertisement of the /20 address block
(aggregate) ensures continued connectivity.
14. Clean up the configuration of AS10, AS20, AS30 and AS40. Remove the configuration which
set the weighting for the previous example – specifically the route-maps. They must be removed
from the BGP configuration, and from the main configuration.
15. Configure the address block and subprefixes in the customer ASes. Modify the router
configuration so that the /20 address block and the two /21 subprefixes are present in the BGP
table. Also set up prefix lists to cater for these blocks. For example:

!
!
!
!
router bgp 10
network 10.10.0.0 mask 255.255.240.0
network 10.10.0.0 mask 255.255.248.0
!
ip route 10.10.0.0 255.255.240.0 null0
ip route 10.10.0.0 255.255.248.0 null0
16. Configure BGP in the customer ASes. For AS10, the link between Router1 and Router7 in
AS100 is the first link – the link between Router2 and Router8 in AS200 is the second (and is the
5
one which should also announce the subprefix). Configure BGP on the border routers in the
customer ASes so that the prefix and one sub prefix is announced to the direct peer as described
earlier. For example, Router1 could announce aggregate and path1 as above, whereas Router2
could announce aggregate and path2. For example on Router13:
!
router bgp 40
network 10.40.0.0 mask 255.255.240.0
network 10.40.8.0 mask 255.255.248.0
neighbor <router8> prefix-list path2 out
!
ip route 10.40.8.0 255.255.248.0 null0
17. Connectivity targets. So that connectivity via each /21 can be tested, Routers 3, 6, 11 and 13
should set up a second loopback interface with an IP address from their ASN’s respective second
/21 block. Note that the /32 address should be announced by the IGP so that other routers in the
ASN know how to get to the destination. The following configuration snippet shows a possible
configuration for Router 11 using OSPF:
ip address 10.30.15.10 255.255.255.255
ip ospf 30 area 0 ! ONLY for IOS 12.4 and later
!
router ospf 65533
network 10.30.15.10 0.0.0.0 area 0 ! ONLY for IOS <12.4
!
The following configuration snippet shows a possible configuration for Router 11 using ISIS:
ip address 10.30.15.10 255.255.255.255
!
router isis workshop
!
STOP AND WAIT HERE
6
ISP Workshop Lab
Scenario Three – More Controlled Loadsharing

The third scenario is a variation on the second scenario and provides another example.
As before the whole address block is announced out of both links. In fact, one of the key features of
multihoming, and providing redundancy, is that the ISP’s address blocks are always announced out of
each external link. The key to loadbalancing is how those external announcements are made. In this
example, one /21 is taken out of the /20 address block and announced on one link between the
customer and the ISP – and the /20 is announced with a longer AS-PATH on this link also. The other
link sees just the standard announcement of the /20.
19. Clean up the configuration of AS10, AS20, AS30 and AS40. Remove the configuration which
subdivided the address space for the previous example. Remember it is always very important to
remove any configuration which isn’t being used from the router.
20. Configure the address block and subprefixes in the customer ASes. Modify the router
configuration so that the /20 address block and one /21 subprefix is present in the BGP table. Also
set up prefix lists to cater for these blocks. For example:

!
ip prefix-list subblocks permit 10.10.0.0/20 le 21
!
!
router bgp 10
network 10.10.0.0 mask 255.255.240.0
network 10.10.0.0 mask 255.255.248.0
!
ip route 10.10.0.0 255.255.240.0 null0
ip route 10.10.0.0 255.255.248.0 null0
21. Configure BGP in the customer ASes. For AS10, the link between Router1 and Router7 in
AS100 is the first link – the link between Router2 and Router8 in AS200 is the second (and is the
one which should also announce the subprefix). Configure BGP on the border routers in the
customer ASes so that the prefix and one sub prefix is announced to the direct peer as described
earlier. For example, Router1 could announce aggregate as above, whereas Router2 could
announce aggregate with a lengthened AS Path, and announce subblock1 as is. For example on
Router2:

match ip address prefix-list aggregate
!
!
router bgp 10
network 10.10.0.0 mask 255.255.240.0
neighbor <router8> prefix-list subblocks out
7

!
STOP AND WAIT HERE
Scenario Four – Introducing BGP Communities
23. Introducing BGP Communities. The purpose of this scenario is to introduce BGP communities
to assist with the multihoming requirements of each autonomous system. So far the examples have
used prefix filters at the edge of each AS. We will now replace these prefix filters with BGP
community filters to allow us greater flexibility in scaling our future traffic engineering needs.
First of all, we will assign specific communities to specific policies. Here is the list of what we
will support in this scenario:
ASn:1000 aggregate
ASn:1001 subprefixes of the aggregate
ASn:1005 other addresses we announce
ASn:9999 black hole this route
Each router team in each AS should set up 4 IOS community-lists matching the above
communities. Here is an example configuration for a router in AS 20:

!
!
!
!
24. Tagging prefixes with communities. The next step is for each router team to tag the prefixes they
are introducing into BGP with the appropriate communities. Each AS generates an aggregate, and
this aggregate should be tagged with the ASn:1000 community. Here is an example configuration
for a router in AS 20:
route-map tag-aggregate permit 10

set community 20:1000
8
ISP Workshop Lab
!
router bgp 20
network 10.20.0.0 mask 255.255.240.0 route-map tag-aggregate
!
The BGP table should now show that the aggregate has the community 20:1000 set on it. Each
router team should do likewise, for their ASN. Check the BGP table afterwards to make sure that
the community is visible on the aggregate originated for the ASN.
25. Replacing prefix filters with community filters. The router teams with eBGP sessions should
now create route-maps which will replace the current prefix filters configured. First create a route
map to cover the outbound filters (which allow the local AS originated aggregate out). Here is
another example – the outbound prefix-list is removed and replaced with the community route-
map:
route-map ebgp-out permit 10

match community 10
!
router bgp 20
neighbor x.x.x.x remote-as 200
neighbor x.x.x.x route-map ebgp-out out
!
And then do the same for the inbound filter, if applicable. This will depend where in the network
the ASN is though – if an edge AS, the only incoming prefix would be the default route from the
upstream provider, so no need to change.
26. Check the network paths. Run traceroutes between your router and other routers in the
classroom. Ensure that all routers are reachable. If any are not, work with the other router teams to
establish what might be wrong.
27. Summary. This module has covered the major situations where a customer requires to
multihomed onto more than one service provider backbone. It has demonstrated how to implement
this multihoming using prefix-lists, AS Path Length modifications and local-preference where
appropriate.
9
ISP Workshop Lab
Module 14 – Transit
Objective: To investigate methods for providing transit services.
Prerequisites: Modules 12 and 13, and the Transit Presentation
Figure 1 – ISP Lab Transit Configuration
1
Lab Notes
The purpose of this module is to demonstrate how to configure backbones to provide transit services.
Transit in its basic form means carrying packets with source and destination addresses which are not in
the local AS. Not being in the local AS implies that these packets have originated in another AS, are
being carried across the local AS, and transmitted to another connected AS.
The lab covers some of the configuration possibilities. AS10 and AS40 are edge ASNs and use the
other ASes in the network to get transit between each other. AS20 and AS30 also get transit from
AS100 and AS200 to reach each other – and provide transit to AS10 and AS40 respectively.
Lab Exercise
1. Basic Configuration. Each router team should configure their router to fit into the network
topology depicted in Figure 1. Check all connections.
2. Addressing Plan. These address ranges, which we have retained from Module 13, should be used
throughout this module – it means we don’t have to renumber or reconfigure the OSPF or iBGP.
(You are welcome to use your own range within an AS if you desire, just so long as you consult
with the teams in other ASes to ensure there is no overlap.)
AS10 10.10.0.0/20 AS40 10.40.0.0/20

AS20 10.20.0.0/20 AS100 10.100.0.0/20
AS30 10.30.0.0/20 AS200 10.200.0.0/20
3. Routing Protocols. Check that OSPF/ISIS is still working as per the previous module – also check
that iBGP is still functioning as expected. Any interfaces which should not be running an IGP
MUST be marked as passive in the configuration.
reachable (i.e. you can ping the other routers, and see BGP and OSPF prefixes in the routing table),
please let the instructor know.
Scenario One – Simple transit example

The first example is that of very simple transit. The example only uses prefix lists to configure
filtering, and only provides transit from one AS, across an intermediate AS, to a remote AS. The
example is typically that of a backbone ISP (NSP) providing connectivity services for access or
content providers.
4. Configuring edge ASes. The router teams in ASes 10 and 40 should now configure their routers
to peer with their transit providers. As in previous modules, they will only receive the default route
from their transit provider. Also, they will only announce their own network block to their transit
provider. This is very important to remember – ASes on the edge of the “Internet” should always
attempt to minimise the complexity of their configuration. An example for Router5 in AS10 might
be:
2
ISP Workshop Lab

!
router bgp 10
network 10.10.0.0 mask 255.255.240.0
neighbor <router6> description My Transit Provider to AS20
!
ip route 10.10.0.0 255.255.240.0 null0
This should now be a very familiar configuration as it has been used several times in previous
Modules. It is the best way of configuring an edge AS to connect to any Internet service provider.
5. Configuring border routers in AS20 and AS30. Next the routers in AS2 and AS6 which are
providing the connection to the edge ASes need to be configured. Specifically these are Routers 2,
6, 9 and 11. The configuration again is similar to what has been repeatedly used in previous
modules. Basically you want to announce default route only to your customer, and only accept the
prefix block which they are entitled to send to you. A sample configuration for Router2 might be:
ip prefix-list Customer1 permit 10.10.0.0/20

!
router bgp 20
neighbor <router3> description Link to BGP Customer 1
neighbor <router3> prefix-list Customer1 in
!
The configurations for the other routers will be similar.
6. Configure the transit of AS20 and AS30 across AS100 and AS200. The next step is to
configure the routers in AS20, AS30, AS100 and AS200 such that AS20 transits AS100 and
AS200 to reach AS30 – and vice-versa, of course. Remember that AS20 should not announce
AS10 to AS100/200 and AS30 should not announce AS40 to AS100/200. So Routers 2, 4, 11 and
13 should only announce their own ASes to AS100 and AS200. A sample configuration for
Router2 might be:

!
router bgp 20
network 10.20.0.0 mask 255.255.240.0
neighbor <router7> description Link to Transit Provider AS100
!
ip route 10.20.0.0 255.255.240.0 null 0
Notice that only the locally originated prefix is announced by AS20 to AS100. Later on in this
module we will change the configuration so that AS20 and AS30 provide transit to other ASes too.
3
7. Configure the AS100 and AS200 routers. Finally we need to configure the routers in AS100 and
AS200. The configuration is very similar to what was seen in Step 5 – only announce the default to
the customer, only accept what the customer is supposed to announce to you.
able to see across their transit ASes to the routers in the opposite customer AS. When you are
satisfied that BGP is working correctly, try running traceroutes to ensure that the proper paths are
being followed. Note that the connectivity will be such that not every AS can see the entire
network.
STOP AND WAIT HERE
Scenario Two – More complex transit

This situation above was a simple example of how to configure transit simply using prefix-list filters.
However, this doesn’t really represent reality in the majority of Internet cases. This next example
introduces transit beyond the backbone AS we saw above. But now we want each AS to provide
transit to the other ASNs, rather than the limited connectivity we saw in the first scenario.
9. Reconfigure the network. All routers should remove any prefix-list filtering and eBGP peering
which has been configured. It is important that this is done so that the differences between this and
the previous scenario can be understood.
10. Configuring edge ASes. The router teams in ASes 10 and 40 should now configure their routers
to peer with their transit providers. AS10 and AS40 again receive global transit so only need to
receive the default route from their transit provider. But this time we will send them the “full BGP
table”, hence required to do different filtering. An example for Router3 in AS10 might be:

!
router bgp 10
network 10.10.0.0 mask 255.255.240.0
neighbor <router2> description My Transit Provider to the World
neighbor <router2> prefix-list bogons in
!
ip route 10.10.0.0 255.255.240.0 null0
The bogons prefix list was discussed during the presentation. It is good practice to have this prefix
list configured on all eBGP peerings, in addition to any other filtering required. Refer to the BGP
presentation material for the configuration of the list.
11. Configuring border routers in AS20 and AS30. Next the routers in AS20 and AS30 which are
providing the connection to the edge ASes need to be configured. Specifically these are Routers 2,
6, 9 and 11. The configuration is slightly modified from the previous section, with the routers
providing a full BGP feed to their customers. A sample configuration for Router2 might be:
4
ISP Workshop Lab
ip prefix-list Customer1 permit 10.10.0.0/20

!
router bgp 20
neighbor <router3> description Link to BGP Customer 1
neighbor <router3> prefix-list Customer1 in
neighbor <router3> prefix-list bogons out
!
12. Configure the transit of AS20 and AS30 across AS100 and AS200. The next step is to
configure the routers in AS20 and AS30 such that AS20 transits AS100/200 to reach AS30 – and
vice-versa, of course. AS20 and AS30 announce their customer address space as well as their own
address space to AS100 and AS200. A sample configuration for Router 4 might be:

!
router bgp 20
network 10.20.0.0 mask 255.255.240.0
neighbor <router8> description Link to Transit Provider AS200
neighbor <router8> filter-list 5 out
!
ip route 10.20.0.0 255.255.240.0 null0
Notice the application of the bogons prefix list mentioned previously. Get into the habit of using it!
Also there is no inbound filter-list. Some ISPs construct those too, using entries in the Internet
Routing Registry (IRR)
13. Configure the AS100 and AS200 routers. Finally we need to configure the routers in AS100 and
AS200. The configuration is very similar to the above – but use as-path filters to filter what your
peers are sending you. An example for Router 8 is included:

ip as-path access-list 15 permit ^20_10$
!
!
router bgp 200
network 10.200.0.0 mask 255.255.240.0
neighbor <router11> description Link to Customer AS30
neighbor <router11> filter-list 15 out
!
ip route 10.200.0.0 255.255.240.0 null0
Notice the as-path filter this time – basically anything heard from AS20 is allowed to transit to
AS30. Also notice the prefix-lists!
able to see across their transit ASes to the routers in the opposite customer AS. When you are
5
satisfied that BGP is working correctly, try running traceroutes to ensure that the proper paths are
being followed.
15. Summary. This module has given examples of configurations used by Internet Transit Providers
(NSPs). They have concentrated on using prefix-lists and as-path filters – more sophisticated
configurations are possible by using communities. These examples are left to the reader to
consider. If there is time at the end of the workshop, ask the Instructor to test out some other
scenarios.
6
ISP Workshop Lab

Prerequisites: Modules 12 and 13, and the Exchange Points Presentation
1
Friday, September 05, 2014
Lab Notes
It has seven ASNs participating at the IX, with two routers being assigned to each autonomous system.
One router peers at the IX, the other router is internal to the autonomous system (so will have an ISIS
and iBGP session with the IX facing router). It also has three routers operating as three transit provider
or Tier1 ISPs. This version of the module is incremental over Module 10 and covers correct ISIS and
iBGP configuration practices for connecting the IXP facing router to the rest of the ISP network.
Lab Exercises
1. A More Advanced IXP. This IXP example has seven participating ASNs with each ASN having
two routers. One router peers at the IXP; the other router is internal to the ISP’s network and
connects to an upstream ISP. The two routers in each ASN present at the IXP use ISIS and iBGP
to communicate routing information with the each other. The three upstream ASNs are also
interconnected with each other, to represent the high speed Internet core. What we have done here
is to simulate an Internet Exchange Point and its participants’ connections to upstream ISPs.
depicted in Figure 1. Check all connections. Note that all links to the IX switch are by Ethernet.
The links between each router in each AS is by serial connection (to represent the WAN
connection from the IX back to the ISP’s core network). The links from each AS to their upstream
is by Ethernet (to represent the WAN link to the upstream).
3. Addressing Plan. These address ranges should be used throughout this section of the module. You
are welcome to use your own range within an AS if you desire, just so long as you consult with the
teams in other ASes to ensure there is no overlap.
AS100 10.1.0.0/20 AS105 10.6.80.0/20

AS101 10.2.16.0/20 AS106 10.7.96.0/20
AS102 10.3.32.0/20 AS16384 10.8.112.0/20
AS103 10.4.48.0/20 AS16385 10.9.128.0/20
AS104 10.5.64.0/20 AS16386 10.10.144.0/20
basic security, IP addressing etc. The addressing for the point-to-point link between the two routers
in each AS is left as an exercise to the Router Teams. But remember, a point-to-point link requires
a /30 address block. Don’t forget to configure the loopback interfaces too.
5. IXP LAN. The address range used for the IXP LAN is 172.17.10.0/24 – the route server (if
provided in this module) has an IP address of 172.17.10.254. Each of the ASes is assigned
addresses sequentially for use on the exchange point LAN. So, for example, AS100 has
172.17.10.1; AS101 has 172.17.10.2; AS102 has 172.17.10.3. And so on.
2
ISP Workshop Lab

ip address 172.17.10.1 255.255.255.0
no ip proxy-arp
no ip redirects
!
7. Configuring ISIS (part 1). Obviously each IXP team will need to configure ISIS between the two
routers in their AS. The router which is internal to the ASN’s network should be straightforward to
configure. It has one loopback interface, one interface connecting to the router at the IXP, and one
interface connecting to the upstream. ISIS should only be activated on the internal interface, with
the loopback marked as passive. Note we do not configure the interface pointing to the upstream as
we will be using next-hop-self for our iBGP sessions. Example for Router 11 might be:
router isis as105

net 49.0001.0100.0609.6001.00
metric 100000
!
ip router isis as105
!
8. Configuring ISIS (part 2). The ISIS configuration on the router connecting to the IXP needs
more care. It is highly important that the IXP LAN network is not advertised into the ISP’s iBGP.
If it does, there is the possibility of leaking the IX LAN network by the ISP’s BGP to other ASNs,
providing transit to the IXP. This might not be a good thing, and many IXPs have rules against this
behaviour. What we do want to do is include the IX LAN in the ISIS, and like the loopback
interface, we mark the IX LAN interface as passive in ISIS.
The ISIS configuration for the IXP router looks something like this, for example for Router 8:
router isis as103

net 49.0001.0100.0404.8001.00
metric 100000
passive-interface Loopback0
passive-interface fastethernet0/0
!
ip router isis as103
3

!
NB. There is NO ISIS configuration applied to the Ethernet interface. Do NOT put any in, repeat,
do NOT! It is NOT a mistake.
9. Configuring iBGP between upstream facing routers and the IXP facing router. Any router
which is connected to other ASNs (the ISP border routers) such as the ISP’s upstreams or ISP’s
peers needs to have a modified iBGP configuration when peering with the IXP facing router. If all
the prefixes learned from the upstream ISP are passed onto the IXP facing router, then there is the
possibility that the IXP peers (and non-peers) can point static routes for those destinations at the
IXP facing router, thereby gaining outbound transit across the local ASN. This is undesirable, and
is basically a security risk. The configuration in the previous step therefore needs to be modified
for IXP facing routers (the only router in this lab module example) so that external prefixes are not
announced to the IXP facing router. In real life, ISPs create a peer-group which is used only when
peering with IXP-facing routers, for example:
router bgp 103

no synchronization
network 10.4.48.0 mask 255.255.240.0
neighbor ibgp-ixp peer-group
neighbor ibgp-ixp remote-as 103
neighbor ibgp-ixp update-source loopback0
neighbor ibgp-ixp next-hop-self
neighbor ibgp-ixp password cisco
neighbor ibgp-ixp description iBGP peering with IXP routers
neighbor ibgp-ixp send-community
neighbor ibgp-ixp filter-list 10 out
neighbor router8-loopback peer-group ibgp-ixp
no auto-summary
!
ip route 10.4.48.0 255.255.240.0 null0
!
!
Note the addition of the as-path access-list 10 – this basically allows only locally originated
prefixes to reach the IX facing router. For example, Router7 with this configuration will now no
longer send externally learned prefixes to Router8, thereby guaranteeing the security of the AS103
network.
10. Configuring iBGP on IXP facing routers (Part 1). The iBGP configuration on the IXP facing
router needs much more care. We do NOT want the IXP LAN to appear in our iBGP. And because
the IXP LAN is not in ISIS, we can’t use this net for valid next-hops. So that BGP has a valid
next-hop, we again use the next-hop-self BGP configuration as we have done for our other
iBGP sessions earlier on. For example, for Router 6:
router bgp 102

no synchronization
neighbor ibgp-peers update-source loopback0
neighbor ibgp-peers next-hop-self ! set external next-hops to us
neighbor ibgp-peers descr iBGP peers
4
ISP Workshop Lab

neighbor router5-loopback peer-group ibgp-peers
no auto-summary
!
This configuration is also valid for other iBGP peers in the network if there were more than two
routers in the ASN. ISPs try and keep configuration differences to the absolute minimum, and
having two different peer-groups defined for iBGP is usually sufficient!
11. Configuring iBGP on IXP facing routers (Part 2). The second important difference required for
the iBGP at the IXP router is that the local address block must NOT be originated here. If the IX
router is disconnected from the core backbone for whatever reason, if it originated the ISP’s
address block, it would end up blackholing all traffic from the IXP participants to the local
network. This is highly undesirable, so the correct procedure is to originate the ISP address block
only in the core of the network, not the edge, so that normal BGP failover rules can apply.
Q. What do you think the normal BGP failover rules might be in this case?
A. If the IXP facing router is disconnected from the core network, the iBGP fails and the ISIS
fails. So the router no longer hears the announcement of the ISP’s address block from the core, so
no longer announces it to the IXP peers, its eBGP neighbours. When this happen, the other ASNs
will use fall back paths (alternative to the IXP) to reach the local ASN. These are the normal BGP
failover rules.
12. Configuring iBGP on the IXP facing routers (Part 3). The second very important requirement
at an Internet exchange point is that a default route (or the routes not originated by the local ASN)
should not be made available at the peering LAN. If this requirement is not followed, then it is
possible for the Exchange Point participants to use the local network to get transit to the rest of the
Internet.
In this Module, each ASN will be receiving a default route from their upstream provider (a
common scenario in the Internet today). This default route must NOT be made available at the
ISP’s Exchange Point router. There are two ways to do this. Either block the announcement of the
default route by iBGP at the borders router which connect to the upstreams, or set up a static
default route to the Null interface at the IXP router (or indeed do both!). For example:
ip route 0.0.0.0 0.0.0.0 null0
Then if any of the IXP participants point a default route to the local network, the traffic will simply
be dumped in the Null interface of their IXP router. Only traffic for specific destinations which
available in the routing table on the IXP router will be forwarded to the rest of the network. This is
a very important network security requirement.
13. Configuring eBGP on the IXP facing routers. Next, eBGP needs to be set up on the IXP routers.
Create a peer-group and apply that peer-group to each eBGP neighbour. A sample configuration
for Router8 might be:

..
!
5
router bgp 103

no synchronization
..
no auto-summary
!
!
Note the prefix-lists. There is a per-peer inbound prefix-list. Some service providers only filter
14. Set up passwords on the eBGP sessions at the IXP. Negotiate with each ASN a password which
you can use on your BGP session with them. And then agree to cut the eBGP session over to using
Module 2). An excerpt from Router 8’s configuration might be:
router bgp 103

...
...
!
15. Configuring eBGP between the routers in AS16384, AS16385 and AS16386. The instructors
will operate the three transit/Tier1 ASNs; they will now configure eBGP between each other. No
filtering is expected or required between the three ASNs, so the eBGP configuration is quite
straightforward, for example as on Router 15:
router bgp 16384

no synchronization
neighbor <router16> password as16385-password
no auto-summary
!
6
ISP Workshop Lab
16. Configuring eBGP with the routers in AS16384, AS16385 and AS16386. All IXP members
must now configure eBGP sessions with the routers in the three transit ASNs. They must expect to
receive the default route from their upstreams, and only send their address block to the upstream.
An example configuration for Router 3 might be:

!
router bgp 101
no synchronization
neighbor <router15> prefix-list myprefixes out
no auto-summary
!
Router9#trace 10.7.112.1
1 10.5.64.6 4 msec 4 msec 4 msec

2 172.17.10.4 4 msec 4 msec 4 msec
3 10.7.112.1 [AS 106] 4 msec * 4 msec
Router9#
STOP AND WAIT HERE
18. Implementing a Route Server at the IXP. With route server functionality added to Cisco IOS as
from the 15.2 release, it is now possible to operate Router 18 as a route server. The lab instructors
will have configured Router 18 to be a route server.
The following information is included for the information of each router team member only. For
those in the workshop interested in setting up an Internet Exchange Point following the completion
of the workshop, this section contains the information necessary to configure and operate a Cisco
IOS based Route Server.
The Route Server has eBGP sessions configured with each member of the IXP in this workshop
lab (in real life, the Route Server has an eBGP session configured with those members of the IX
who wish to use its facilities.) Each eBGP session is configured with the “route-server-client”
command included in the eBGP peer-group.
7
This is an excerpt from the configuration of Router 18 after it has been configured to be a route
server:
router bgp 65534

neighbor ixp-peers description Route Server peers
neighbor ixp-peers password cisco
neighbor 172.17.10.1 peer-group ixp-peers
neighbor 172.17.10.1 description AS100 peer
!
address-family ipv4
neighbor ixp-peers route-server-client
neighbor 172.17.10.1 activate
exit-address-family
!
Notice that in this simple route-server example, there are no BGP filters at all – it is left to the IXP
member to do necessary filtering.
This is just a simple example. Most IXPs will insist on the route server doing the basic bogon
filtering which has been discussed elsewhere during this workshop. Some IXPs operate their route
servers to filter according to the information contained in the Internet Routing Registry.
19. Set up eBGP session with Router 18. Each team member should now configure their router to
peer eBGP with the route server. We will leave the existing bi-lateral peering sessions in place for
the time being. Here is an example configuration:
router bgp 103

...
neighbor 172.17.10.254 description eBGP with the IX Route Server
8
ISP Workshop Lab

...
Notice that the route server is running in a private AS – there isn’t really any need for it to use a
public AS as the Server does not need to be directly visible outside of the IXP.
Note also that there is no inbound prefix filter. This is not normally the case for an IXP. Even if an
ISP peers with the Route Server, they should still filter inbound BGP prefixes they receive via the
Route Server, just in case a peer who uses the Route Server advertises prefixes they should not be
advertising.
20. Modifications to the ISP’s IXP routers to talk to the Route Server. Route servers, as a rule, do
not include their AS number in BGP announcements to their clients; they are simply there to
distribute the BGP table they have learned from their clients. Historically RSd and routing
software platforms such as Zebra and Quagga have always behaved this way when running in
route server mode. Some IXPs used routers instead of these software routing implementations, and
routers have not had the functionality to drop the local ASN, meaning that the Route Server’s AS
has appeared first in the AS path.
Cisco routers participating at IXPs with the software based route servers have long had a capability
to accept peerings with a full Route Server AS – but it is not IOS’s default behaviour. IOS’s
default behaviour is to check that all prefix announcements from an eBGP peer have the peer AS
as the first AS in the AS path. To support an eBGP peering where the Route Server AS is not
included in the AS path, the router needs the following configuration:
router bgp 103

no bgp enforce-first-as
...
which stops the router rejecting the prefix updates if the peer AS is not first in the AS path. You
will notice that before entering this command, the BGP peering will be up, but no prefixes will be
received by the ISP’s IXP router – and the router logs will have messages about malformed
attributes being sent by the Route Server.
Checkpoint #3: Now look at your BGP routing table. You should see two paths to each destination,
one via the RS, the other via the bi-lateral BGP peer.
21. Removing non-RS BGP sessions. Once the peering session has been established with the Route
Server and prefixes from all ASNs are being received from the Route Server, then each router
team can remove their direct eBGP configurations with each AS at the IXP. While many ISPs
choose to continue with bi-lateral peering configuration as well as the Route Server peering, this
step here shows what happens when just the RS is used for the eBGP sessions at the IX:
router bgp 100

9
!
22. Local Preference on RS supplied prefixes. The prefixes heard from the Route Server all have the
default local preference of 100. What each team should do now is use the previous route-map set-
local-pref an applied it to the BGP peering with the RS. For example:
router bgp 102

and then refresh the BGP session with the route-server. All the prefixes learned from the route-
server will now have the local-preference set to 150.
23. Filtering RS supplied prefixes. It is generally quite a good idea not to trust what the route-server
is sending to you. Each team should now take the individual prefix filters they had for each
neighbour and replace them with a combined prefix filter for the route-server. This way each AS
is still filtering what it receives from each AS. Note that in practice, many IXPs operating a route-
server will filter the prefixes they receive from each participant in the route-server, and several
more will filter what they send to each participant as well. This all helps with the integrity of the
IXP peering mesh. For example:
router bgp 104

neighbor 172.17.10.254 prefix-list RS-prefixes in
!
ip prefix-list RS-prefixes permit 10.1.0.0/20
!
Don’t forget to tidy up unused configuration by removing the individual prefix-lists used in the
previous section for the individual (bi-lateral) peerings.
Checkpoint #4: Now look at your BGP routing table. You should only see prefixes from each AS, but
learned via the route server.
24. Summary. This module has given example configurations as used by Internet Service Providers at
Internet Exchange Points. It has concentrated on using prefix-lists only – more sophisticated
configurations are possible by using as-path filters and BGP communities. These examples are left
to the reader to consider. If there is time at the end of the workshop, ask the Instructor to test out
some other scenarios.
10
ISP Workshop Lab

Prerequisites: Modules 12 and 13, and the Exchange Points Presentation
1
Lab Notes
It has seven ASNs participating at the IX, with two routers being assigned to each autonomous system.
One router peers at the IX, the other router is internal to the autonomous system (so will have an
OSPF and iBGP session with the IX facing router). It also has three routers operating as three transit
provider or Tier1 ISPs. This version of the module is incremental over Module 10 and covers correct
OSPF and iBGP configuration practices for connecting the IXP facing router to the rest of the ISP
network.
Lab Exercises
1. A More Advanced IXP. This IXP example has seven participating ASNs with each ASN having
two routers. One router peers at the IXP; the other router is internal to the ISP’s network and
connects to an upstream ISP. The two routers in each ASN present at the IXP use OSPF and iBGP
to communicate routing information with the each other. The three upstream ASNs are also
interconnected with each other, to represent the high speed Internet core. What we have done here
is to simulate an Internet Exchange Point and its participants’ connections to upstream ISPs.
depicted in Figure 1. Check all connections. Note that all links to the IX switch are by Ethernet.
The links between each router in each AS is by serial connection (to represent the WAN
connection from the IX back to the ISP’s core network). The links from each AS to their upstream
is by Ethernet (to represent the WAN link to the upstream).
3. Addressing Plan. These address ranges should be used throughout this section of the module. You
are welcome to use your own range within an AS if you desire, just so long as you consult with the
teams in other ASes to ensure there is no overlap.
AS100 10.1.0.0/20 AS105 10.6.80.0/20

AS101 10.2.16.0/20 AS106 10.7.96.0/20
AS102 10.3.32.0/20 AS16384 10.8.112.0/20
AS103 10.4.48.0/20 AS16385 10.9.128.0/20
AS104 10.5.64.0/20 AS16386 10.10.144.0/20
basic security, IP addressing etc. The addressing for the point-to-point link between the two routers
in each AS is left as an exercise to the Router Teams. But remember, a point-to-point link requires
a /30 address block. Don’t forget to configure the loopback interfaces too.
5. IXP LAN. The address range used for the IXP LAN is 172.17.10.0/24 – the route server (if
provided in this module) has an IP address of 172.17.10.254. Each of the ASes is assigned
addresses sequentially for use on the exchange point LAN. So, for example, AS100 has
172.17.10.1; AS101 has 172.17.10.2; AS102 has 172.17.10.3. And so on.
2
ISP Workshop Lab

ip address 172.17.10.1 255.255.255.0
no ip proxy-arp
no ip redirects
!
7. Configuring OSPF (part 1). Obviously each IXP team will need to configure OSPF between the
two routers in their AS. The router which is internal to the ASN’s network should be
straightforward to configure. It has one loopback interface, one interface connecting to the router
at the IXP, and one interface connecting to the upstream. Therefore OSPF should have the
loopback and the one internal interface configured. Note we do not configure the interface pointing
to the upstream as we will be using next-hop-self for our iBGP sessions. Example for Router 11
might be:
router ospf 105

!
interface serial 1/0 ! link to Router 12
ip ospf 105 area 0
!
ip ospf 105 area 0
!
8. Configuring OSPF (part 2). The OSPF configuration on the router connecting to the IXP needs
more care. It is highly important that the IXP LAN network is not advertised into the ISP’s iBGP.
If it does, there is the possibility of leaking the IX LAN network by the ISP’s BGP to other ASNs,
providing transit to the IXP. This might not be a good thing, and many IXPs have rules against this
behaviour. What we do want to do is include the IX LAN in the OSPF, and like the loopback
interface, we mark the IX LAN interface as passive in OSPF.
The OSPF configuration for the IXP router looks something like this, for example for Router 8:
router ospf 103

!
interface serial 1/0 ! link to Router 7
ip ospf 103 area 0
!
ip ospf 103 area 0
3
!
ip ospf 103 area 0
!
NB. Do NOT mark OSPF as “no passive” on the Ethernet interface. Do NOT mark it as “no
passive”, repeat, do NOT! It is NOT a mistake.
9. Configuring iBGP between upstream facing routers and the IXP facing router. Any router
which is connected to other ASNs (the ISP border routers) such as the ISP’s upstreams or ISP’s
peers needs to have a modified iBGP configuration when peering with the IXP facing router. If all
the prefixes learned from the upstream ISP are passed onto the IXP facing router, then there is the
possibility that the IXP peers (and non-peers) can point static routes for those destinations at the
IXP facing router, thereby gaining outbound transit across the local ASN. This is undesirable, and
is basically a security risk. The configuration in the previous step therefore needs to be modified
for IXP facing routers (the only router in this lab module example) so that external prefixes are not
announced to the IXP facing router. In real life, ISPs create a peer-group which is used only when
peering with IXP-facing routers, for example:
router bgp 103

no synchronization
network 10.4.48.0 mask 255.255.240.0
neighbor ibgp-ixp peer-group
neighbor ibgp-ixp remote-as 103
neighbor ibgp-ixp update-source loopback0
neighbor ibgp-ixp next-hop-self
neighbor ibgp-ixp password cisco
neighbor ibgp-ixp description iBGP peering with IXP routers
neighbor ibgp-ixp send-community
neighbor ibgp-ixp filter-list 10 out
neighbor router8-loopback peer-group ibgp-ixp
no auto-summary
!
ip route 10.4.48.0 255.255.240.0 null0
!
!
Note the addition of the as-path access-list 10 – this basically allows only locally originated
prefixes to reach the IX facing router. For example, Router7 with this configuration will now no
longer send externally learned prefixes to Router8, thereby guaranteeing the security of the AS103
network.
10. Configuring iBGP on IXP facing routers (Part 1). The iBGP configuration on the IXP facing
router needs much more care. We do NOT want the IXP LAN to appear in our iBGP. And because
the IXP LAN is not in OSPF, we can’t use this net for valid next-hops. So that BGP has a valid
next-hop, we again use the next-hop-self BGP configuration as we have done for our other
iBGP sessions earlier on. For example, for Router 6:
router bgp 102

no synchronization
neighbor ibgp-peers update-source loopback0
neighbor ibgp-peers next-hop-self ! set external next-hops to us
4
ISP Workshop Lab

neighbor ibgp-peers description iBGP peers
neighbor router5-loopback peer-group ibgp-peers
no auto-summary
!
This configuration is also valid for other iBGP peers in the network if there were more than two
routers in the ASN. ISPs try and keep configuration differences to the absolute minimum, and
having two different peer-groups defined for iBGP is usually sufficient!
11. Configuring iBGP on IXP facing routers (Part 2). The second important difference required for
the iBGP at the IXP router is that the local address block must NOT be originated here. If the IX
router is disconnected from the core backbone for whatever reason, if it originated the ISP’s
address block, it would end up blackholing all traffic from the IXP participants to the local
network. This is highly undesirable, so the correct procedure is to originate the ISP address block
only in the core of the network, not the edge, so that normal BGP failover rules can apply.
Q. What do you think the normal BGP failover rules might be in this case?
A. If the IXP facing router is disconnected from the core network, the iBGP fails, the OSPF fails.
So the router no longer hears the announcement of the ISP’s address block from the core, so no
longer announces it to the IXP peers, its eBGP neighbours. When this happen, the other ASNs will
use fall back paths (alternative to the IXP) to reach the local ASN. These are the normal BGP
failover rules.
12. Configuring iBGP on the IXP facing routers (Part 3). The second very important requirement
at an Internet exchange point is that a default route (or the routes not originated by the local ASN)
should not be made available at the peering LAN. If this requirement is not followed, then it is
possible for the Exchange Point participants to use the local network to get transit to the rest of the
Internet.
In this Module, each ASN will be receiving a default route from their upstream provider (a
common scenario in the Internet today). This default route must NOT be made available at the
ISP’s Exchange Point router. There are two ways to do this. Either block the announcement of the
default route by iBGP at the borders router which connect to the upstreams, or set up a static
default route to the Null interface at the IXP router (or indeed do both!). For example:
ip route 0.0.0.0 0.0.0.0 null0
Then if any of the IXP participants point a default route to the local network, the traffic will simply
be dumped in the Null interface of their IXP router. Only traffic for specific destinations which
available in the routing table on the IXP router will be forwarded to the rest of the network. This is
a very important network security requirement.
13. Configuring eBGP on the IXP facing routers. Next, eBGP needs to be set up on the IXP routers.
Create a peer-group and apply that peer-group to each eBGP neighbour. A sample configuration
for Router8 might be:

..
5

!
router bgp 103
no synchronization
..
no auto-summary
!
!
Note the prefix-lists. There is a per-peer inbound prefix-list. Some service providers only filter
14. Set up passwords on the eBGP sessions at the IXP. Negotiate with each ASN a password which
you can use on your BGP session with them. And then agree to cut the eBGP session over to using
Module 2). An excerpt from Router 8’s configuration might be:
router bgp 103

...
...
!
15. Configuring eBGP between the routers in AS16384, AS16385 and AS16386. The instructors
will operate the three transit/Tier1 ASNs; they will now configure eBGP between each other. No
filtering is expected or required between the three ASNs, so the eBGP configuration is quite
straightforward, for example as on Router 15:
router bgp 16384

no synchronization
6
ISP Workshop Lab
no auto-summary
!
16. Configuring eBGP with the routers in AS16384, AS16385 and AS16386. All IXP members
must now configure eBGP sessions with the routers in the three transit ASNs. They must expect to
receive the default route from their upstreams, and only send their address block to the upstream.
An example configuration for Router 3 might be:

!
router bgp 101
no synchronization
no auto-summary
!
Router9#trace 10.7.112.1
1 10.5.64.6 4 msec 4 msec 4 msec

2 172.17.10.4 4 msec 4 msec 4 msec
3 10.7.96.1 [AS 106] 4 msec * 4 msec
Router9#
STOP AND WAIT HERE
18. Implementing a Route Server at the IXP. With route server functionality added to Cisco IOS as
from the 15.2 release, it is now possible to operate Router 18 as a route server. The lab instructors
will have configured Router 18 to be a route server.
The following information is included for the information of each router team member only. For
those in the workshop interested in setting up an Internet Exchange Point following the completion
of the workshop, this section contains the information necessary to configure and operate a Cisco
IOS based Route Server.
The Route Server has eBGP sessions configured with each member of the IXP in this workshop
lab (in real life, the Route Server has an eBGP session configured with those memebrs of the IX
7
who wish to use its facilities.) Each eBGP session is configured with the “route-server-client”
command included in the eBGP peer-group.
This is an excerpt from the configuration of Router 18 after it has been configured to be a route
server:
router bgp 65534

neighbor ixp-peers description Route Server peers
neighbor ixp-peers password cisco
!
address-family ipv4
neighbor ixp-peers route-server-client
exit-address-family
!
Notice that in this simple route-server example, there are no BGP filters at all – it is left to the IXP
member to do necessary filtering.
This is just a simple example. Most IXPs will insist on the route server doing the basic bogon
filtering which has been discussed elsewhere during this workshop. Some IXPs operate their route
servers to filter according to the information contained in the Internet Routing Registry.
19. Set up eBGP session with Router 18. Each team member should now configure their router to
peer eBGP with the route server. We will leave the existing bi-lateral peering sessions in place for
the time being. Here is an example configuration:
8
ISP Workshop Lab
router bgp 103

...
neighbor 172.17.10.254 description eBGP with the IX Route Server
...
Notice that the route server is running in a private AS – there isn’t really any need for it to use a
public AS as the Server does not need to be directly visible outside of the IXP.
Note also that there is no inbound prefix filter. This is not normally the case for an IXP. Even if an
ISP peers with the Route Server, they should still filter inbound BGP prefixes they receive via the
Route Server, just in case a peer who uses the Route Server advertises prefixes they should not be
advertising.
20. Modifications to the ISP’s IXP routers to talk to the Route Server. Route servers, as a rule, do
not include their AS number in BGP announcements to their clients; they are simply there to
distribute the BGP table they have learned from their clients. Historically RSd and routing
software platforms such as Zebra and Quagga have always behaved this way when running in
route server mode. Some IXPs used routers instead of these software routing implementations, and
routers have not had the functionality to drop the local ASN, meaning that the Route Server’s AS
has appeared first in the AS path.
Cisco routers participating at IXPs with the software based route servers have long had a capability
to accept peerings with a full Route Server AS – but it is not IOS’s default behaviour. IOS’s
default behaviour is to check that all prefix announcements from an eBGP peer have the peer AS
as the first AS in the AS path. To support an eBGP peering where the Route Server AS is not
included in the AS path, the router needs the following configuration:
router bgp 103

no bgp enforce-first-as
...
which stops the router rejecting the prefix updates if the peer AS is not first in the AS path. You
will notice that before entering this command, the BGP peering will be up, but no prefixes will be
received by the ISP’s IXP router – and the router logs will have messages about malformed
attributes being sent by the Route Server.
Checkpoint #3: Now look at your BGP routing table. You should see two paths to each destination,
one via the RS, the other via the bi-lateral BGP peer.
21. Removing non-RS BGP sessions. Once the peering session has been established with the Route
Server and prefixes from all ASNs are being received from the Route Server, then each router
team can remove their direct eBGP configurations with each AS at the IXP. While many ISPs
choose to continue with bi-lateral peering configuration as well as the Route Server peering, this
step here shows what happens when just the RS is used for the eBGP sessions at the IX:
router bgp 100

9
!
22. Local Preference on RS supplied prefixes. The prefixes heard from the Route Server all have the
default local preference of 100. What each team should do now is use the previous route-map set-
local-pref an applied it to the BGP peering with the RS. For example:
router bgp 102

and then refresh the BGP session with the route-server. All the prefixes learned from the route-
server will now have the local-preference set to 150.
23. Filtering RS supplied prefixes. It is generally quite a good idea not to trust what the route-server
is sending to you. Each team should now take the individual prefix filters they had for each
neighbour and replace them with a combined prefix filter for the route-server. This way each AS
is still filtering what it receives from each AS. Note that in practice, many IXPs operating a route-
server will filter the prefixes they receive from each participant in the route-server, and several
more will filter what they send to each participant as well. This all helps with the integrity of the
IXP peering mesh. For example:
router bgp 104

neighbor 172.17.10.254 prefix-list RS-prefixes in
!
!
Don’t forget to tidy up unused configuration by removing the individual prefix-lists used in the
previous section for the individual (bi-lateral) peerings.
Checkpoint #4: Now look at your BGP routing table. You should only see prefixes from each AS, but
learned via the route server.
24. Summary. This module has given example configurations as used by Internet Service Providers at
Internet Exchange Points. It has concentrated on using prefix-lists only – more sophisticated
configurations are possible by using as-path filters and BGP communities. These examples are left
to the reader to consider. If there is time at the end of the workshop, ask the Instructor to test out
some other scenarios.
10
ISP Workshop Lab
Module 17 – Overseas Co-location

Objective: To investigate methods for connecting to Internet backbones overseas.
Prerequisites: Modules 12, 14 and (optionally) 16, and the Co-location Presentation
Figure 1 – International Configuration
1
Lab Notes
The purpose of this module is to investigate the principles and practices surrounding overseas co-
location. The BGP presentation should be reviewed during the study of this module as it provides
some of the technical motivation behind locating equipment overseas.
This example assumes an Asian ISP looking for co-location space in the west coast of the United
States, but it can apply equally well to any ISP operating a network which interconnects to other
networks overseas.
Lab Exercise
1. Basic Configuration. Each router team should configure their router to fit into the network
topology depicted in Figure 1 and physical layout depicted in Error! Reference source not
found.. Check all connections.
2. Address Ranges. These address ranges should be used throughout this module. You are welcome
other ASes to ensure there is no overlap.
AS1 118.11.0.0/19 AS6 119.64.0.0/19

AS2 118.35.0.0/19 AS7 119.99.0.0/19
AS3 118.76.0.0/19 AS8 120.10.0.0/19
AS4 119.13.0.0/19 AS9 120.19.0.0/19
AS5 119.58.0.0/19 AS10 120.73.0.0/19
3. Basic Router Setup. With the exception of the routers in AS8, set up the routers as you would
have done in previous modules. That is, basic security, the BGP outline configuration, IOS
Essentials, etc. Routers AS9 and AS10 should do the basic configuration for their routers – don’t
set up eBGP to AS8 yet.
4. AS8 routers. The routers making up AS8 should set up OSPF and iBGP in the AS. Note the two
links between Routers 8 and 10. Both of these should be activated – they are intended to simulate
the transoceanic links which many ISPs install to reach the US Internet from Asia. Ask the lab
instructors for any longer cables if they are available. Routers 8 and 10 should be route reflectors
for the colocate and domestic network respectively. Using a route reflector is more efficient than a
full mesh iBGP, certainly at this level. Note that Router 8 and 10 have normal iBGP between
them.
Hint: Router8 is a reflector for Router7 and Router9, Router10 is a reflector for Router12 and
Router14.
Hint: The overseas routers in AS8 MUST NOT originate the AS8 address block. Why not?
5. IXP Participants. Those routers participating in the IXP (Routers 1 to 6) should use the
120.5.10/24 network for the IP addresses of the IXP LAN. As in Module 16, ASes 1 to 6 should
set up eBGP between each other. Remember how the configuration was implemented in Module
19? Only announce your prefix, only accept your peer’s prefix, uRPF checks, etc. If in doubt,
please refer to your notes from Module 16.
2
ISP Workshop Lab
reachable (i.e. you can ping the other routers), please let the instructor know. Routers in AS8 should
have iBGP and OSPF set up, and all the routers visible and pingable.
Configuring the Peerings with the Exchange Point Participants
6. Configure AS8 relationship with the IXP. The Domestic Internet Exchange Point provides AS8
with access to the regional domestic market at the co-locate site. This assumes of course that all
the service providers at the IXP would be willing to peer. (In practice they will be as they gain a
valuable access to an overseas market without paying their transit provider, and it is good for the
overseas provider not having to pay so much for transit to their upstream. Win-win situation for
both AS8 and the IXP participants.)
7. Router6 Route Reflector configuration detail. AS8 has to be extremely careful how it peers with
the IXP participants. Recall from the presentation that Router7 absolutely must not have a default
route or the full routing table on it. (Why not?) This is ensured by careful configuration of the
route reflector Router8 – it only announces AS8 and it’s customer prefixes to Router7, nothing
more. The team operating Router8 should now configure their router so that it only sends AS8
prefixes to Router7. The easiest way for this module is to use an outbound prefix-list. An example
configuration for Router8 might be:

!
router bgp 8
neighbor <router10> description My Home Route Reflector
neighbor <router7> description Router at the PacificIX
neighbor <router7> route-reflector-client
neighbor <router7> prefix-list myprefixes out ! NOTE THIS LINE
neighbor <router9> description Router connecting to my upstreams
!
A prefix-list was used rather than a filter-list. Why? A filter list on filters ASes – it does not filter
prefixes. Remember that AS9 and AS10 were announcing the default route to AS8 – the default
route would not be blocked by a filter list.
The alternative to using a single prefix-list as above is to use a combination of prefix-list and filter-
list. The prefix-list would block the default prefix, the filter-list would only allow the local and
customer ASes. In this case the configuration for Router8 might be:
ip prefix-list nodefault deny 0.0.0.0/0

!
!
router bgp 8
neighbor <router10> description My Home Route Reflector
neighbor <router7> description Router at the PacificIX
3
neighbor <router7> prefix-list nodefault out ! NOTE THIS LINE

neighbor <router7> filter-list 10 out ! NOTE THIS LINE
neighbor <router9> description Router connecting to my upstreams
!
Note that as-path access-list 10 can be added to for however many customer ASes AS8 has. AS9
and AS10 must not be included in this list.
It is important not to announce AS9 or AS10 prefixes to Router8 either. If IXP participants
discover that there is a path through AS8 to AS9 or AS10 they may use that rather than their own
paths – costing AS8 money.
Finally, all this can be easier to handle/manage using communities. That configuration is left as an
exercise to the reader!
8. Router 7 configuration at the IXP. As with the IXP Module, care is required configuring a router
participating at any IXP. Router8 is only sending local prefixes to Router7, so this ensures some
degree of safety. The team operating Router7 should now configure the router to peer with the IXP
participants. Remember the concepts from Module 19. Only announce your prefixes to the IXP,
only accept the prefixes your peers are entitled to send you.
9. Connectivity Test. Check connectivity throughout the IXP network. Each router team in the IXP
and AS8 should be able to see all the other routers at the IXP. When you are satisfied that BGP is
working correctly, try running traceroutes to check the paths being followed.
Checkpoint #2: Once the BGP configuration has been completed for AS8, check the routing table and
ensure that you have complete reachability from AS8 to the IXP network. If there are any problems,
work with the other router teams to resolve those.
Configuring the Links to the Backbone Providers

10. Configure AS9 and AS10 relationship with AS8. AS9 and AS10 are the upstream US Tier One
ISPs of AS8. Basically AS8 has bought Internet transit from these two ISPs. Why two? If one has
service problems, the other provides “backup” or redundancy. The configuration for Router 11 and
Router 13 are very similar to what we have covered in earlier modules. Basically Router 11 and
Router 13 treat AS8 as a customer, so announce only the default to the customer, and only accept
the customer’s prefixes. The teams operating Router11 and Router13 should now set up the serial
interface connecting to AS8 and configure eBGP on their routers. Note that it is common
convention that the point to point link between backbone ISP and their customer comes from the
ISP address block…
11. Configure Router7’s eBGP peering with AS9 and AS10. The team operating Router9 should
configure eBGP peering with AS9 and AS10 routers. Don’t forget the good practices learned
earlier. You want to announce only your prefix, and only accept default from the upstream. It is
also good practise to disable vulnerable services on serial interfaces of the router. For example:

description 2MBps connection to AS9
ip address 120.19.31.2 255.255.255.252
4
ISP Workshop Lab
no ip proxy-arp
no ip redirects
!
Once the interfaces to AS9 and AS10 are functioning (you can ping the other end of the link),
eBGP should be set up. An example might be:

!
router bgp 8
neighbor <router11> description Connection to AS9 Transit Provider
neighbor <router13> description Connection to AS10 Transit Provider
!
Note one thing. Router 9 DOES NOT originate AS8’s prefix. Indeed, none of the routers in the
overseas part of AS8 should originate AS8’s prefix. If the transoceanic cable is broken for
whatever reason, the routers in the overseas part of AS8 will still announce the aggregate – this
will create a blackhole for AS8 traffic if AS8 has another Internet connection elsewhere in its
backbone.
12. Connectivity Test. Check connectivity throughout the entire network. AS9 and AS10 should only
be able to see AS8. AS8 should be able to see everything. The IXP participants should be able to
see each other, but not AS9 or AS10. When you are satisfied that BGP is working correctly, try
running traceroutes to check the paths being followed.
The Transoceanic Circuit(s)
13. Loadsharing on the Transoceanic Circuits. If OSPF has been set up properly traffic on the two
“transoceanic” links should be load shared. To test this, either try running traceroute from
domestic to co-locate parts of the network, or connect laptops to various points of the network and
send data across the backbone. If loadsharing doesn’t seem to be working, check the OSPF
configuration – did you remember to set the bandwidth on the interface to match the clockrate on
the circuit, for example? Disconnect one of the “transoceanic” cables and see what happens.
Routing should failover gracefully.
14. Satellite links. One of the circuits will now be converted to simulate a simplex satellite connection
(uni-directional link). The dotted cable in Figure 1 will be the satellite connection. Policy Routing
will be used to send delay insensitive traffic over that connection, with the remaining traffic going
over the “submarine” cable. Refer to Module 8 on policy routing if you don’t remember how to
configure policy routing in IOS. Before configuring policy routing, lower the clockrate on the
“circuit” from 2000000 to 64000bps. Don’t forget to change the “bandwidth” command on the
5
interfaces at either end. And remove the circuit from OSPF – it is unidirectional, and we will be
using policy routing to put traffic on to it.
15. Configuring Policy Routing. Policy Routing will now be configured on Router 8 to manage the
two links back to the domestic network. Basically port tcp/80 will be redirected over the “satellite”
connection. An example configuration might be:
access-list 1 permit tcp any any eq www

!
route-map divert-web permit 10
match ip address access-list 1
set interface ser 0/1
route-map divert-web permit 20
!
description connection to Router9
ip policy route-map divert-web
no ip proxy-arp
no ip redirects
!
description connection to Router8
ip policy route-map divert-web
no ip proxy-arp
no ip redirects
!
description Satellite connection Home
ip address 120.10.31.2 255.255.255.252
no ip proxy-arp
no ip redirects
!
16. Connectivity Test. When configured, test the set up by attempting to connect to a webserver
which the workshop instructors will have connected to AS10. What happens?
Checkpoint #4: Once the policy routing configuration has been completed, check the routing table
and ensure that you have complete reachability over the entire network. If there are any problems,
work with the other router teams to resolve those. Only tcp/80 will be diverted over the satellite link.
17. Summary. This module has given a detailed example of how an ISP would configure an overseas
co-locate site. It has highlighted (again) the care required when peering at a public exchange point,
and given the necessary configuration tips to ensure a successful peering. It has pointed out how to
connect to an upstream ISP, and how to configure a transoceanic connection back to the domestic
network. It has also briefly looked at some of the possibilities for using a satellite based
transoceanic connection for non-delay sensitive traffic.
6
ISP Workshop Lab
Module 18 – Multihoming Case Study

Objective: Starting from a single homed ISP, the objective is to build a network where the local
ISP is multihomed to their upstream and to a local peer.
Prerequisites: Modules 11, 12 and 14
1
Lab Notes
The purpose of this module is to construct a network building from a very basic singlehomed scenario
to end with a situation where the ISP is multihomed between their upstream and another local ISP.
This module demonstrates the good principles of configuring eBGP to not provide transit to local
peers, and configuring iBGP to ensure that functions with the correct next-hop values. The further aim
of the module is to show how straightforward it is to transition from a single homed connection to an
upstream provider, to having a true multihomed configuration for the local AS.
Lab Exercise
1. Remove any configuration from the previous Modules. Tidy up from the previous Modules so
that the router has no configuration on it. Either do a “write erase” and then reboot the router
(strongly recommended) or carefully remove the IP addressing, OSPF and BGP configuration
from previous modules (not really recommended as classroom teams invariably leave some old
configuration on the router, causing confusion further on in the module).
depicted in Figure 1. Check all connections. Note that most links are using Ethernet cables.
Remember what was covered in Module 11! The workshop instructors will configure router15 to
provide the necessary network configuration (they will reconfigure one of the Ethernet switches to
use .1q VLAN trunking, thereby making Router15 appear as though it has 14 Ethernet ports on it).
to use your own range within your AS if you really desire, just so long as you consult with the
teams in other ASes to ensure there is no overlap, and that you let the workshop instructors know
that you’ve used different address blocks. In the every day Internet, such address assignment is
carried out by the Regional Internet Registries. AS100 and AS200 are the upstream providers used
in this module. A /16 network block has been assigned to those providers – you will find out what
this block is later on in this module.
AS1 10.10.0.0/19 AS8 10.33.32.0/19

AS2 10.10.32.0/19 AS9 10.47.0.0/19
AS3 10.19.0.0/19 AS10 10.47.32.0/19
AS4 10.19.32.0/19 AS11 10.65.0.0/19
AS5 10.23.0.0/19 AS12 10.65.32.0/19
AS6 10.23.32.0/19 AS13 10.84.0.0/19
AS7 10.33.0.0/19 AS14 10.84.32.0/19
4. Addressing the router. Each router team should come up with an appropriate addressing plan for
their router. Remember that a Loopback interface will be required (for BGP router-id), as well as
addresses for future point to point links. The lab instructors will have assigned point-to-point link
addresses from Router15 to each of the classroom routers. The individual address blocks for these
are as follows – Router15 uses the low address of the /30, the classroom router uses the high
address:
2
ISP Workshop Lab
Router1 192.168.250.0/30 Router2 192.168.251.0/30

Router3 192.168.250.4/30 Router4 192.168.251.4/30
Router5 192.168.250.8/30 Router6 192.168.251.8/30
Router7 192.168.250.12/30 Router8 192.168.251.12/30
Router9 192.168.250.16/30 Router10 192.168.251.16/30
Router11 192.168.250.20/30 Router12 192.168.251.20/30
Router13 192.168.250.24/30 Router14 192.168.251.24/30
5. Static Routes. With the basic configuration in place on the router, set up a static route so that you
can ping the upstream (AS100 or AS200) router. Also talk to the lab assistant who is configuring
the upstream router to ensure that they will put in a static route which allows them to see your
router. Note that the odd numbered routers are connecting to AS100, the even numbered routers
are connecting to AS200.
6. Summary. Once all the configuration has been completed, each router team should be able to ping
every router in the classroom. This is quite a typical scenario, where a transit ISP is providing
connectivity to several customer networks, with single-homing only.
STOP AND WAIT HERE
Scenario One – eBGP with upstream

7. Set up eBGP with upstream ISP. For the first exercise we are simply going to configure eBGP
with our upstream ISP, AS100 or AS200.This typifies the situation where an ISP is only connected
to a single upstream, and is about to consider the migration to a multihoming situation. An extract
of a configuration might be:
!
router bgp 10
network 10.47.32.0 mask 255.255.224.0
neighbor x.x.x.x description Peering with Upstream AS200
neighbor x.x.x.x password cisco
!
ip route 10.47.32.0 255.255.224.0 null0
Note that the upstreams AS100 or AS200 will only originate a default route. They will not
announce any routes apart from the default to you as you don’t need any other routing information
to see the whole Internet. Also note that we are not using any prefix filtering here. Prefix filtering
will be covered later on in this module.
Don’t forget the required BGP configuration, so items like disabling synchronization and auto-
summarisation, enabling logging, and setting the BGP distances appropriately. If you have
forgotten, refer to your configuration in Module 11 and subsequent modules. Note that the odd
numbered routers are connecting to AS100, the even numbered routers are connecting to AS200.
3
8. Remove Static Routes. Once the BGP peering is up and running, remove the static default route
pointing towards your upstream provider. Also talk to the lab assistant who is monitoring the
upstream provider router (Router 15) so that they will remove the static route pointing to your
network. This step represents the step when you migrate from using static routing with your
upstream to using BGP. Once you see your prefix in the Internet Routing Table, and are seeing the
default from the upstream by BGP, you then know that you can safely remove the static routes
which were set up.
9. Status of BGP peering. Use the BGP show commands to find out what is in your BGP table.
What do you see? You should see a default route from your upstream, your own address block and
the address blocks from the other ASNs transited by AS100 and AS200. If this is not the case,
either try and work out what is wrong, or ask the lab assistants to help you debug the problem.
10. Status of connectivity. Use the “show ip route” commands to find out how routing is on the
system. Also use trace and ping to check the routing to other ASNs in the classroom. You should
find that AS100 and AS200 are providing transit to all the other ASNs in the network.
STOP AND WAIT HERE
Scenario Two – Adding in local peer.

11. eBGP with local peer. We now need to add configuration to the network such that ASx sets up a
local eBGP session with ASx+1 – in other words, AS1 needs to set up eBGP with AS2, AS3 needs
to set up eBGP with AS4, and so on. This situation is depicted in Figure 2. The connections
between neighbour ASNs are formed using serial cables, connecting the serial interfaces on each
router. The lab instructors will have already connected the cables between the routers before you
start this exercise.
12. Serial Connections. Verify the serial connections to the neighbouring router. Establish whether
you have the DTE or DCE cable connected to your router (if the latter, you will need to provide
clock for the “circuit”). Agree on the IP addresses for the point-to-point link (remember, you will
use a /30).
13. Configuration. Leaving the existing eBGP configuration with your upstream on the router, now
add in the necessary eBGP configuration to your local peer. An example configuration might be:
!
router bgp 7
neighbor y.y.y.y remote-as 8
neighbor y.y.y.y description Peering with Local Peer AS8
neighbor y.y.y.y password cisco
!
ip route 10.33.0.0 255.255.224.0 null0
4
ISP Workshop Lab
Note that we still have no prefix filtering here. The purpose is to demonstrate how eBGP can go
really badly wrong if no filters are used. This configuration must NEVER be used on the public
Internet without proper filters.
Figure 2 - Peering with local peer
5
What do you see? You should see a default route from your upstream, your own address block, the
address block announced to you by your neighbouring AS, as well as the address blocks from all
the other ASNs in the network. If this is not the case, either try and work out what is wrong, or ask
the lab assistants to help you debug the problem.
system. Also use trace and ping to check the routing to other ASNs in the classroom. How do you
reach the other ASNs in the classroom? Why?
STOP AND WAIT HERE
Scenario Three – Filtering using AS PATH filters.

16. Filtering BGP sessions using AS-PATH. So far we have used no filtering whatsoever on the
BGP sessions. And as you saw, the BGP table ended up in a significant mess, with surplus
information from the upstream ISPs AS100 and AS200, and with your local peering AS providing
you with free transit to other ASNs in the network. The latter situation is commercially a disaster
for any ISP, the former simply unnecessarily wastes lots of router memory.
17. Configuration. Leaving the existing eBGP configuration with AS100/200 and your local peer on
the router, now add in the necessary configuration so that you are using AS-PATH filters on your
eBGP peerings. AS100/200 should be filtered so that only the prefixes it originates (the default
route) is permitted in. The peer AS should also be filtered in a similar fashion. An example
configuration might be:
!
router bgp 7
neighbor x.x.x.x filter-list 1 out
neighbor x.x.x.x filter-list 2 in
neighbor y.y.y.y filter-list 1 out
neighbor y.y.y.y filter-list 3 in
!
ip as-path access-list 2 permit _100$
ip as-path access-list 3 permit _8$
!
Note that the neighbouring AS will only announce the prefixes they originate. Also note that the
upstream will only send you a default route – a default route implies the whole Internet so there is
really no need to get any further information from the upstream. The inbound AS-PATH filter is
really important. You simply cannot assume that your eBGP peers will do prefix filtering as you
are doing, so it is vitally important that you have inbound (and outbound) route filters on your
eBGP sessions.
6
ISP Workshop Lab
What do you see? You should see a default route from your upstream, your own address block,
and the address block announced to you by your neighbouring AS. If this is not the case, either try
and work out what is wrong, or ask the lab assistants to help you debug the problem.
find that AS100 & AS200 are providing transit to all the other ASNs apart from the local ASN you
are peering with.
STOP AND WAIT HERE
Scenario Four – Filtering using prefix filters.

20. Filtering BGP sessions using prefix filters. Remove the AS-PATH filters and their application in
the eBGP peering we used in the previous scenario. We are now going to look at a more strict
filtering mechanism, using prefix-lists. While these are higher maintenance, they are generally
preferred, as they don’t imply a trust of what the peer AS is introducing into the routing system.
21. Configuration. Leaving the existing eBGP configuration with AS100 and AS200 as appropriate
and with your local peer on the router, now add in the necessary configuration so that you are
using prefix filters on your eBGP peerings. An example configuration might be:
!
router bgp 7
neighbor x.x.x.x prefix-list ourblock out
neighbor x.x.x.x prefix-list default in
neighbor y.y.y.y prefix-list ourblock out
neighbor y.y.y.y prefix-list AS8block in
!
ip prefix-list ourblock permit 10.33.0.0/19
ip prefix-list AS8block permit 10.33.32.0/19
!
Note that the neighbouring AS will only announce the prefixes they originate. Also note that the
upstream will only send you a default route – a default route implies the whole Internet so there is
really no need to get any further information from the upstream. The inbound prefix filter is really
important. You simply cannot assume that your eBGP peers will do prefix filtering as you are
doing, so it is vitally important that you have inbound (and outbound) route filters on your eBGP
sessions.
What do you see? You should see a default route from your upstream, your own address block,
7
and the address block announced to you by your neighbouring AS. If this is not the case, either try
and work out what is wrong, or ask the lab assistants to help you debug the problem.
find that AS100 and AS200 are providing transit to all the other ASNs apart from the local ASN
you are peering with.
24. AS-PATH and prefix-lists. Some ISPs merge the configurations we examined in Scenarios 3 and
4 so that they have prefix-lists and AS-PATH filters applied to each eBGP session. The reason for
doing this is for backup – if the prefix-list becomes broken for some reason, the AS-PATH filter
still provides some suitable filtering to protect the local ASNs routing system.
STOP AND WAIT HERE
Scenario Five – Providing backup for the local peer.

25. Background. The next exercise in the BGP labs is to modify the policy configuration on the local
routers so that the local peers provide backup for each other. The network layout is still as depicted
in Figure 2.
26. Configuration. The technique for providing local backup is a modification of the previous step.
The router in the local AS will accept the prefix announced by its local peer, and announce this
prefix to the upstream with a longer AS-PATH. In this example we will use a 4 times prepend. The
router in the local AS will also announce the default route to the local peer, again with a prepend –
we will use a 2 times prepend in this case.
27. Reconfiguring eBGP with the upstream. Because we want to send our local peer’s address block
to our upstream, we need to modify our filters to allow their address block out to our upstream.
But, unlike Scenario 2, we need to apply outbound policy on their address block so that they don’t
use us as the primary path into their network. To do this, we use a route-map with an as-path
prepend option. An example configuration might be:
!
router bgp 7
neighbor x.x.x.x prefix-list AS7out out
neighbor x.x.x.x route-map AS8-backup out
!
ip prefix-list AS7out permit 10.33.0.0/19
ip prefix-list AS8block permit 10.33.32.0/19
!
route-map AS8-backup permit 10
match ip address prefix-list AS8block
8
ISP Workshop Lab
set as-path prepend 7 7 7 7

route-map AS8-backup permit 20
!
Note that the local peer’s address block has been added to our outbound prefix-list to our
upstream. A separate prefix-list has been set up for their address block for use in the pre-pending
route-map. There is no way of concatenating prefix-lists in IOS at the moment.
28. Reconfiguring eBGP with our local peer. We also want to send the default route we hear from
our upstream to our local peer. It is very important that we transit the default route, rather than
originating a default route ourselves. If we do the latter, a failure in our upstream link will mean
that we end up blackholing our local peers traffic as well as our own. An example configuration
might be:
!
router bgp 7
neighbor y.y.y.y prefix-list AS8out out
neighbor y.y.y.y prefix-list AS8in in
neighbor y.y.y.y route-map default-prepend out
neighbor y.y.y.y route-map set-lp-low in
!
ip prefix-list AS8in permit 10.33.32.0/19
ip prefix-list AS8in permit 0.0.0.0/0
!
route-map default-prepend permit 10
set as-path prepend 7 7
route-map default-prepend permit 20
!
route-map set-lp-low permit 10
route-map set-lp-low permit 20
!
Note that the default route has been added to our outbound prefix-list to our local peer. Once you
have applied the new configuration, don’t forget to use route refresh to update the routing
information heard from and sent to your BGP peers.
What do you see? You should still see a default route from your upstream, a default route from
your local peer which is prepended by two of their ASN, your own address block, and the address
block announced to you by your neighbouring AS. If this is not the case, either try and work out
what is wrong, or ask the lab assistants to help you debug the problem.
30. Testing the backup. The lab assistants will now demonstrate what is seen on the two upstream
routers in AS100 and AS200. You should see your path learned directly from your router, as well
as a path with a 4 times prepend learned via your local peer. If this is the case, try disconnecting
the link between you and your upstream, and see if the backup works. If not, try and work out
what is wrong, or ask the lab assistants for help.
9
STOP AND WAIT HERE
Scenario Six – Adding iBGP.

31. Background. The final exercise in the BGP labs is to simulate a situation which is commonly
found in many service provider networks around the world. The situation is where an ISP
multihomes between two upstream ISPs, and uses two local routers for redundancy. This
necessitates using eBGP towards the upstreams, and iBGP between the two local routers. The
network layout is depicted in Figure 3.
32. Reconfiguring BGP between local ASNs. The eBGP session between ASx and ASx+1 should be
removed now. We will be converting this to iBGP. For example, the router in AS1 should remove
the eBGP peering with the router in AS2, etc.
33. Changing ASN. The routers in AS1, AS3, AS5, AS7, AS9, AS11 and AS13 should now
reconfigure their BGP session so that they are now members of AS2, AS4, AS6, AS8, AS10,
AS12, and AS14 respectively. There is no simple way of doing this on the router, apart from
simply deleting the entire BGP configuration and re-entering the configuration with the new AS
number. Note that these routers should still peer with their upstream AS100.
34. Implementing iBGP. We are now going to set up iBGP between neighbouring routers. The
important point to remember here is that iBGP runs between the loopback interfaces of the router.
For the loopback to be functional, OSPF needs to run on the router first. OSPF is used so that the
two routers in the network can work out how to reach each other.
35. Setting up OSPF. Make sure that the loopback interface is configured on the router. Now set up
OSPF – there is only one active interface, the interface pointing towards the neighbouring router in
the AS. Also make sure that the loopback interface address is configured in OSPF. Do not put the
external peering interfaces in OSPF (so those towards AS100 and AS200 respectively) as we will
be using the BGP next-hop-self concept to ensure reachability of external networks. A
configuration example might be:
router ospf 41
!
ip ospf 41 area 0
ip ospf message-digest-key 1 md5 #####
ip ospf 41 area 0
ip ospf message-digest-key 1 md5 #####
!
10
ISP Workshop Lab
Figure 3 - iBGP and eBGP

36. Setting up iBGP. Once you can ping the loopback interface of the neighbouring router in your
AS, set up iBGP between your and the neighbouring router. Because we are not carrying the
external facing addresses in OSPF, we need to use the BGP next-hop-self concept to ensure
reachability of external networks. An example configuration might be:
11
!
router bgp 8
network 10.33.0.0 mask 255.255.224.0
network 10.33.32.0 mask 255.255.224.0
neigbhor ibgp-peers send-community
neighbor x.x.x.x prefix-list ourblock out
neighbor x.x.x.x password cisco
neighbor y.y.y.y peer-group ibgp-peers
!
!
What do you see? You should see a default route from your upstream and your own address
blocks. If this is not the case, either try and work out what is wrong, or ask the lab assistants to
help you debug the problem.
find that either AS100 or AS200 is providing transit to all the other ASNs in the network.
39. Extra tasks. If you have time, notice that you can aggregate the two /19 address blocks used in
your ASN. So change the configuration on both router so that you are announcing a /18 rather than
two /19s. Change the filters and the BGP configuration to suit. Do you still have the same
connectivity? If not, investigate why not. Do you need to tell any BGP neighbours about your
changes?
40. Summary. This module has given an example of the kind of migration strategy and steps which
are necessary to move from a single statically routed connection to an upstream, to using a BGP
multihomed connection with two routers. Each router team has been encouraged to save their
configuration steps covered in each scenario – as these configuration steps are exactly those
required in a real live situation on the Internet.
12
ISP Workshop Lab
IPv4 ADDRESSING SCHEME – MODULES 1 to 5

Figure 1 below displays the addressing plan to be used for Modules 1 to 5. The plan itself is explained
in the notes accompanying the workshop Modules. Each subnet is a /30.
Figure 1 – Addressing Scheme for Modules 1 to 5
1
IPv4 Addresses – Modules 1 to 5

Router Loopback Address Router Loopback Address
R1 10.0.15.241 R8 10.0.15.248
R2 10.0.15.242 R9 10.0.15.249
R3 10.0.15.243 R10 10.0.15.250
R4 10.0.15.244 R11 10.0.15.251
R5 10.0.15.245 R12 10.0.15.252
R6 10.0.15.246 R13 10.0.15.253
R7 10.0.15.247 R14 10.0.15.254
Table 1 – IPv4 Loopback Address assigned to each Router in Modules 1 to 5
Router “Customer” Address Router “Customer” Address

R1 10.0.0.0/26 R8 10.0.1.192/26
R2 10.0.0.64/26 R9 10.0.2.0/26
R3 10.0.0.128/26 R10 10.0.2.64/26
R4 10.0.0.192/26 R11 10.0.2.128/26
R5 10.0.1.0/26 R12 10.0.2.192/26
R6 10.0.1.64/26 R13 10.0.3.0/26
R7 10.0.1.128/26 R14 10.0.3.64/26
Table 2 – IPv4 “Customer” Addresses assigned to each Router in Modules 1 to 5
2
ISP Workshop Lab
IPv4 Netmask Table

Network Host
Subnet Mask Wild-Card Mask Prefix
Bits Bits
0.0.0.0 255.255.255.255 0 32 /0
128.0.0.0 127.255.255.255 1 31 /1
192.0.0.0 63.255.255.255 2 30 /2
224.0.0.0 31.255.255.255 3 29 /3
240.0.0.0 15.255.255.255 4 28 /4
248.0.0.0 7.255.255.255 5 27 /5
252.0.0.0 3.255.255.255 6 26 /6
254.0.0.0 1.255.255.255 7 25 /7
255.0.0.0 0.255.255.255 8 24 /8
255.128.0.0 0.127.255.255 9 23 /9
255.192.0.0 0.63.255.255 10 22 /10
255.224.0.0 0.31.255.255 11 21 /11
255.240.0.0 0.15.255.255 12 20 /12
255.248.0.0 0.7.255.255 13 19 /13
255.252.0.0 0.3.255.255 14 18 /14
255.254.0.0 0.1.255.255 15 17 /15
255.255.0.0 0.0.255.255 16 16 /16
255.255.128.0 0.0.127.255 17 15 /17
255.255.192.0 0.0.63.255 18 14 /18
255.255.224.0 0.0.31.255 19 13 /19
255.255.240.0 0.0.15.255 20 12 /20
255.255.248.0 0.0.7.255 21 11 /21
255.255.252.0 0.0.3.255 22 10 /22
255.255.254.0 0.0.1.255 23 9 /23
255.255.255.0 0.0.0.255 24 8 /24
255.255.255.128 0.0.0.127 25 7 /25
255.255.255.192 0.0.0.63 26 6 /26
255.255.255.224 0.0.0.31 27 5 /27
255.255.255.240 0.0.0.15 28 4 /28
255.255.255.248 0.0.0.7 29 3 /29
255.255.255.252 0.0.0.3 30 2 /30
255.255.255.254 0.0.0.1 31 1 /31
255.255.255.255 0.0.0.0 32 0 /32
Table 3 – IPv4 Subnets
Subnet Mask The subnet mask of the CIDR block.

Wild-Card Mask Wild-Card Mask used in OSPF and packet filters
Network Bits The number of bits in the network part of the mask
Host Bits The number of bits in the host part of the mask
Prefix Short hand syntax of the CIDR block.
3
ISP/IXP Networking Workshop Lab
IPv4 ADDRESSING SCHEME – MODULES 6 to 9

Figure 1 below displays the addressing plan to be used for Modules 6 through 9. The plan itself is
explained in the notes accompanying the workshop Modules. Each subnet is a /30.
Figure 1 - Addressing Scheme for Modules 6 to 9
1
IPv4 Addresses – Modules 6 to 9

ASN Address Block ASN Address Block
10 10.10.0.0/20 30 10.30.0.0/20
20 10.20.0.0/20 40 10.40.0.0/20
Table 1 – IPv4 Address Blocks assigned to each ASN, Modules 6 to 9
Router Loopback Address Router Loopback Address

R1 10.10.15.224 R8 10.30.15.224
R2 10.10.15.225 R9 10.30.15.225
R3 10.10.15.226 R10 10.30.15.226
R4 10.20.15.224 R11 10.40.15.224
R5 10.20.15.225 R12 10.40.15.225
R6 10.20.15.226 R13 10.40.15.226
R7 10.20.15.227 R14 10.40.15.227
Table 2 – IPv4 Loopback Address assigned to each Router, Modules 6 to 9
Router “Customer” Address Router “Customer” Address

R1 10.10.0.0/26 R8 10.30.0.0/26
R2 10.10.0.64/26 R9 10.30.0.64/26
R3 10.10.0.128/26 R10 10.30.0.128/26
R4 10.20.0.0/26 R11 10.40.0.0/26
R5 10.20.0.64/26 R12 10.40.0.64/26
R6 10.20.0.128/26 R13 10.40.0.128/26
R7 10.20.0.192/26 R14 10.40.0.192/26
Table 3 – IPv4 “Customer” Address blocks assigned to each Router, Modules 6 to 9

Isp Workshop PDF

Uploaded by

Copyright:

Available Formats

Isp Workshop PDF

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Isp Workshop PDF

Uploaded by

Copyright:

Available Formats

ISP Workshop Lab

Module 1a – Basic Topology and OSPF

Prerequisites: Knowledge of Cisco router CLI, previous hands on experience.

Figure 1 – ISP Lab Basic Configuration

! Finally, documentation. Documentation is often overlooked or forgotten. It is an ongoing process

telnet 10.0.2.1 2001

telnet 10.0.2.1 2012

bash-3.2$ telnet 10.0.2.1 2001

User Access Verification

Router1 (config)# no ip domain-lookup

Router1 (config)# line con 0

Router1 (config)# no ip source-route

Router1 (config)# username isplab secret lab-PW

Router1 (config)# aaa new-model

Router1 (config)# no logging console

10.0.0.0/20 network block

10.0.0.0 10.0.14.255 10.0.15.0 10.0.15.223 10.0.15.224 10.0.15.255

Customer Address Space Network Infrastructure Loopbacks

Figure 3 – Extract from an ISP addressing plan

Router2(config)# interface serial 1/0

Q: What network mask should be used on point-to-point links?

show arp : Shows the Address resolution protocol

Q: Why do we use /32 masks for the loopback interface address?

Router1(config)#interface serial 1/0

show ip route : see if there is a route for the intended destination

Router1# trace 10.0.15.252

Q. Why do some trace paths show multiple IP addresses per hop?

Type escape sequence to abort.

1 fe0-0.router2.workshop.net (10.0.15.2) 4 msec

1. What IP Protocol does Ping and Traceroute use?

Module 1b – Basic Topology and ISIS

Prerequisites: Knowledge of Cisco router CLI, previous hands on experience.

Figure 1 – ISP Lab Basic Configuration

! Finally, documentation. Documentation is often overlooked or forgotten. It is an ongoing process

telnet 10.0.2.1 2001

telnet 10.0.2.1 2012

bash-3.2$ telnet 10.0.2.1 2001

User Access Verification

Router1 (config)# no ip domain-lookup

Router1 (config)# line con 0

Router1 (config)# no ip source-route

Router1 (config)# username isplab secret lab-PW

Router1 (config)# aaa new-model

Router1 (config)# no logging console

10.0.0.0/20 network block

10.0.0.0 10.0.14.255 10.0.15.0 10.0.15.223 10.0.15.224 10.0.15.255

Customer Address Space Network Infrastructure Loopbacks

Figure 3 – Extract from an ISP addressing plan

Router2(config)# interface serial 1/0

Q: What network mask should be used on point-to-point links?

show arp : Shows the Address resolution protocol

Q: Why do we use /32 masks for the loopback interface address?

Router1(config)# interface fastethernet 0/0

Combining the above, gives the example:

Router1(config)# router isis workshop

Router1(config)#router isis workshop

Router1(config)#router isis workshop

show ip route : see if there is a route for the intended destination

Router1# trace 10.0.15.252