How to troubleshoot EIGRP?

In the previous entry I explained how EIGRP is configured. I would like to follow on that, how to check the operation of EIGRP, and the commands required for troubleshooting.

First, to check the neighbor:

#show ip eigrp neighbors
IP-EIGRP neighbors for process 10
H   Address                 Interface   Hold Uptime   SRTT   RTO  Q  Seq Type
(sec)         (ms)       Cnt Num
8   10.254.26.1             Vl1           12 5d22h      25   200  0  76435
7   10.254.58.1             Vl1           12 5d22h      25   200  0  4134
3   10.254.0.13             Vl1           10 5d22h      23   200  0  571
1   10.254.0.12             Vl1           11 5d22h      22   200  0  54511
0   10.254.58.2             Vl1           14 5d22h      18   200  0  3354
10  10.15.255.254           Gi2/9         13 4w0d        1   200  0  764
19  10.60.255.254           Gi3/8         12 13w4d       2   200  0  4008
16  10.10.255.254           Gi3/1         11 15w4d       1   200  0  1007
13  10.24.255.254           Gi3/6         10 15w4d      21   200  0  1010
5   10.5.255.254            Gi2/3         11 16w4d       1   200  0  37489
31  10.17.255.254           Gi2/11        10 16w4d      15   200  0  54827
2   10.3.255.254            Gi2/1         14 16w6d       2   200  0  4024
47  10.4.255.254            Gi2/2         14 17w0d      16   200  0  2925

H is the sequence of neighbor discovery. Interface, is where the neighbor is located. Hold is the timer responsible to consider the neighbor dead in case Hellos ceased to receive. up-time is obvious. SRTT is the time between transmission of hello to receiving acknowledgment. RTO – in the case of multicast failure, the router will send a unicast to the neighbor. RTO is the time wait for acknowledgment for the unicast packet. Q number of queued packets. Seq Num is the sequence number of the last EIGRP packet received.

Second is to check the topology, it will indicate the cost, and how many routes are available for a unique destination network.

#show ip eigrp topology
IP-EIGRP Topology Table for AS(10)/ID(10.250.0.10)

Codes: P – Passive, A – Active, U – Update, Q – Query, R – Reply,
r – reply Status, s – sia Status

P 10.18.24.0/22, 2 successors, FD is 28928
via 10.254.0.11 (28928/28672), Vlan1
via 10.5.255.254 (28928/28672), GigabitEthernet2/3
P 10.9.255.252/31, 1 successors, FD is 3072
via 10.254.0.11 (3072/2816), Vlan1
P 0.0.0.0/0, 1 successors, FD is 281600
via Rstatic (281600/0)
P 10.10.0.0/20, 1 successors, FD is 3072
via 10.10.255.254 (3072/2816), GigabitEthernet3/1
P 10.11.255.254/31, 1 successors, FD is 2816
via Connected, GigabitEthernet3/3
P 10.10.255.254/31, 1 successors, FD is 2816
via Connected, GigabitEthernet3/1
P 10.11.0.0/20, 1 successors, FD is 3072
via 10.11.255.254 (3072/2816), GigabitEthernet3/3
P 10.8.255.252/31, 2 successors, FD is 3072
via 10.8.255.254 (3072/2816), GigabitEthernet2/6
via 10.254.0.11 (3072/2816), Vlan1
P 10.24.16.0/20, 1 successors, FD is 3072
via 10.24.255.254 (3072/2816), GigabitEthernet3/6
P 10.9.255.254/31, 1 successors, FD is 2816
via Connected, GigabitEthernet2/7
P 10.11.255.252/31, 2 successors, FD is 3072
via 10.11.255.254 (3072/2816), GigabitEthernet3/3
via 10.254.0.11 (3072/2816), Vlan 1

P means passive, this indicate a stable route. Active is a lost route which the protocol will try to find alternative path for it through quires. The first entry has FD of 28928. (this is Feasible Successor Plus the cost to that neighbor). This is considered the total cost. The second number 28672 is the Feasible Successor (the cost advertise by the neighbor to that network). For any path to become eligible as a successor, the FD should be equal or greater than FS. This ensures a loop free routing.  via are the neighbors, and which interface they connected through.

Lastly,the command that will give the summary of all routing protocols running in the router/switch.

#show ip protocols
Routing Protocol is “eigrp 10″
Outgoing update filter list for all interfaces is not set
Incoming update filter list for all interfaces is not set
Default networks flagged in outgoing updates
Default networks accepted from incoming updates
EIGRP metric weight K1=1, K2=0, K3=1, K4=0, K5=0
EIGRP maximum hopcount 100
EIGRP maximum metric variance 1
Default redistribution metric is 10000 100 255 1 1500
Redistributing: static, eigrp 10
Automatic network summarization is not in effect
Maximum path: 4
Routing for Networks:
10.0.0.0
172.22.0.0/30

Routing Information Sources:
Gateway         Distance      Last Update
10.11.255.254         90      05:15:10
10.10.255.254         90      05:15:10
10.9.255.254          90      05:15:10

Here, we can see which networks are advertised, any access-list filtering for routes, the K values, redistribution, and in other cases any passive-interfaces. The last column showing the neighbors connected.

How to congifure EIGRP?

EIGRP is a Cisco proprietary protocol. It is one of the most widely used within enterprises that use Cisco switches/ routers. It stands for Enhanced Interior Gateway Routing Protocol. The reason for such widely deployment is the ease of use compared to OSPF, and the effectiveness of the protocol.

Before EIGRP can update and send topology information, building relationships between EIGRP enabled routers is the first process. For two routers to become neighbors, the following conditions should be met.

• Autonomous System number should be same.
• The K values should be same. (they are same if left on default).
• The routers should be in the same subnet.

Here is the syntax for configuring EIGRP

Router> enable
Router# config terminal
Router(config)# router eigrp 1
Router(config-router)# network 10.2.9.0 ?
  A.B.C.D  EIGRP wild card bits
  <cr>
Router(config-router)# network 10.2.9.00 0.0.0.255
Router(config-router)# no auto-summary
Router(config-router)# end

EIGRP by default uses Auto-Summary for routes within certain condition. The router will summarize when it is residing between two different Networks (not subnets). For example between 192.168.1.0 and 192.168.2.0 since they are two different class C networks.

The calculation of Metrics (cost) is complicated slightly, but when using default K values =1. The equation is straight forward. Here is the Equation

Cost = [(K1 X Bandwidth + ((K2 X bandwidth)/(256-load)) +K3 X delay) X K5/(K4+reliability)] X 256

With default values K1, K3=1, and K2, K4, K5 = 0. the equation becomes.

Cost = (Bandwidth + delay) X 256

Where bandwidth is the minimum in the link, and delay is cumulative.

What is the difference between M1 and F1 Cisco Nexus Line cards?

Cisco Nexus series switches brought a new technology to the data center. The whole designed is changed from the Catalyst 6500 series. Nexus is no longer dependent on SUP’s backplane, it is more like a midplane architecture. Let me elaborate a little on this, what that statement means that currently if there is any limitation of speed, then it is posed by the Line Card. Then how the Line cards communicate with each other, they do with Fabric Modules. Read for further details into basic architecture difference between Catalyst 6500 vs Nexus 7000

Nexus Line card modules fall into two major categories. M1, and F1. There is another variation to the M1 which is M1-XL. Brad Hedlund wrote a good article that can be referenced for reading, titled “Cisco Nexus 7000 connectivity solutions for Cisco UCS”

M1, M1-XL

M1 Series were the introductory line cards that were offered by Cisco for Nexus. They come with a fabric of 80GB. These cards have 10Gig links making them ideal for Distribution layer. Lets put down the specifications or performance Metrics from the data sheets. These cards provide the Layer 2 and Layer 3 connectivity! You can always multiply these numbers with the maximum line cards possible to install into a chassis to get the marketing figures.
1- Delivery at 60 Million Packets per second (Mpps) for layer 2,3 IPv4.
2- Delivery at 30 Mpps IPv6 unicast.
3- Delivery of Access Control List (ACL) to 64k entries per module. The entries include address of Layer 2,3,4 and Cisco’s Metadata fields- security group tags (SGTs)
4- in 32 Port line card, each 4 ports share 10GB of Fabric. They can run either 1 port 10GIG disable 2,3, and 4 OR all 4 in shared mode.
5- Memory 1GB DRAM
6- Network management: Cisco DCNM 4.0
7- Mac addresses table size of 128k entry
8- FIB table of 128k entry
9- Netflow supports 512k Entry in both Ingres and Egress
10- 16384 bridge domains and 4096 vlan per Virtual Device Context (VDC)
11- Policers of 16k entry

M1-XL Series offers the flexibility or the performance to be internet-facing deployment with wider transceivers module support. What it basically offers the possibility of larger FIB. This can be seen from the following:
* up to 1M IPv4 routes (depending on prefix distribution)
* up to 350k IPv6 routes (depending on prefix distribution)

This was not possible in the M1 Line Cards. M1-XL does provide extra ACL entries support compared to M1, which increased DRAM
1- Memory 2GB DRAM
2- Delivery of Access Control List (ACL) to 128k entries per module.
3- Network management: Cisco DCNM 5.1

F1
F1 Series Line Cards were introduced after the M1. They provide a slight cheaper and more port density with ONLY layer 2 forwarding. This makes an ideal Line card for Access layer. What happens if layer three processing is required? The Line card will forward that traffic to M1, M1-XL cards for processing. These cards have Fabric of 230 GB.

1- 480 Mpps layer two forwarding
2- Delivery of Access Control List (ACL) to 32k entries per module. The entries include address of Layer 2,3,4 and Cisoc’s Metadata fields- security group tags (SGTs)
3- in 32 Port line card with 230GB of fabric.
4- Memory 1GB DRAM
5- Network managment: Cisco DCNM 5.1
6- Mac addresses table size of 16k entry per forwarding engine.

The forwarding engine is something new. Every two ports are connected by a switch on chip. (SoC), these SoC are the forwarding engine. So each SoC supports 16k. What this implies (How marketing figured came) that for 32 port, we have 16 SoC. With careful planning, if we use one VLAN per SoC we get total of 256k of Mac address support. But if we span one vlan among all SoC then we are bounded by max limit of 16k MAC entry.

These cards have the Cisco FiberPath Technology. From the data sheet

The benefits of Cisco FabricPath include:

• Operational simplicity: Cisco FabricPath embeds an autodiscovery mechanism that does not require any additional platform configuration. By offering Layer 2 connectivity, this “VLAN anywhere” characteristic simplifies provisioning and offers workload flexibility across the network.

• High resiliency and performance: Since Cisco FabricPath is a Layer 2 routed protocol, it offers stability, scalability, and optimized resiliency along with network failure containment.

• Massively scalable fabric: By building a forwarding model on 16-way ECMP, Cisco FabricPath helps prevent bandwidth bottlenecks and allows capacity to be added dynamically, without network disruption.

They also have the ability to connect FCoE. these features include
1-Virtual Sans (VSANs)
2-Inter-VSAN Routing
3-PortChannels (UP to 16 links)
4- Storage VDC.

This sums up what I found. I would include or add more things later as I learn or gather them.

Nexus 7000 Vs Catalyst 6500 (Backplan capacity)

Cisco has introduced Nexus. Nexus are the new line of data center switches. They come in the variables of 7000, 5000, 2000, and lastly 1000.

Nexus 7000 with their functionality sit at distribution layer, while Nexus 5000 come in the access layer. Nexus 2000 are nothing but extension to the 5k switches. In easier analogy, they work as line cards in 6500 chassis.

In this article, I would say why or when to use Nexus 7000 in the enterprise core layer. By purpose, the Nexus was designed for data center. But with the increased requirements of backbone network, and network growth the current top of the line 6500 switches comes short.

The backplane/fabric of 6500 switch is part of Supervisor Engine, in the case of SUP720 a 40GB per line module is the maximum bandwidth. What happens if you connect 8 ports with speed of 10G line card is that we are oversubscribing 1:2 ratio. This will be doubled with 16 ports of 10 GB line card to 1:4. The issue when multiple (30 or more) distribution switches are linked with 10G then the chassis with 9 slots becomes not enough. That is in the case of connecting without oversubscription.

The Nexus switches have different architecture. They line cards don’t depend on Supervisor Engine’s fabric for traffic processing exclusively. Each module will have its own fabric. This fabric rather connected by a fabric module that can be upgraded by itself. Each fabric supports 46GB per slot. Nexus 7000 with 10 slots support 5 fabrics, that is equal of 230GB per module slot. This is 5.75 times more than the original 6500 fabric. Still, a 32 10GIG port have a 80GP backplan, thus these 32 ports line card are oversubscripted at the rate of 1:4. while the 8 port Line cards are Non-oversubscripting.

Honestly, I still can’t figure out the reason for their fabric modules, since their line card modules are all having a limit 80GP fabric. So with 8 line cards, we have requirement of 640 GP, and we still lacking the support from the fabric module for these line card.

Solved: Distribution Switch Acting Weird

In my previous entry of “Troubleshoot: Distribution Switch Acting Weird” I have mentioned a strange problem that was happening in the network. So, I have gone through a lot of trouble to find out what was going on. We checked Spanning-Tree with full details drawing all ports, roots, etc. We were sure that something stopping traffic of our Server Farm Vlan from propagating into the Routing Vlan. As mentioned in the earlier post, we knew the general idea but we were looking at the wrong place.

We concentrated on the distribution switch since the traffic was stopping over there. The problem was at the core switch! We never applied VTP pruning on the interface level, so we never really thought it could be an issue. What we found that the command vtp pruning was enabled. Checking the operation of vtp pruning requires the following command to be typed “show vtp status | in pruning”. Since, it was “enabled”. The following scenario happened.

The core switches were the servers, while wireless devices were client. Between those two devices were the Server Farm Distribution switch which was operating in Transparent mode. This meant that the distribution switch will pass all VTP packets, just wont process them, in other words effective communication was taking place between client and server in the VTP domain. What happened is that when we were shutting down the Vlan Interface in the VTP client. It sent a message to server that the Server Farm Vlan could be pruned. This caused the communication to stop in the link between the server farm distribution switch and core switch for that vlan. The scenario can be seen from the figure below.

This is why, we should never use “vtp pruning”. Just prun the vlans we want from the trunk links manually. It gives the proper control and predicted behavior of network.

In general, it is best to do everything manually in networks. Never use the “auto” let it be speed, duplex negotiation, trunk, etherchannel, routing summary, or anything. The most predictable network behavior, the easier to troubleshoot.

Solving the stackwise problem in Cisco Catalyst 3750.

In our previous entry, we had the stack issue. The stack connection dropped between two switches. This caused a major concern. Going through the Cisco documentation about stack troubleshoot. We did change the stack cable, but it did not solve the problem. Since the switch was in our data center, any troubleshoot efforts might become an issue in case of downtime.

It was decided that to resolve this issue, the best course of action would be to remove the stack. We took a downtime, and we have removed the stack wise connection. The moment we took the stack cables out. The isolated switch became stand alone switch with the current running configuration. Since, we figured that the switch should be removed from the network so that the real (master) switch and the one just removed (slave) no to cause issues in the network by having similar configurations.

CCNA vs JNCIA-Junos

Lets look at CCNA and JNCIA-Junos in terms of knowledge that you gain as an entry level engineer into the realm of TCP/IP. I can garuntee that the rabbit hole goes deep, too deep to be accurate (that is still subjective term. I fail to see how subjective term can be accurate, lol).

JNCIA-Junos is not the best certificate for entry level engineers from my prospective. Why? here are the reasons although they are few. First, it assumes you are familiar with Cisco devices and Cisco CLI. Second, It is exam to for Junos and Juniper. It assumes you already learned or familiar with TCP/IP. I think any entry level certification should cover the topics of TCP/IP. The foundation of the Network, the OSI layers, Subnetting, IP addresses.

I think the biggest concern i have with entry level certificates is that basic of TCP/IP should be covered, and the exam should focus on it. It should not focus on How to run Junos. If i hire a network engineer who knows how to use Junos, but he has no clue of Networking, what use i have of him? Yes, the certification does require you to have the basic knowledge. But from what i gathered in the studying guides, they are not covered in the exam.

Once i pass the exam, I’ll update this entry, or make a new one. Just to be sure that I’m representing accurate facts.

Running Juniper with GNS3

It was rather a very intense Google journey to find out all the details on how to emulate Junos. Since i intend to learn juniper, i needed a platform to work on. after two days of research and work, i managed to a results.

There are various things required to make things work. I would list them down here so they are easy to find. VMware Player, GNS3, Cisco router IOS, and VMware Olive (Google is your friend). Once you have all these, you are ready to start!

Running the VMware machine will be an easy task, but connecting the VMware Olive with Cisco in GNS3 is the one requiring some work. But with my guide, it should be as easy as 1,2 and 3.

After installing VMware Player, check for adapter settings in windows.

By default VMware player will install to VMware virtual Ethernet adapters, i’m not sure what are their numbers. but for my case, they were vmnet1 and vmnet8. These are significant to know how to connect VMware machine to Cisco router in GNS3.

Open the .vmx file in notepad. Here we can edit the fields in order to make VMware Olive machine use the virtual Ethernet adapters in windows.

The Olive VMware has three network interfaces, two are bridged and first one is in “costum” we change the adapter to the one to fit the Ethernet adapter in our network devices (from the first figure). I already highlighted it in red. Ethernet 0 will be reflected as interface em0 in Junos. ethernet1, and ethernet2 will be bridge on the virtual Interface, so you can connect to other Olive Machines to ethernet1, ethernet2 (em1, em2). my assumption is, if you want to connect say Olive1 and Olive 3 using em2 then you change ethernet2 in vmx file of both olive 1 and 3 to a bridge mode with a common adapter.

That is the topology i created for the simulation. basic two Juniper routers connected to a Cisco router. and the two Juniper routers are connected as well (virtually in VMware). It was tested, and pings were working.

In GNS3, add the VMware as a Cloud, of course the cloud will be not associated with the VMware Olive till you select the adapter that you set up in the vmx file. In a screen shot, you will see that i have Chosen Vmnet1 for this particular Olive.

Last step, would be to do the appropriate configurations in the Olives, and Cisco Router, here is the screenshot of the sample configuration i used to ping.

Don’t forget to add the following before you can commit any configuration into Juniper Router

set system root-authentication plain-text-password

Cisco’s Configuration as simple:

interface FastEthernet0/0
ip address 192.168.1.2 255.255.255.252
duplex auto
speed auto
!
interface FastEthernet1/0
ip address 192.168.1.10 255.255.255.252
duplex auto

Configuring IP DHCP Snooping.

The other day, a smart user (i consider him evil) attached a ADSL modem to the network port. What he didn’t realize that his device was programmed to work as DHCP server. As a result, the whole vlan started getting the wrong IP address, and connectivity was lost. After investigation, we realized a rouge DHCP server, tracked it, and finally blocked the port manually.

Of course, the best was to enable DHCP snooping in the switch, and not worry about anyone attaching any funny thing to the network. To get an idea with DHCP snooping is, please read Cisco’s document. I will quote the basic idea.

DHCP snooping is a DHCP security feature that provides security by filtering untrusted DHCP messages and by building and maintaining a DHCP snooping binding table. An untrusted message is a message that is received from outside the network or firewall and that can cause traffic attacks within your network.

The DHCP snooping binding table contains the MAC address, IP address, lease time, binding type, VLAN number, and interface information that corresponds to the local untrusted interfaces of a switch; it does not contain information regarding hosts interconnected with a trusted interface. An untrusted interface is an interface that is configured to receive messages from outside the network or firewall. A trusted interface is an interface that is configured to receive only messages from within the network.

DHCP snooping acts like a firewall between untrusted hosts and DHCP servers. It also gives you a way to differentiate between untrusted interfaces connected to the end-user and trusted interfaces connected to the DHCP server or another switch.

Just following the normal command line will surely hinder the network, in fact it blocked all DHCP requests. the  information option should be disabled. That what i realized, and i found in many other forums.

Second issue i encounter, that the DHCP snooping will not be enabled on any switch with VTP mode other than transparent. What i found that it was disabled in other modes for security reasons. If a VTP domain is compromised, then the attacker can remove vlans or add vlans, thus compromising the integrity of vlans and this will cause an issue with IP DHCP snooping. Since DHCP snooping will start effecting other vlans, etc.

The following should be used to configure DHCP Snooping

ip dhcp snooping

ip dhcp snooping vlan vlan-number

no ip dhcp snooping information option

int gig 0/1 \\ this is the uplink

ip dhcp snooping trust

here is the output for various commands:

L2CS-B851-01#show ip dhcp snooping
Switch DHCP snooping is enabled
DHCP snooping is configured on following VLANs:
851
DHCP snooping is operational on following VLANs:
851
DHCP snooping is configured on the following L3 Interfaces:

Insertion of option 82 is disabled
circuit-id default format: vlan-mod-port
remote-id: 2893.fef7.f280 (MAC)
Option 82 on untrusted port is not allowed
Verification of hwaddr field is enabled
Verification of giaddr field is enabled
DHCP snooping trust/rate is configured on the following Interfaces:

Interface                  Trusted    Allow option    Rate limit (pps)
———————–    ——-    ————    —————-
GigabitEthernet0/1         yes        yes             unlimited
Custom circuit-ids:
L2CS-B851-01#show ip dhcp snooping statistics
Packets Forwarded                                     = 415328
Packets Dropped                                       = 7601
Packets Dropped From untrusted ports                  = 0

L2CS-B851-01#show ip dhcp snooping binding
MacAddress          IpAddress        Lease(sec)  Type           VLAN  Interface
——————  —————  ———-  ————-  —-  ——————–
00:1A:80:EE:79:8A   10.80.51.38      420810      dhcp-snooping   851   FastEthernet0/5
E0:CB:4E:06:FC:3E   10.80.51.99      372172      dhcp-snooping   851   FastEthernet0/45
00:21:9B:E2:87:C6   10.80.51.80      421750      dhcp-snooping   851   FastEthernet0/4
00:26:6C:78:00:F3   10.80.51.174     425902      dhcp-snooping   851   FastEthernet0/36
00:1B:38:AF:81:DD   10.80.51.186     423185      dhcp-snooping   851   FastEthernet0/43
Total number of bindings: 5

Applying IP dhcp snooping in the access switch is enough, as long as the uplinks are trusted. It is not required to apply these configurations to distribution switch (assuming that no one has access to them. )

Over all, this should have been used ages ago. I am very glad that i implemented this in our network.

Juniper Certification Track

I’ve decided that since I shall start learning Juniper, widen the horizon. First anyone starting Juniper will have to get the associate certificate, JNCIA-JUNOS. The topics that are covered can be seen from the website exam topics.

They include:

Junos OS Fundamentals

• Explain the Junos OS architecture
• Describe the functions of the control and forwarding planes

User Interface Options

• Describe the functions of the CLI modes
• List options for getting help and filtering output
• Explain options for moving around in the CLI
• Explain the Junos batch configuration model
• Modify and manage configuration files
• Describe the J-Web user interface

Junos Configuration Basics

• Describe initial configuration steps and system maintenance
• Save and restore a rescue configuration
• Describe network interfaces and their associated properties
• Configure various types of interfaces
• List and describe user authentication methods
• Configure user accounts with default and custom login classes
• Describe, configure and monitor syslog and tracing
• Describe, configure and monitor configuration tasks, such as NTP, SNMP, and configuration archival

Operational Monitoring and Maintenance

• Explain how to view relevant interface statistics and errors in the CLI
• List ways of gathering state and health information for a Junos device
• Explain the purpose and syntax for various network utilities such as ping, traceroute, telnet, ssh etc.
• Identify the procedure for installing or upgrading Junos
• Explain how to reset a lost root password

Routing Fundamentals

• Define route preference and list some default route preferences
• Explain how to view the contents of the routing tables
• Explain how the routing and forwarding tables differ
• Describe and configure static routing
• List the advantages and applications of a dynamic routing protocol

Routing Policy and Firewall Filters

• Define a term
• Explain when import and export policies are evaluated in relation to the learning and advertising of prefixes
• List several match criteria and actions for firewall filters and routing policy
• Explain some common match criteria and
• Identify the results of a route or packet with a given filter or policy
• Configure a routing policy and firewall filters

Class of Service

• Describe class of service

Networking Fundamentals

• Convert decimal to binary and binary to decimal
• Define the term broadcast domain and the purpose of a router
• Describe Ethernet operation
• Describe the purpose of the subnet mask
• Compare and contrast connection oriented and connnectionless protocols
• Explain how to segment large networks into smaller ones
• Identify the longest match from a routing table for a given destination

For the rest of track, please refer to the official website.