Introducing Cisco Catalyst 6500 Series Supervisor Engine 2T

Cisco very recently introduced the new SUP engine for the much trusted Catalyst 6500, the Supervisor Engine 2T (S2T). It is much welcomed sup line to already strong and dominant SUP720. You would expect that the new SUP should be better. That is how technology goes forward. I think the most important feature is the increase of the backplane capacity by double. That is 80Gbps.

The S2T gets its name from the fact that the It has 26 Fabric Channels boosting 80Gbps. So 80Gx26 = 2.08Tera. That is why it is called S(up)2T(era) The S2T has 26 Fabric Channel. Each Fabric channel is unidirectional with speed of 80Gbps. Thus, it can support 13 line cards with two Fabric Channels for bidirectional data transfer. 13 line cards? welcome the biggest Catalyst 6500 Chassis 6513-E

The main components of S2T are: Policy Feature Card 4 (PFC4), and Multilayer Switch Feature Card 5 (MSFC5). Lets have a look at what Cisco has to say about them.

Policy Feature Card 4

Supervisor Engine 2T features the integrated Policy Feature Card 4 (PFC4), which improves performance and scalability and provides new and enhanced hardware features. The PFC4 is equipped with a high-performance ASIC complex that enables hardware acceleration for existing and new software features. The PFC4 supports Layer 2 and Layer 3 forwarding, QoS, Netflow and Access Control List (ACLs) and multicast packet replication and processes security policies such as access control lists (ACLs) operations all simultaneously enabled with no performance impact. The PFC4 supports all of these operations for both IPv4 and IPv6. PFC4 also provides enhanced performance and scalability and supports many new innovations such as native VPLS, flexible NetFlow, egress NetFlow, Cisco TrustSec, distributed policers, control plane policing, and comprehensive IPv6 features.

Multilayer Switch Feature Card 5

Supervisor Engine 2T features the Multilayer Switch Feature Card 5 (MSFC5), providing high-performance, multilayer switching and routing intelligence. Equipped with a high-performance processor, the MSFC5 runs both Layer 2 protocols and Layer 3 protocols on the dual-core CPU complex. These include routing protocol support, Layer 2 protocols (for example, Spanning Tree Protocol and VLAN Trunking Protocol), and security services. The MSFC5 builds the Cisco Express Forwarding information base (FIB) table in software and then downloads this table to the hardware application-specific-integrated circuits (ASICs) on the PFC4 and Distributed Forwarding Card 4 (DFC4), if present on a module, which make the forwarding decisions for IP unicast and multicast traffic.

The main Features are:

• Platform scalability: Delivering up to 80 Gbps per slot of switching capacity on E-Series chassis; 2-Terabit aggregate bandwidth capacity using the 6513-E chassis, scaling to 4-Terabit capacity with VSS. Support for up to 1056 ports of 1Gbps and 352 ports of 10Gbps systems deployed with VSS. Providing 1Gbps/10Gbps and 40Gbps interface support to address future customer bandwidth growth requirements.

• Security: Support for Cisco TrustSec, CTS, providing MacSec encryption and Role-Based ACL. Providing control plane policing to address denial of service attacks.

• Virtualization: Native support for VPLS, as well as enhancements such as VPN-aware NAT, VPN statistics, and VPN netflow as important features needed for deployment of network virtualization.

• Netflow application monitoring: Supervisor Engine 2T supports enhanced application monitoring such as Flexible and Sampled Netflow for intelligent and scalable application monitoring.

It is a nice welcome for the Catalyst 6500. S2T will surely be used in Data Center and at Core layers. The nonblocking 8 ports 10G links would be a real advantage to support the current infrastructure. while, it is yet to be seen how the 40G ports will integrate into the data center.

Cisco Catalyst Sup720 V S2T

Cisco has recently introduced the S2T. We ran a comparison from the prospective of backplane capacity between the Nexus 7000k and Catalyst 6500 previously. We would like to run another comparison between the latest S2T and SUP720-10G-3C. The Supervisor shown below has 2 X2 10G ports, 3 SFP 1G ports, and one 1G management port, not to mention the console port. Pic courtesy of Cisco.

It would be best made in a table format. so that it can be seen easily without confusing or rechecking the paragraphs. The table was made after going through the data sheets of both SUP720 and S2T.

* The data sheet did not mention it, further research is required to find out.

** Requires DFC3

As it can be seen from the table above. That when it comes to numbers, then the S2T boost as much as 50% increase in terms of performance above the strongest SUP720, which is SUP720-10G-3CXL.

GRE Tunnel ARP entry never times out! – part 3

For people who were following on this issue, i would like to post some updates. But before that, you can read what is the problem by going to GRE Tunnel ARP entry never times out! and GRE Tunnel ARP entry never times out! – part 2.

So after various troubleshooting. It was concluded that these log messages were nothing but cosmetic. Meaning although they are showing, they don’t change the behavior of the operation. It was stated that the bug causing this is CSCsa83049.

duplicate ip addr message seen on tunnel due to ARP entry not aged out

A mGRE tunnel on a sup720 may report ARP entries that never age out. This can happen when a mobile node sends a unicast ARP directed to the default gateway, a.k.a sup720. These entries are not used by the switch to make a forwarding decision therefore can be ignored. Workaround: Clear the ARP table using the “clear arp” command. Solution: – A solution to prevent the tunnel from either receiving or learning the ARP entries is been investigated.

We have thought of using Automated “clear arp” using scripts. but, the issue is. “clear arp” will not work for cause we have bug CSCtf16300.

clear arp-cache is not working correctly

Symptom: “clear arp-cache” command is not removing the stale entries from arp table. Condition: -Use cat6500 on 12.2(18)SXF16 or later. Workaround: -Use “clear ip arp x.x.x.x” command.

The solution is to change the IOS. But we do required the (SFX) IOS version to run the WLSM.

Anywho, Cisco said that they are investigating and planning on resolving bug CSCsa83049 By end of July, start of Aug. We shall wait till then.

Review – SolarWinds Engineer’s Toolset – Part3

IP Address Management has some interesting tools that can be used in the SolarWinds Engineer’s Toolset. One of the tools that make the life of network engineer easy, (which i think should not have been created!!. let the engineers do use their brains for this) is the advance subnet calculator. As it says, it does many other things. From finding subnets, defining the subnet, and all the expected from IP addressing tool.

DNS analyzer is such interesting tool. it will show how a domain name is being used, which servers, their real names, ip address, etc. For example i used www.kfpum.edu.sa since it is hosted in single site, the analysis was straight simple. But doing a www.Yahoo.com is just over killing it, as shown in the second image below. Just looking at it makes my head starts spinning. Although www.google.com provided much simpler output.

DNS resolver will give you the IP address, and the who’s information you would expect from various websites. So, it is really a good tool to track the traffic, and who is using what. The second tool will take a range of IP addresses, and do the who’s analysis.

Lastly, one of the most important tools, that i’ve been using so often is the ConfigViewer. (once you run it, you can upload, download and find the difference). I’ve been using it to download, and mostly compare. I have used it for upload as well after i finished edited the config file. So, it really is very good tool. Eases the operation of working with configuration files.

As seem above, screenshots. First to download the config file. Second image to view it, write on it. (go find what i wrote!!!) and Lastly, to find the difference.

This was the third part of this series. Hopefully soon i will have more features covered.

Review – SolarWinds Engineer’s Toolset – Part2

Once the Toolset is installed, you would want to run the SolarWinds Workspace Studio. It is the main application window that you use to monitor the devices. It has limited management capabilities since it was not designed for such thing.

The Studio enables to use few features that actually matter, most of the features mentioned in the first entry are stand alone applications. The studio let you monitor CPU, Memory, and Links. It further enables you to see the Routes, port maps, and neighbors.

The CPU and Link utilization can be configured thoroughly with proper alarms to be shown in case of exceeding the limits. The image below shows the CPU and Link utilization example

The Utilization can be seen in percentage, Gauge or in Charts. Each has its own use, from very abstract values, to detailed (in the charts) value.

The neighbors View is such a good tool to ease the drawing! I mean, it actually traces the network to the number of wanted hops. I have chosen only 1 hop in the image below. It shows the device, and which devices it is connected to with their IP addresses, and interface information. It really extracts the CDP information, which is very rich.

This feature will surely help any new network engineer, or even a consultant to have a look and the feel of the network when troubleshooting or understanding the network.

The above image shows the Route Viewer feature. The user choose the router, and the tool will find the routing table. What i really wanted to see is the incorporation of both map view and route view. So you can see the traceroute but instead of the normal output, you see it graphically.

The last feature in this entry is the Port Mapper. This tool will find all the ports, the macs associated with it, and many other information such as: IP, DNS name, Link speed, Link queue, and the other information that you can see in cisco devices when using the command “show interface type num/num ”

This concludes part 2 of the review. Tune in for part 3. Lastly, don’t mind my mspaint skills. i had to hide the IP addresses.

Review – SolarWinds Engineer’s Toolset – Part1

SolarWinds provide various solutions for Network management, network monitoring, storage, VMware, and servers monitoring and management. We have purchased the license for Engineer’s Toolset. It certainly made my life easier when it comes to managing Cisco devices, and monitoring some other critical Cisco devices.

In this entry, I would highlight all the functions that can be done using the Engineer’s Toolset, later in other entries we will see some of them, and how they work.

Once the Toolset is installed you can operate the following.

1. SolarWinds Engineer’s Toolset
   1. WorkSpace Studio
   2. Classic Tools
      1. Cisco Tools
         1. Cisco Router Password Decryption
         2. Compare Running vs Startup Configs
         3. Config Downloader
         4. Config Transfer
         5. Config Upload
         6. Config Viewer
         7. CPU Gauge
         8. IP Network Browser
         9. Netflow configurator
         10. Netflow Realtime
         11. Proxy Ping
         12. Router CPU load
         13. TFTP server
      2. IP address Management
         1. Advanced Subnet Calculator
         2. DHCP Scope Monitor
         3. DNS & Who Is resolver
         4. DNS Analyzer
         5. DNS Audit
         6. IP address Management
         7. IP Network Browser
         8. Ping Sweep
      3. Network Discovery
         1. DNS Audit
         2. IP Address Management
         3. Mac Address Discovery
         4. IP Network Browser
         5. Network Sonar
         6. Ping Sweep
         7. Ping
         8. Port Scanner
         9. SNMP Sweep
         10. Subnet List
         11. Switch Port Mapper
      4. Network Monitoring
         1. Advance CPU load
         2. Bandwidth Gauges
         3. Network Monitor
         4. Network Performance Monitor
         5. Real time interface Monitor
         6. Router CPU Load
         7. SNMP Real time Graph
         8. Syslog Server
         9. Watch it!
      5. Ping & diagnostic
         1. DNS Analyzer
         2. Enhanced Ping
         3. Ping Sweep
         4. Ping
         5. Proxy Ping
         6. Send Page
         7. Spam Blacklist
         8. TraceRoute
         9. Wake-On-LAN
         10. WAN Killer
      6. Security
         1. Cisco Router Password Decryption
         2. Edit Dictionaries
         3. Port Scanner
         4. Remote TCP Session Reset
         5. SNMP Brute Force Attack
         6. SNMP Dictionary Attack
         7. Spam Blacklist
      7. SNMP Tools
         1. MIP Viewer
         2. MIP Walk
         3. SNMP MIP Browser
         4. SNMP Trap Editor
         5. SNMP Trap Receiver
         6. Update System MIB

Although the list have some duplicated items. It is cause these tools can be categorized under more than one name.

How to manipulate BGP Routes

Border Gateway Protocol (BGP) is the back bone protocol that connects the internet. It falls under the External Gateway Protocols (EGP), interestingly it is the only routing protocols used in the external networks.

BGP is a robust protocol that can handle 100k routes, which are increasing. That as for IPv4 addresses, IPv6 addresses will have even more routes!

Manipulation of routes within the BGP cloud is one of the most challenging tasks a network engineer will be given. To manipulate the routes various Path Attributes (PAs) can be changed. They are done mainly by using:

• neighbor distribute-list (standard ACL / extended ACL)
• neighbor prefix-list
• neighbor filter-list
• neighbor route-map

Review “Manage Enginer IT360″

We had to test Manage Engine IT360 for the use in our enterpirse network. IT360 is a move toward IT service management (ITSM) which is a part or ITIL.

In brief words IT360 is

IT360 is an Integrated IT management solution by ManageEngine designed to Monitor and Manage IT Infrastructure for Medium and Large Enterprise. ManageEngine IT360 adds a business context to monitoring IT Resources, there by helping the various stakeholders understand the impact of downtimes on the business.

This review is actually old. But im making a formal point where all the links to the blog entries can be found from one place. The review consisted of three parts as following:

1. Manage Engine IT360 Review – Part 1
2. Manage Engine IT360 Review – Part 2
3. Manage Engine IT360 Review – Part 3

GRE Tunnel ARP entry never times out! – part 2

I have been trying to figure out why the APR entries don’t timeout as they should do naturally from the tunnels. As it seems, the natural time of 4hr is not being applied here. For some uknown reason yet. We have opened up a TAC case with Cisco. Roger Nobel (CCIE WIreless#23679) is really helpful and efficient.

So, in our troubleshooting so far, we tested how the MN is associated with AP, is the association with AP remains after MN is disconnected, does the SUP720 maintains a record for this MN. what we found so far is the following.

After the MN is disconnected from AP. The AP will clear the association in less than 1 min. and in another 5 mins this association will be cleared from the SUP720 as well. it can be seen from the following commands

WLAN-CORE-1#show mobility mn ip 10.13.115.150
MN Mac Address  MN IP Address  AP IP Address  Wireless Network-ID  Flags
————–  ————-  ————-  ——————-  —–
b407.f9ea.a941  10.13.115.150  10.254.14.172  8                      F

Flags: D=Dynamic network ID, F=Fresh, G=Grace Period

WLAN-CORE-1#show mobility mn ip 10.13.115.150
MN with ip 10.13.115.150 is not found in database

Now naturally, the ARP entry should stay for 4 hrs (default Cisco). but in our case it says forever! we have ARP entries as old as 10 days without adding any configurations. The command does not even show any timer for timeout as it shows in other physical interfaces.

WLAN-CORE-1#show int gig 5/1
GigabitEthernet5/1 is up, line protocol is up (connected)
Hardware is C6k 1000Mb 802.3, address is 0011.5cb4.c2a4 (bia 0011.5cb4.c2a4)
MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, loopback not set
Keepalive set (10 sec)
Full-duplex, 1000Mb/s, media type is T
input flow-control is off, output flow-control is off
Clock mode is auto
ARP type: ARPA, ARP Timeout 04:00:00

here is how the tunnel interface looks like

WLAN-CORE-1#show int tunnel 1
Tunnel1 is up, line protocol is up
Hardware is Tunnel
Description:
Internet address is X.X.X.253/20
MTU 1514 bytes, BW 1000000 Kbit, DLY 500000 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation TUNNEL, loopback not set
Keepalive not set
Tunnel source X.X.X.1 (Loopback1), fastswitch TTL 255
Tunnel protocol/transport multi-GRE/IP, key disabled, sequencing disabled
Checksumming of packets disabled, fast tunneling enabled
Last input 00:00:00, output 00:00:01, output hang never
Last clearing of “show interface” counters never
Input queue: 0/75/125/37 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/0 (size/max)
5 minute input rate 318000 bits/sec, 226 packets/sec
5 minute output rate 3458000 bits/sec, 355 packets/sec
L2 Switched: ucast: 0 pkt, 0 bytes – mcast: 0 pkt, 0 bytes
L3 in Switched: ucast: 0 pkt, 0 bytes – mcast: 0 pkt, 0 bytes mcast
L3 out Switched: ucast: 0 pkt, 0 bytes mcast: 2989660 pkt, 922842977 bytes
249194378 packets input, 54362827775 bytes, 0 no buffer
Received 1308901 broadcasts (71327 IP multicasts)
0 runts, 0 giants, 18 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
327413145 packets output, 259801658657 bytes, 0 underruns
0 output errors, 0 collisions, 0 interface resets
0 output buffer failures, 0 output buffers swapped out

I would wait for Mr. Roger to come back and see what possible thing is causing this.

GRE Tunnel ARP entry never times out! – part 1

I would like to clear the ARP entries automatically from the GRE tunnel made by the WLSM to the AP. here are the configurations of the tunnels.

interface Loopback1
description tunnel_source
ip address 10.x.x.1 255.255.255.255

interface Tunnel1
description TO_Wireless_Faculty
bandwidth 1000000
ip address 10.x.x.253 255.255.240.0
ip access-group deny_nbns in
ip helper-address 10.x.x.100
ip helper-address 10.x.x.101
no ip redirects
ip mtu 1476
ip pim sparse-dense-mode
tunnel source Loopback1
tunnel mode gre multipoint
mobility network-id 1
mobility trust
mobility tcp adjust-mss
mobility multicast

The output of show ip arp

show ip arp
Protocol  Address          Age (min)  Hardware Addr   Type   Interface
Internet  10.x.x.114        5652   3038.5541.5214  TUNNEL Tunnel8
Internet  10.x.x.126         994   9084.0da7.e68d  TUNNEL Tunnel8
Internet  10.x.x.66          6696   dc2b.6151.9bb4  TUNNEL Tunnel5
Internet  10.xx.124        1226   8c71.f8e5.ae28  TUNNEL Tunnel8
Internet  10.x.x.68         11103   a86a.6fa7.dc11  TUNNEL Tunnel5
Internet  10.x.x.115       11206   581f.aa17.dbda  TUNNEL Tunnel8
Internet  10.x.x.70          2333   b407.f938.c36b  TUNNEL Tunnel5
Internet  10.x.x.122       13955   e4ec.1047.a562  TUNNEL Tunnel8
Issue is, that these entries never time out (we found as entries as old as 10 days). As some of the Mobile Nodes leave, and never come back. the ARP remains there for 8 days (our DHCP lease time), then when a new Mobile Node get that IP address we get a message like this

*May 22 02:24:17: %L3MM-4-DUP_IPADDR: MN 5c57.c8ed.d0ba is requesting ip 10.13.66.81 which is being used by MN 7c6d.6215.6dcd

So, i would like to make the ARP entry in the TUNNEL to expire in 8 days (exactly the same timer as the DHCP lease time or lesser). This is something that has been happening for quite some time. I would like to solve this issue for once and all.