In my previous post I was talking about the problem we were facing with connectivity between Cisco 6500 Series switch and Cisco Nexus 7010 Switch. While troubleshooting we figured out there were no issues either with configuration of physical media (Fiber optic cable), the problem turned out to with 10GBASE X2 modules. Though the distance between these two locations is hardly 250 meters and there was a multimode fiber optic cable (supports 10G) running between these two locations. Initially a Cisco X2-10 GB-LX4 module was installed at Cisco 6500 Series Switch and at the Nexus 7010 we had a SFP SR Module.

Table 1. X2 Port Cabling Specifications

Intially everything looked good but after three months the problems aroused and the link start disconnecting, with the following errors

2011 Jun 12 09:05:03 VDC-Admin-CO01-MB-1256-001 %ETHPORT-5-IF_UP: Interface Ethernet1/17 is up in Layer3

2011 Jun 12 09:05:04 VDC-Admin-CO01-MB-1256-001 %ETHPORT-5-IF_DOWN_LINK_FAILURE: Interface Ethernet1/17 is down (Link failure)

2011 Jun 12 09:05:49 VDC-Admin-CO01-MB-1256-001 %ETHPORT-5-SPEED: Interface Ethernet1/17, operational speed changed to 10 Gbps

We figured out the multimode fiber optic cable used is of 62.5/125 microns. As per the Cisco X2 port cabling specifications we were supposed to use either Cisco X2-10GB-SR or Cisco X2-10GB-LX4 module at Cisco 6500 Series Switch and a SFP SR Module at Nexus 7010 Switch.

Seems like the problem occurred with the wavelength of fiber, once we replaced the Cisco X2-10GB-LX4 module with the Cisco X2-10GB-SR Module the problem was solved.

Upon looking at the log in Nexus 7010 Core Switch we found the following entries.

2011 Jun 12 09:05:03 VDC-Admin-CO01-MB-1256-001 %ETHPORT-5-IF_UP: Interface Ethernet1/17 is up in Layer3

2011 Jun 12 09:05:04 VDC-Admin-CO01-MB-1256-001 %ETHPORT-5-IF_DOWN_LINK_FAILURE: Interface Ethernet1/17 is down (Link failure)

2011 Jun 12 09:05:49 VDC-Admin-CO01-MB-1256-001 %ETHPORT-5-SPEED: Interface Ethernet1/17, operational speed changed to 10 Gbps

2011 Jun 12 09:05:49 VDC-Admin-CO01-MB-1256-001 %ETHPORT-5-IF_DUPLEX: Interface Ethernet1/17, operational duplex mode changed to Full

2011 Jun 12 09:05:49 VDC-Admin-CO01-MB-1256-001 %ETHPORT-5-IF_RX_FLOW_CONTROL: Interface Ethernet1/17, operational Receive Flow Contol state changed to off

2011 Jun 12 09:05:49 VDC-Admin-CO01-MB-1256-001 %ETHPORT-5-IF_TX_FLOW_CONTROL: Interface Ethernet1/17, operational Transmit Flow Contol state changed to off

2011 Jun 12 09:05:49 VDC-Admin-CO01-MB-1256-001 %ETHPORT-5-IF_UP: Interface Ethernet1/17 is up in Layer3

2011 Jun 12 09:05:49 VDC-Admin-CO01-MB-1256-001 %ETHPORT-5-IF_DOWN_LINK_FAILURE: Interface Ethernet1/17 is down (Link failure)

2011 Jun 12 09:07:37 VDC-Admin-CO01-MB-1256-001 %ETHPORT-5-SPEED: Interface Ethernet1/17, operational speed changed to 10 Gbps

2011 Jun 12 09:07:37 VDC-Admin-CO01-MB-1256-001 %ETHPORT-5-IF_DUPLEX: Interface Ethernet1/17, operational duplex mode changed to Full

2011 Jun 12 09:07:37 VDC-Admin-CO01-MB-1256-001 %ETHPORT-5-IF_RX_FLOW_CONTROL: Interface Ethernet1/17, operational Receive Flow Contol state changed to off

2011 Jun 12 09:07:37 VDC-Admin-CO01-MB-1256-001 %ETHPORT-5-IF_TX_FLOW_CONTROL: Interface Ethernet1/17, operational Transmit Flow Contol state changed to off

2011 Jun 12 09:07:37 VDC-Admin-CO01-MB-1256-001 %ETHPORT-5-IF_UP: Interface Ethernet1/17 is up in Layer3

2011 Jun 12 09:07:38 VDC-Admin-CO01-MB-1256-001 %ETHPORT-5-IF_DOWN_LINK_FAILURE: Interface Ethernet1/17 is down (Link failure)

2011 Jun 12 09:16:40 VDC-Admin-CO01-MB-1256-001 %ETHPORT-5-IF_HARDWARE: Interface Ethernet1/17, hardware type changed to No-Transceiver

2011 Jun 12 09:16:54 VDC-Admin-CO01-MB-1256-001 %ETHPORT-5-IF_HARDWARE: Interface Ethernet1/17, hardware type changed to 10G

2011 Jun 12 09:17:00 VDC-Admin-CO01-MB-1256-001 %ETHPORT-5-SPEED: Interface Ethernet1/17, operational speed changed to 10 Gbps

2011 Jun 12 09:17:00 VDC-Admin-CO01-MB-1256-001 %ETHPORT-5-IF_DUPLEX: Interface Ethernet1/17, operational duplex mode changed to Full

2011 Jun 12 09:17:00 VDC-Admin-CO01-MB-1256-001 %ETHPORT-5-IF_RX_FLOW_CONTROL: Interface Ethernet1/17, operational Receive Flow Contol state changed to off

2011 Jun 12 09:17:00 VDC-Admin-CO01-MB-1256-001 %ETHPORT-5-IF_TX_FLOW_CONTROL: Interface Ethernet1/17, operational Transmit Flow Contol state changed to off.

We tried to reset the Ethernet Interface in the Nexus 7010 Switch by using Cisco IOS command “shutdown” and “no shutdown”  but it’s of no use. The only way to solve this problem is to manually remove and reconnect the fiber patch cord connected to the Ethernet port in the Nexus 7010 Switch. Currently we are looking for the solution, the moment we find I will update you. Meanwhile if any solutions available make use of comments to help me.