Vendor Tech Talk

Nov 21 2012   1:02PM GMT

Troubleshooting Network Problems – Part 3: Bandwidth & Traffic Analysis



Posted by: SolarWinds
Tags:
Network Management
Networking

We will be providing a four part blog series on troubleshooting network problems.  The series will address:

The only things in life that are certain are death, taxes and network issues.  Okay, I added the last one but we all know that you have undoubtedly heard complaints from end users that “the network is slow”.  In Troubleshooting Network Problems – Part 1: Network Device Performance I provided some tips on troubleshooting network performance issues by establishing a performance baseline and then collecting network device performance statistics.  Today’s post will provide some insight on using bandwidth monitoring and traffic analysis to troubleshoot network performance.

Performing each correctly assists the network administrator with identifying network bottlenecks that could be the cause of your “slow network”. It helps the admin identify the network needs and uses of servers and their hosted applications, as well as how the network needs of one IT service impacts the needs of another. It also delivers hard data that objectively verifies the ability of the network to meet stated Service Level Agreements (SLAs).

The two most common ways in which network traffic can be monitored and measured for performance are through packet analysis and flow analysis. Traditional packet-based monitoring tools enable peering into individual packets to determine their contents, the transactions between systems, and the details of communications being passed along that network.  The packet-based approach is a lot like attempting to determine the cause of a traffic jam by peeking into each individual vehicle. Knowing what people and cargo are travelling within each vehicle may be helpful in answering some questions, but it’s not likely to illuminate the cause of the system-wide slowdown.

Flow analysis, on the other hand, provides insight into the flow of traffic within the network, specifically the “who” and “what” of traffic consumption.  Flow analysis allows us to step back to see conditions on the system as a whole. To help you understand the differences in perspective here, let’s take a look at common ways used to measure traffic on a network:

  • Protocol analyzers – Protocol analyzers take a look at network conditions from the perspective of the packet. These tools analyze conversations between devices on the network from the location where the analyzer is measuring. This information gives the network administrator an extremely detailed view of individual transactions between two devices and the specific data being transferred between them.
  • Hardware probes and distributed analyzers – Hardware probes and distributed analyzers are an early attempt to overcome the limitations of an individual protocol analyzer. These tools can be positioned all across the network for the gathering of information. They go far in providing the whole-system perspective that is so difficult to gather through the previous two perspectives.
  • Traffic flow analyzers – These tools overcome the administration headaches of hardware probes and distributed analyzers by leveraging the data flow capture capabilities of the network device (router) itself. Traffic flow analyzers receive flow data directly from monitored devices and analyze that data to gain the high-level perspective needed for troubleshooting incidents across the network system.

NetFlow

NetFlow is a network traffic monitor protocol developed by Cisco Systems for collecting IP traffic information and monitoring network traffic. While the term NetFlow has become a de-facto industry standard many other manufacturers support alternative flow technologies including; Juniper (Jflow); 3Com/HP, Dell and Netgear (s-flow); Huawei (NetStream); Alcatel-Lucent (Cflow); and Ericsson (Rflow).

Routers and switches that support NetFlow collect IP traffic statistics on all interfaces where NetFlow is enabled, and later export those statistics as NetFlow records, toward at least one NetFlow collector – typically a server that does the actual traffic analysis. The NetFlow collector then processes the data to perform the traffic analysis and presentation in a user-friendly format.  NetFlow collectors can take the form of hardware based collectors or probes, or as network monitoring software collectors. SolarWinds NetFlow Traffic Analyzer (NTA) is an example of a software based NetFlow collector that collects traffic data, correlates it into a useable format, and then presents it to the user in a web based interface.

Monitoring and analyzing NetFlow will help obtain valuable information about network users and applications, peak usage times, and traffic routing.  In contrast with traditional SNMP-dependent systems, NetFlow-based traffic monitoring has the ability to characterize traffic from applications and users, understand the traffic patterns, provide a holistic view into bandwidth utilization and WAN traffic, support CBQoS validation and performance monitoring, be used for network traffic forensics, and aid in compliance reporting.

Configuring NetFlow on a Cisco router is a very straightforward and easy process.  You can use a free tool such as SolarWinds NetFlow Configurator or you can manually configure using the following steps:

Command Purpose
Step 1 Router> enable Enters privileged EXEC modeEnter your password if prompted
Step 2 Router# configure terminal Enters global configuration mode
Step 3 Router(config)# ip flow-export Version 9 Enables v9 data export for the main cache
Step 4 Router(config)# ip flow-export templates refresh-rate 15 (Optional) Specifies the refresh rate in number of export packets. packets is an integer from 1 to 600. The default is 20 packets.
Step 5 Router(config)# ip flow-export template timeout-rate 90 (Optional) Specifies the timeout rate in minutes. minutes is an integer from 1 to 3600.  The default is 30 minutes
Step 6 Router(config)# ip flow-export template options export-stats Specifies the options template export statistics, including how many export packets have been sent and how many flows have been exported.
Step 7 Router(config)# ip flow-export template options refresh-rate 25 (Optional) Specifies the refresh rate in number of export packets. packets is an integer from 1 to 600.  The default is 20 packets.
Step 8 Router(config)# ip flow-export template options timeout-rate 120 (Optional) Specifies the timeout rate in minutes. minutes is an integer from 1 to 3600.  The default is 30 minutes.
Step 9 Router(config)# end Ends the configuration session and returns to privileged EXEC mode

To display the statistics from the NetFlow data export, including statistics for the main cache and all other enabled caches, use the show ip flow export command in user EXEC or privileged EXEC mode.  The following is sample output from the show ip flow export command:

Router# show ip flow export

Flow export is enabled
Exporting flows to 10.42.42.1 (9991) 10.0.101.254 (9991)

  Exporting using source IP address 10.0.101.203
Version 5 flow records

Export Stats for 10.42.42.1 (9991)

3 flows exported in 3 udp datagrams
0 flows failed due to lack of export packet
3 export packets were sent up to process level
0 export packets were dropped due to no fib
0 export packets were dropped due to adjacency issues
0 export packets were dropped enqueuing for the RP
0 export packets were dropped due to IPC rate limiting

Export Stats for 10.0.101.254 (9991)

7 flows exported in 7 udp datagrams
0 flows failed due to lack of export packet
6 export packets were sent up to process level
0 export packets were dropped due to no fib
0 export packets were dropped due to adjacency issues
0 export packets were dropped enqueuing for the RP
0 export packets were dropped due to IPC rate limiting

There are a number of commercially available flow analysis and network bandwidth monitor products that greatly simplify the process of enabling NetFlow and then display the raw numbers into easy-to-interpret charts and tables.

Let’s take a look at three particular use cases for using flow analysis for troubleshooting bandwidth and traffic.

Applications

SolarWinds Netflow Traffic Analyzer
Top 10 Applications View

When an application on the network begins consuming more than its fair share of network bandwidth, its use will impact the capacity available for other network services. The problem with identifying these incidents using other types of network tools is that the reporting of problems tends to focus on the network service being impacted. For example, when the problem occurs, the network administrator usually starts with knowledge that Application B “is slow today.” The job is then theirs to determine why the service is slow and what is inhibiting its desired level of performance. Using effective flow analysis tools, the administrator can easily view the traffic and usage patterns across the entire network to identify that Application A is actually the culprit. Conversely, using tools with a closer perspective may incorrectly focus the administrator’s troubleshooting on Application B, while ignoring the impact of Application A.

Protocols

A second and similar issue occurs when a specific protocol over consumes network resources. Streaming protocols are an excellent example of this type of constant and predictable network flow. When users on a network make use of streaming applications, their consumption typically occurs at a constant level over an extended period of time.

Different than transaction-based protocols, streaming protocols have the tendency to saturate available network resources due to the additive effect of multiple streams. One user making use of one stream may not be likely to cause a network problem, but 50 or 100 users employing an equal number of streams quickly begins saturating the network. Unlike packet-based tools that analyze individual pieces as they go by, flow analysis tools enable the identification of the source, destination, and protocol of streams across the network. The end result is the ability to craft effective network policies that enable streaming protocols where necessary while preventing those that negatively impact the functionality of the network.

Top Talkers

A final area for which flow analysis tools are particularly well suited is the identification of top talkers or, who is consuming the bandwidth.  The Top Talkers feature of NetFlow can be useful for analyzing and troubleshooting network traffic in any one of the following ways:  Security by viewing a list of the top talkers to see if traffic patterns are consistent with Denial of Service (DoS) attacks; Load balancing through the identification of the most heavily used parts of your network; and general traffic study and planning for your network.

Tools for Network Troubleshooting

There are many open source, free, or commercially licensed products available to monitor and troubleshoot traffic and bandwidth.  Here are a few some guidelines on picking the right tool for your needs.

  • Multiple vendor device support – It would be very difficult in this day and age to find a network that consists of equipment from a single vendor.  While all vendors provide some type of tool or utility that will manage and monitor their own equipment, it is critical that you look for a tool that allows you to monitor all of your different vendors in a single pane of glass.
  • Support for multiple standard protocols including: SNMP, ICMP, and Syslog for network management; RDP, WMI, and WS –Management for Windows management; and NetFlow, J-Flow, sFlow, IPFIX, and NetStream for flow based traffic monitoring.
  • Real-time and historical analysis capabilities. Although most problems in network administration directly relate to how the network operates right now, the only effective way to ascertain today’s behaviors is to view them in comparison with yesterday’s or last week’s.
  • Visualizations accessible from anywhere. As a network administrator, you’re not always sitting in your office. Problems and issues tend to pop up all across the network, some of which require on-site support. In these cases, having visualizations that can be accessed from anywhere—for example, using a standard Web browser—gives you the ability to take your toolset to wherever the problem exists.
  • Drill-down support. With drill-down support it is possible to quickly move from the highest-level view down into specific problems as needed. Drill-down support reduces on-screen clutter, enabling a single-glimpse and high-level view during periods of nominal activity.
  • Affordability. Lastly, any toolset used in troubleshooting and resolving issues must cost less than the amount of benefit it provides. Expensive solutions take longer to pay for themselves and may be more difficult to obtain in a time of shrinking IT budgets. Finding the tool that meets your needs at an acceptable cost is important to gaining the biggest return on your investment.

By Brad Hale, Product Marketing Principal for SolarWinds. SolarWinds (NYSE: SWI) provides powerful and affordable IT management software to customers worldwide.

 Comment on this Post

 
There was an error processing your information. Please try again later.
Thanks. We'll let you know when a new response is added.
Send me notifications when other members comment.

REGISTER or login:

Forgot Password?
By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy

Forgot Password

No problem! Submit your e-mail address below. We'll send you an e-mail containing your password.

Your password has been sent to: