Measuring application performance: Synthetic transactions vs. Real End User Monitoring
Posted by: Matt Stansberry
Performance Management, DataCenter
There are two ways of measuring application performance from the end user’s perspective: Synthetic transactions and real end user monitoring.
With synthetic transactions, an admin performs typical user functions while on a scripting screen; those actions are captured and run across multiple locations. “Companies with dispersed offices often use synthetic transactions to get a baseline of what a user is seeing for an ERP, HR or other corporate software application,” said David Langlais, director of product management at BMC Software. If an application takes longer than the determined baseline time to respond, IT managers would get an alert.
Synthetic transactions can also help IT managers track down a problem in the WAN. Is the problem inside or outside of the firewall? Run a synthetic transaction inside and outside the firewall to find out. “If my transactions are essentially the same from inside and outside the firewall, it’s not the outside network that’s impeding performance,” Langlais said.
Real end user monitoring tools use probes to track real user transactions in real time. “Companies that tend to face the public on the internet, Expedia, eTrade, Amazon, all use real user experience,” Langlais said.
So what model is the best fit for your organization?
Langlais outlined his recommendation in a recent email:
In general, I think corporate applications which are much more structured are well suited for synthetic. If there are many remote sites with many users (not so common in corporates) then RUM is a good complement. In general, RUM is well suited for mass public applications into e-commerce type applications where you are trying to find out how large groups from geographical areas are getting response. Here synthetic is a good complement to get the baseline response to critical paths through an application.
Synthetic is best for laying down baseline response times for the most common and most critical paths through an application. This is also very important when the application goes over either the Internet (as in a site like Expedia or Yahoo) or over a VPN over a corporate Intranet or public Internet. The implementation of synthetic there works best if synthetic transactions are used outside the firewall and inside the firewall in order to get the comparison of the effect of the Intranet or Internet.
RUM (or real user experience) is best at giving two main perspectives; the real responsiveness of an application (or portion thereof) for a group of users and the actual behavior of users of an application. This second part lets you know where customers thread through an application and where they have problems or abandon an application session.
RUM is limited in that you have no control over what most customers do (not so true for in-house applications with strict flows). In many public facing applications, users may go in and not follow all the way through (a good example is in Expedia when you don’t go all the way to book a flight that you have searched for).
Many proponents of RUM are satisfied with this limitation however and spend a lot of effort pulling out the information they want.
There are also outside services that will do synthetic transaction for you. Give them a script of your transactions, and they replay them from all over the world. “A lot of the Internet-facing sites do that, as well as a lot of large corporations,” Langlais said. “The cons of outside services is that you have less granular control over what you are measuring and the costs can escalate very quickly. Basically, the more you measure and make use of the measurements, the more you want to measure.”
For more info, check out our recent Application performance management tutuorial.
0 Comments
RSS Feed
Email a friend
