Gartner

Gartner and the Magic Quadrant: Send in the lawyers

This is a story of a small-time vendor who, after years of being stepped on by a billion-dollar analyst firm, throws open the windows of its San Jose office and screams, “I’m mad as hell and I’m not going to take it anymore.”

The question is, will anyone else join them?

ZL Technologies, an email archiving company who has languished in the niche corner of Gartner’s Magic Quadrant for four or five years, is suing the analyst firm for $1.6 billion, claiming defamation, trade libel, unfair competition and negligent market interference, etc.

Gartner contends that it has a First Amendment right to express its opinions on technology markets. It’s hard to argue with that.

As I have mentioned before, there are many people who dislike Gartner and its influence over the IT industry and claim that its Magic Quadrants are more about who you know and what you spend with Gartner than the quality of your product and your company.

My colleague Mark Fontecchio did a nice job of collecting some reaction to this lawsuit from the people who matter: IT decision makers.  It’s clear that for all the ridicule that bloggers, out-of-favor vendors and competing analysts heap on the Magic Quadrant, the market measuring tool still has significant influence over buying decisions. One IT veteran said the Quadrant represents about 20% of the decision-making process while another said a CIO should always review a Quadrant with his CEO before buying a product.

IT executives have a fair amount of skepticism about the Quadrant but they still rely on it to some degree. Every time I write about one, the page views come pouring in. Cisco might be the leader for the 100th time in a row for enterprise Ethernet switching or wireless LAN, but everyone still wants to know the nitty gritty about the Quadrant.

So what could Gartner do to reduce criticism? Well, how about more disclosure? Exactly how does that magic box get drawn up? Maybe the mathemtaical model should be disclosed. How much money does each vendor on the Quadrant spend with Gartner?  How many hours has the analyst spent with each vendor? How many customers of each vendor were consulted? These are facts that inquiring minds want to know.

More love lost for Gartner

As a follow-up to my post on analyst integrity, here is a rather visceral rant against Gartner from David Amsler, founder and CIO at ForeGround Security, an infosecurity consulting and services firm. Amsler’s post is a somewhat cheeky response to Larry Chaffin’s evisceration of Gartner at Cisco Subnet posted back in August.

Amsler’s main argument is that Gartner is an arrogant organization that brings little value to the IT industry.  Well, his words are actually, “they truly don’t provide any real value to our industry.”  For proof of arrogance, he points to this privacy debate which broke out on Burton Group analyst Bob Blakeley’s blog. He also deconstructs Gartner’s ballyhooed Magic Quadrants, calling Gartner’s method for constructing them “fishy.”

I’ve never had a problem with Gartner, myself. I have had run-ins with vendors who were unhappy with Gartner (and my coverage of Gartner’s research).  I find that most Gartner analysts — at least those who return my calls — are as knowledgeable as any other subject matter experts I talk to on a daily basis.

Since I’m a member of the press, people aren’t always candid with me about their views on subjects like this. Or their candor is issued to me under the blanket cover of off-the-record comments, which means I can’t share them with anyone.  In any case, I remain curious about this whole issue and I hope to hear from readers who have an opinion (whether on or off the record).

Industry analysts: A problem of integrity

Gartner VP and distinguished analyst Tom Bittman wrote a passionate rant on his blog yesterday defending his and his coworkers’ integrity as analysts.

In my 14+ years at Gartner, I have never, ever allowed a vendor to influence my opinion with anything but facts. Period. They have certainly tried to influence me with non-facts. I can say this definitively – it has never worked.

Tom’s rant has sparked an interesting debate among current and former analysts, bloggers and vendors in his blog’s comments section. It’s worth reading.

There has been plenty chatter in the blogosphere and elsewhere about whether analysts are offering any value to enterprise clients. Many analyst firms, such as Gartner, count both IT vendors and end user enterprises as customers. It’s hard to serve both masters while maintaining an appearance of integrity. This is probably why Gartner has an ombudsman... the only ombudsman I know of in the analyst industry.

Gartner’s use of an ombudsman doesn’t protect it from attacks on its integrity. James Watters, for instance, suggested recently that IBM’s favorable position Gartner’s Magic Quadrant for Web Hosting and Hosted Cloud System Infrastructure Services was due to IBM being a very important paying client of Gartner.  Trapeze Networks came to me after I wrote about Gartner’s last wireless LAN Magic Quadrant and suggested that Gartner had unfairly placed Meru Networks ahead of Trapeze because Meru’s current VP of marketing is a former Gartner analyst and one of the authors of the MQ was very briefly an employee of Meru.

It’s hard to prove your integrity. For some people, there just isn’t enough proof in the world to erase any doubts.  Journalists face the same challenge. In my newspaper days people would occasionally accuse me of being a mouthpiece for politicians. I knew I was doing my best to write fair and balanced stories, but the accusations still stung.

Gartner’s use of an ombudsman is significant, I think, although the office would have more impact if the ombudsman’s blog were updated more than once a month. An ombudsman is the voice of the client (or in a the traditional journalism sense, the reader). Clients will have more faith in a an analyst’s advice if the firm has someone on staff who tackles each and every significant attack on its integrity head-on, in full view of the public.

Gartner and other analyst firms should also disclose the nature of their paid relationships with the vendors that they evaluate for enterprise clients. This way, clients can understand the full context of the advice they are receiving.  Some firms do this. Others don’t.

If you could talk to the CEOs of the leading analyst firms out there, what advice would you give them to help them establish that they are giving unbiased and independent advice to enterprises?

Shocker! Cisco leads the pack in Gartner’s Magic Quadrant for enterprise LAN

When I saw that Gartner had published a new Magic Quadrant for enterprise local area network (LAN) infrastructure, I knew one thing was for certain. Cisco Systems would be THE leader in the market. The only question was for me was - how would the rest of the market shake out?

In this blog post I’ll review this year’s Magic Quadrant for the LAN market, and I’ll compare it to last year’s Magic Quadrant for Campus LAN infrastructure, which is essentially a measure of the same market.

As I wrote above, Cisco is THE leader in the LAN market, scoring high in both of Gartner’s criteria for the quadrant: completeness of vision and ability to execute.  In their assessment of Cisco’’s position, analysts Mark Fabbi and Tim Zimmerrman noted that Cisco maintains the broadest portfolio of LAN switching and WLAN technology on the market. The introduction of its Nexus switches have shown that Cisco is providing some leadership in addressing emerging connectivity demands in data centers.

However, Gartner cautioned that Cisco remains the high-priced vendor, with some workgroup switching products being twice as much as alternative products on the market. Gartner also said Cisco might be taking its customers for granted, especially those customers who believe in buying networking gear from more than one vendor. The analysts wrote:

We are hearing increasing concerns about Cisco’s presales organization taking customers for granted, and not providing expected levels of service, especially for customers that have not endorsed an end-to-end Cisco solution.

The only other leader in this Magic Quadrant is HP ProCurve, which was a leader last year as well.  Gartner described ProCurve as the fasted growing LAN switch vendor during the past two years and when clients speak with Gartner about their shortlists for vendors, ProCurve is the the second-most-asked-about vendor after Cisco. Gartner praised ProCurve’s integration into HP’s Technology Services group, which gives it access to HP’s broader sales force. It also praised ProCurve’s low cost of ownership and the successful integration of the WLAN technology it acquired with Colubris Networks.

But Gartner cautioned that ProCurve still lacks high-end core switches (An acquisition of a high end core switching vendor like Arista Networks or Blade Network Technologies would do the trick!). The company also needs to expand its channel for larger sales opportunities. ProCurve has in the past been known as a good vendor for SMBs.

A third leader from last year’s campus LAN Magic Quadrant fell down a notch in this year’s quadrant. Foundry Networks, now known as Brocade, the storage networking company that bought Foundry last year, was classified as a visionary in this year’s Quadrant, scoring high on its completeness of vision but scoring a little lower than last year in its ability to execute.

Gartner praised Brocade’s integration of Foundry but said Foundry lost momentum last year due to its U.S.-centric and data-center-centric sales focus. Gartner said it wants to see market evidence that Brocade’s integration of Foundry is successful and that Brocade can regain market momentum.  I have no doubt that last week’s announcement of a new Ethernet switching OEM agreement between IBM and Brocade will go a long way toward helping Brocade regain some of that lost momentum that Gartner is looking for.

Gartner identified three other visionaries in this year’s Quadrant: 3Com, Enterasys/Siemens and Extreme Networks.

Last year Gartner classified 3Com as a niche player, but it elevated the vendor to a visionary in this year’s Quadrant, giving it higher marks for its completeness of vision. Gartner praised 3Com’s revamped product lines and its growing market share in China and other emerging markets. H3C, 3Com’s Chinese subsidiary, has a 35% market share in China, for instance. And 3Com has a very large, low-cost R&D workforce in China. 3Com recently told me H3C has 2,300 engineers in China.  But Gartner cautioned that 3Com and H3C have been, until recently, run as two separate companies. It will be important for the two to integrate. Also, 3Com has very little market penetration outside of Asia. Gartner warned that taking products developed for China and selling them globally will be a challenge.

Enterasys, which merged with Siemens Enterprise Communications last year as part of a Gores Group acquisition, maintained last year’s position as a visionary. It drew praise from Gartner for it full complement of products from the data center to the access layer, its tightly integrated security technology, and good customer buzz around support and services. But Gartner said Enterasys’s market footprint remains small and its distribution channel is limited. Marketing has also been weak, Gartner said, as the market waits for the new combined company Enterasys/Siemens to change its name.

Extreme Networks, the third visionary in the Quadrant, drew praise for broadening its XOS-based switch line and its policy-based configuration and open architecture. But Gartner noted that Extreme is struggling to maintain revenue and it remains one of the smallest vendors in the market. Gartner also cited some support issues affecting the company’s install base.

Gartner identified two niche players in this year’s Magic Quadrant. First there is Nortel, which was downgraded from its visionary status in last year’s Quadrant. Gartner cited Nortel’s bankruptcy as an impediment to the company competing for new business. Gartner is predicting significant loss of market share and revenue for the company as it remains in bankruptcy. Gartner also said Nortel needs a new core switching platform.

The second visionary, Alcatel-Lucent, drew praise for a solid product strategy and its growing market share and revenue; however, Gartner said the company needs to invest more in R&D to keep pace with the latest innovations in data center switching and wireless LAN technology.

Force10 Networks, which was identified as a niche player last year, was dropped altogether from this year’s Magic Quadrant because it no longer meets Gartner’s revenue requirements for inclusion, whch is 1% of ports sold overall or 5% of ports sold in a specific market segment.

Gartner also noted that Juniper Networks has entered the Ethernet switch market, but it hasn’t earned enough of a revenue share to be included in this year’s Magic Quadrant. Juniper’s switches earned the company $56 million in 2008.

So there you have it, for what it’s worth. Cisco remains on top, but the other players in the market continue to make moves. ProCurve and 3Com are on the rise. Nortel and Force10 are in decline. Everyone else is looking to take a step forward.

AT&T climbs to top in Gartner’s Magic Quadrant for global network service providers

Gartner has published its first Magic Quadrant for the global network service providers since the summer of 2007.

As you probably know, the Magic Quadrant is a market assessment tool that rates vendors on their ability to execute (their service quality, pricing and track record) and their completeness of vision (an understanding of what enterprises want from their providers). Vendors who score high in ability to execute are challengers.  Vendors who score high in completeness of vision are named visionaries. Those who excel in both areas are leaders. Those who fail to excel in both areas are niche players.

The key takeaways from this quadrant:

• AT&T has overtaken Orange Business Services and BT Global Services as the highest rated provider in the leaders quadrant.
• The four providers in the leadership quadrant (AT&T, Orange, BT and Verizon Business) are all very tightly grouped together, meaning their abilities to serve enterprises are pretty evenly matched.
• Gartner has added three providers to the quadrant: Telefonica, NTT Communications, and Reliance Globalcom. All are rated as niche players.
• Cable & Wireless has slipped from visionary to a niche player.
• T-Systems has climbed from the niche player quadrant to become the only challenger in the market.

In their market overview, authors Neil Rickard and Robert F. Mason wrote: “There are four leading providers, with the remaining players way behind, often focused on a particular market segment.”

They added that the increase in the number of providers in the Magic Quadrant represents increased competition, which is good for enterprises. However, it also means that smaller providers will struggle to compete in the market because they lack the ability to scale. They predicted that some market consolidation is possible given the global economy.

Trapeze not flying high over Magic Quadrant

Every year Gartner’s Magic Quadrant for wireless LAN infrastructure has some winners and some losers. One or two vendors will emerge from the crowded quadrant of niche players to become a market leader, a visionary or a challenger. And one or two other vendors will slip back into the crowd of niche players. This year, one of the vendors who came out on the losing end is accusing Gartner of having, at the very least, an appearance of a conflict of interest.

Brian Johnson, director of public relations for Trapeze Networks, called me last week and implied that Meru Networks is receiving favorable coverage from Gartner over Trapeze. Gartner placed Meru in the visionary quadrant for the second year in a row, while Trapeze slipped from visionary status to niche player.

Johnson revealed to me that Tim Zimmerman, one of the Gartner analysts who wrote this year’s Magic Quadrant, is a former employee of Meru Networks. I checked around and indeed Zimmerman was director of industry marketing for Meru Networks from Octbor 2007 to January 2008. Johnson also pointed out that Gartner’s former research director for wireless LAN technology, Rachna Ahlawat, is currently the vice president of strategic marketing for Meru.

Johnson explained that Trapeze has had a good year and is a superior company to Meru. He said it “stretches the imagination” that Meru could be ranked higher than his comapny.

“We have a higher market share than Meru,” Johnson said. “We have more OEM relationships. And we are a public company with a large bankroll behind us while Meru is a private company that is rapidly burning through its cash… In terms of ability to execute, I think that Trapeze has a higher ability to execute than Meru can.”

Johnson also told me that Trapeze brought eight products to market this year and three of them won awards (I looked through a list of press releases on Trapeze’s website and didn’t see that many product releases, but perhaps I missed a few). He also pointed out that Trapeze won the largest wireless LAN deployment in the world this year when it closed a deal with the University of Minnesota.

Johnson was reacting to a story about the Magic Quadrant which I wrote last week. When I talked to Mike King, Zimmerman’s coauthor, for that story, he told me that Trapeze’s downgrade was reflective of its relative silence on the market since it was acquired by Belden over the summer. He suggested that things have slowed down at Trapeze while Belden goes through the process of absorbing it. And he predicted that Trapeze could lose some key OEM partners when its deals with those expire in a few months. All this can be fairly typical for mergers and acquisitions. Motorola experienced a similar decline on the Magic Quadrant when it bought Symbol Technologies, but it has since rebounded and is now identified as a market leader by Gartner.

Now any industry veteran will tell you that analysts take jobs with vendors all the time and research firms like Gartner commonly hire analysts from the vendors they cover. Ahlawat left Gartner for Meru in June of 2007 so it’s been well over a year since she’s had any relationship with the firm. However, Zimmerman left Meru less than a year ago, so it was worth my talking to Gartner about this issue.

First I talked to Andrew Spender, Gartner’s vice president of corporate communications. He said Gartner employs a variety of measures to ensure that its analysts are independent and objective.

“First we have our principals of ethical conduct and our code of conduct which all our analysts sign up to as soon as they join the company,” he said. “They have very intensive training in what that code of conduct means and how they need to adhere to it. It’s very specific in terms of accountability.”

Spender also said that no piece of Gartner research is ever the work of one single analyst.

“When you buy a piece of research or become a Gartner client, you obtain the research from Gartner, not from an individual analyst. Each piece of research is peer reviewed. Our community of 650 analysts have a formal obligation to do peer reviews of other analysts’ research to ensure that any kinds of inconsistencies, any errors of data collection or any errors of conclusions are challenged and corrected before the research sees the light of day.”

I also spoke to Larry Perlstein, Gartner’s ombudsman (Gartner is the only analyst firm I know of that employees ombudsmen), about this matter. He has already conducted an investigation of Trapeze’s complaint.

“Basically I didn’t find anything that made me concerned that there was any real fact in Trapeze’s issues. The analyst that they expressed a special worry about, who was formerly at Meru, was there for only a very short period of time, about three months. It wasn’t clear that anything in that involvement was going to dramatically influence this particular piece of research. Most of our analysts come from vendors. As part of our hiring process we try to ensure that people have the capacity and potential to be balanced and objective.”

On the same day that I spoke to Johnson at Trapeze about this issue, I happened to chat with David Callisch, vice president of marketing at Ruckus Networks. Ruckus is another niche player in this year’s Quadrant, ranked a little lower than Trapeze.

“Tim Zimmerman and Mike King are both very stand up guys,” Callisch said. “I thought we had a pretty mediocre spot on the Quadrant, but to be quite objective, who are we to say? Vendors always think they deserve a better spot… But Tim and Mike did a lot of due diligence. I think they did a good job even if we got a lousy spot.”

He said that Gartner placed Meru high probably because the firm likes the innovative single channel approach Meru takes with its access points, which solves voice roaming very well. He said he has doubts about whether this approach can scale as well as more mainstream wireless LAN technologies, but he doesn’t fault Gartner for giving Meru high marks for their technology.

Callisch went on to call out Trapeze for its acquisition by Belden. He said Trapeze has a very good product line, but it had marketed its technology poorly, driving down the value of the company.

“They ended up being sold to Belden for pennies on the dollar and that hurt the valuation of other [wireless LAN] companies” he said.

Grading the analysts

Chances are you deal with industry analysts pretty often, whether your company pays for their services or whether you meet with them at conferences. And if you don’t talk to them, your boss probably does. As a tech journalist, I talk to a lot of industry analysts on a daily basis. I rely on them for insight into nearly all the stories I write and edit.

Over the years I’ve recognized that analysts are like anyone else. Most of them know what they’re talking about. Some of them don’t. Sometimes its hard to know when these people who are paid to be the experts on a subject really are the experts they’re purported to be.

That’s why it’s so useful to hear from paying customers of these analyst firms about their experiences. Sam Lawrence, chief marketing officer of Jive Software, a social software vendor,
blogged this week about his experiences with two shops: Gartner and Forrester Research, two of the the biggest IT research and consulting firms out there. He’s assigned them letter grades based on how they treated his company, both before Jive became a paying customer and after it signed up with them.

Forrester scored a B and Gartner scored a C-. Lawrence was happier with Gartner before he became a paying customer. After he signed on with them, its performance declined, according to his blog. Forrester has been relatively steady in its performance throughout.

From my own experience, I’ve found that Gartner and Forrester both have a number of intelligent and well-informed analysts who are always willing to help me with a story. I couldn’t do my job without them. And both firms have very helpful media relations people on staff. In general, however, I’ve always found Forrester’s analysts more accessible on a daily basis. I’m not sure if their paying clients feel the same way, but with Lawrence and Jive that seems to be the case.

Lawrence also offered praise for some of the smaller, more specialized research firms out there, such as The 451 Group, RedMonk, Jupiter, and specifically Mike Gotta, an analyst with the Burton Group who specializes in collaborative technology. I’ve worked with all of these firms over the years, along with Mike and a bunch of other people at Burton Group, and I agree that they’re a big help.

What firms do you depend on to give you expert advice?