Storage market research reports

TIP: 2010 storage spending outlook slightly optimistic

A report released this week by TheInfoPro says 2010 storage spending will probably be an improvement over 2009, but that’s not saying much.

The numbers released this week are the result of interviews with Fortune 1000 storage professionals, according to the TIP report.

Out of 252 respondents to the ongoing Wave 13 study, 27% said they expect a decrease in spending between 2010 and 2009, 31% said their budgets would remain flat, and 42% said budgets would increase.

This is better than the earlier Wave 13 numbers from the beginning of the year in which 36% of 258 respondents expected a decrease between 2008 and 2009, 26% thought it would be flat and 38% expected an increase.

TIP also broke out the size of the increases or decreases expected next year. Nearly 20% of those who expect an increase expect it to be between 1% and 10 %. Interestingly, the next largest category among those who expect an increase, close to 15%, expect an increase of 50% or more. However, of those who expect a decrease, more than 10% — the largest group — expect that decrease to be more than 25%.

Overall, 71% are in the are ranging from a 10% increase to a 10% decrease. That’s better than a sharp decrease, but hardly the “pent up demand” you hear about in some areas of the market (though maybe what’s being referring to is that 15% who expect a 50% increase, in which case I hope those few customers have someone screening calls and guarding the doors for them…). After spending plummeted between 2008 and 2009, flat-lining into 2010 isn’t necessarily good news, it’s just not more bad news.

Email archiving vendor sues Gartner over Magic Quadrant

Claiming that Gartner’s Magic Quadrant vendor-ranking reports constitute “disparaging, false/misleading, and unfair statements” about its email archiving product that have done damage to its sales prospects, ZL Technologies Inc. said today that it filed suit May 29 in US District Court in San Jose, Cali. against the analyst firm. The suit seeks damages of $132 million to account for what ZL says are lost sales, as well as punitive damages.

Gartner responded with a motion to dismiss the lawsuit in July, to which ZL filed a counter-motion. A hearing on these later filings is scheduled for this Friday.

ZL alleged in its initial complaint that Gartner consistently ranks its product in the lower left quadrant of its report, the “Niche” category, because its sales and marketing are not as strong as Symantec’s Enterprise Vault, which is consistently ranked in the highest Leader category. ZL’s contention is that this consitutes an unfair and defamatory means of evaluating and recommending products that has caused damage to its business. “Gartner continues to harm ZL and help entrench vastly inferior products in the American economy whose principal virtue, according to Gartner, is good sales and marketing,” the complain alleges.

What I found surprising about this initial complaint was the arguments ZL provided about just how much some end users rely on the Magic Quadrant report for purchasing decisions. Some examples:

Purchasers of the ZL Products have consistently and uniformly raised objections to even consider purchasing the ZL Products because of the Defamatory Statements. Even Oracle Corporation, one of the largest software vendors in the world, which resells the ZL Products, complains that it gets “Gartnered” when pursuing prospective customers for the ZL Products, i.e., that a prospect would not even consider looking at the ZL Products because of the low Gartner rankings. The power of a positive ranking in Gartner is immense because it is often the case that large purchases of technology are based exclusively on the MQ Reports.

For instance, the Office of the Inspector General, Department of Veterans Affairs (VA) recently conducted an investigation into the use of the Gartner’s MQ reports in connection with the VA’s $16,000,0000 purchase of certain leases and services from Dell. The Office of Inspector General reported that the VA made this large purchase based solely on the leadership rankings in the relevant Gartner MQ report.

[...]

In March 2009, ZL entered into contract to provide the email archive solution for one of the largest and most influential companies in the Silicon Valley. What makes this customer win especially relevant to this action is that: (a) the customer did not initially invite ZL to compete because of the Defamatory Statements, and (b) ZL won the contract only after beating out Symantec in an exhaustive, side-by-side “proof of concept” evaluation. Such a large customer win demonstrates that, but for the Defamatory Statements, ZL would have made many more sales than it has.

The complaint goes on to cite 10 more examples of potential sales in which ZL claims it was not invited to participate in a customer’s evaluation process, or pre-sales discussions were discontinued on the basis of the Magic Quadrant.

Gartner’s response is that the Magic Quadrant report amounts to a First-Amendment-protected statement of opinion, and that its rankings do not constitute a “false or misleading statement of fact” but rather a subjective conclusion.

ZL’s response in an opposition statement to the motion to dismiss argues, “Gartner tells the public that its research is “objective, defensible and credible”—it cannot now be allowed to escape the consequences of its misconduct by claiming the exact opposite, that its statements cannot be taken as anything more than its subjective opinion based on pure speculation and conjecture.”

ZL further argues that “a defendant may be held liable for statements of opinion that imply the existence of undisclosed facts. Gartner expressly stated that its statements had a factual basis, and intended that they be understood as being derived from a fact-based analysis, and can therefore be held liable even if the statements themselves are couched as opinion.”

“Try as it might, ZL cannot create a dispute where there is none,” Gartner countered further in a reply to ZL’s opposition to the motion to dismiss.

ZL alleges at great length in its Complaint (and recapitulates in its Opposition) that it has a strong product and satisfied customers. The Magic Quadrant reports do not say otherwise; the real point of contention here is not the quality of ZL’s product, but instead the subjective analytical model Gartner used to assess ZL’s market position and prospects. ZL does not contest Gartner’s basic assessments of ZL—that it has a good product but needs to expand its sales and marketing—but ZL challenges its placement on the Magic Quadrant Report because Gartner uses a “misguided analytical model” that gives “undue weight to sales and marketing.” Complaint ¶ 10. As the law makes clear, such analysis constitutes non-actionable opinion. None of ZL’s arguments dispute this bedrock principle. Instead, ZL focuses on a straw-man defamatory statement (that ZL’s product is “inferior”) that never appeared in—and is contradicted by—the plain text of the Magic Quadrant reports.

Personally, though I’m not a lawyer or legal expert, I think it’s unlikely that ZL can win its case. In one recent case in which securities rating agencies’ opinions relating to the subprime mortgage crisis were found not to be protected by the First Amendment, there were some different circumstances, such as one judge’s opinion that plaintiffs had sufficiently proven ratings agencies did not sincerely believe their own statements had basis in fact. The lawsuit between ZL and Gartner, on the other hand, seems to come down to the weight given to sales and marketing strength over technical product specifications rather than Gartner’s sincerity.

It would be easy to point out here that potential customers could also perform more of their own internal testing of products and not weigh the Gartner quadrants so heavily in their purchasing process, but it’s unclear whether that’s realistic in all cases. It also seems to be a matter of subjective opinion how important size of vendor and strength of sales are in the evaluative process of purchasing technical products. In a perfect world, maybe product evaluation would be a true meritocracy, but who among us hasn’t heard the old chestnut that “nobody ever got fired for buying IBM?”

I’m very interested in the peanut gallery’s response to this. Does ZL have a point about the weight being given to a subjective report in technical purchasing decisions? Or is this a case of impugning an evaluative process because of a disliked outcome?

Widespread IT staffing cuts this year, according to new survey

A new survey of some 200 IT executives across a dozen vertical markets by IT research firm Computer Economics found that 46% of respondents plan to reduce headcount this year, while 27% plan to increase headcount.

The report says healthcare and energy are faring better than other industries, with 60% of healthcare respondents and roughly half of energy and utility organizations reporting staffing increases. Retail, manufacturing and insurance will see the biggest declines, according to the report.

While capital purchases seem to be the most sensitive area for organizations with slashed IT budgets, operational expenditures are a murkier area. The question of staffing and how different organizations are addressing storage efficiency - through technology or operational improvements - seems to come down to organizational philosophy. I’ve talked to users during this economic downturn who say that their IT spending is going up because IT projects are being implemented to automate processes or cut down on spending elsewhere.

The Computer Economics report identifies finance as one industry where this phenomenon is taking place. “Certain sectors, however, are showing positive growth in their 2009/2010 IT operational budgets. These sectors include banking and finance at 4.9%, healthcare providers at 4.7%, professional and technical service firms at 4.0%, and utilities and energy at 1.3%.” These operational budget increases seem to run counter to some vendor marketing in the down economy encouraging users to trade some capital costs for a reduction in operational costs through automation.

Some vendors, like EMC Corp. have also been predicting stabilization in the economy and IT organizations by the end of this year, but the survey results show “the worst may not be over,” according Computer Economics’ press release. “Many IT executives expect further budget reductions in the future. About 49% reported that they expect to spend less than the amount allocated in their 2009/2010 IT spending plans compared to only 9% who anticipate being able to increase their IT budgets.”

Though it’s an interesting set of data points within the ongoing discussions of the economy and storage efficiency, I would also point out that with a sample size of 200 administrators, it’s not necessarily a definitive report. I’m hoping more research like this is being done which can be compared and contrasted with these results.

IDC: Storage spending fell off a cliff in first quarter

This had been apparent from the earnings reports of the major storage vendors, but it’s still startling to see the numbers all lined up at once: analyst firm IDC released its worldwide quarterly disk tracker for the first quarter of 2009 today, and the numbers are not pretty.

Worldwide external disk storage systems factory revenues fell 13.6%. The total disk storage systems market declined to $5.6 billion in revenues, an 18.2% decline. Networked storage (NAS plus SAN) fell 12.5%. SAN declined 14.3%. Every one of the top five vendors posted a revenue decline, all but two of them (Dell and Hitachi) in the double digits.

Even in the fourth quarter of 2008 when the the economic crisis was already in full swing, year-over-year revenue only declined 0.5%.

But we knew the first quarter had been awful - maybe it’s even more interesting to see what didn’t decline. For one thing, while revenues plummeted, capacity growth remained positive, up 14.8% compared with last year. “These contrasting results are due to a combination of currency implications, lower overall sales, shifts in product mix, and aggressive pricing actions,” according to Steve Scully, research manager, enterprise storage, as quoted in an IDC press release.

Entry-level price bands (under $14,999) grew 9% and the midrange price band ($15,000 – $49,999) was flat. That might explain all the entry-level NAS systems we’ve seen introduced over the last few months. The very high end ($300,000 and up) saw growth, also the result of discounting, according to IDC.

In what may be the most interesting data point of all, iSCSI SAN showed 40.5% revenue growth compared to the prior year’s quarter. Dell led the market with 36.4% revenue share, followed by EMC with 15.8%. I don’t think small companies have some advantage in this crisis big companies don’t. Companies of all sizes are probably buying low-end products to bridge the gap.

IDC research manager Natalya Yezhkova agreed. “We don’t do analysis by company size,” she said. “But one of the products that’s done really well on the low end is JBOD, which goes into expanding the capacity of servers, specifically blade servers, and that’s usually at larger companies. Users are trying to expand at low cost what they already have.”

Storage vendors like EMC have predicted stabilization in the middle quarters of the year, with at least signs of recovery toward the fourth quarter. Yezhkova said that in IDC’s next quarterly disk forecast “the magnitude and rate of recovery may have changed, but the overall sense that we’ll see recovery by the end of teh year is still the assumption.”

Obama stimulus plan offers funding for storage, IT industries

Although the Senate and House are still hammering out details of the economic stimulus package, the current plan includes provisions that would bring funding to the storage and IT industries.

Broadband networking investment is one big issue addressed by the stimulus. According to a whitepaper posted by the Information Technology and Innovation Foundation (ITIF), “ITIF estimates that spurring an additional investment of $30 billion in America’s IT network infrastructure in 2009 will create approximately 949,000 U.S. jobs …We also estimate that approximately 525,000 of these jobs will be in small businesses [fewer than 500 employees].” The $30 billion referred to includes “spurring or supporting” $10 billion investment in broadband networks over one year.

Storage industry experts who weighed in on Obama’s technology investment plans around the New Year also supported such investments, and respondents to a Web poll identified broadband investment and neutrality as the most important technological issue for the new President’s administration.

Next up on the list are new plans for “smart” sensors on public infrastructure like electrical grids. Every one of those sensors generates data, and IT vendors, most recently IBM, are eagerly anticipating the deluge of infrastructure requirements this will eventually bring about. According to an IBM press release announcing its new dynamic infrastructure and cloud computing initiatives yesterday, the world is now generating 15 PB of new data per day. That’s a lot of disk drives, and at least some job security for those tasked with keeping them spinning.

Perhaps the heftiest chunk of spending, though, is the $20 billion laid aside as part of the plan for digitizing healthcare records. States like Massachusetts have also already passed laws requiring electronic medical records. Something’s got to give there, too, according to industry experts, some of whom, like Compellent Technologies Inc.’s vice president of marketing Bruce Kornfeld, told me they’ve recently gotten a patient’s-eye view of the deficits in how some healthcare records are handled when two of his physicians faxed, in low-quality black and white, some images that had originally been taken in high-res technicolor.

In addition to these items, Seagate’s official blog points out some more tech stimulus in the Obama plan, including “Direct IT investments - government modernization, like a $400 million computer for the Social Security Administration…[and] Direct storage purchases by stimulated consumers - home storage is becoming a mainstream consumer electronics category.”

Meanwhile, even as dismal jobs reports come rolling in from the wider economy, IT industry analysts have suggested that the picture in IT is looking less grim. According to a report released by Foote Partners last week, “IT jobs continue to show strong counter movement against national jobs trends.” Foote’s report found “growth in pay for skills in Architecture, Project Management, IT Security, Database, Networking, Communications, and Methodology and Process skills and certifications over the past three months.” Foote also cited numbers recently released by the Bureau of Labor Statistics (BLS) which show 9,000 jobs were added in October and November in the Management and Technical Consulting Services employment category, and another 11,000 were added in January.

Even though it’s early in the process, some members of the storage blogosphere are already starting to feel the effects of federal spending on IT. “This week the federal government bailed me and my little almost-a-real-company out,” wrote Jesse Gilleland in a post on his blog, SanGod.com, titled A bailout for little old me?

Several years ago I worked for a particular federal client. Data migration, switch migration, cable remediation, and a number of other projects. Apparently it went well because for years they’ve been trying to get me back in and it just came down the wire that I’ve just been awarded a 1-year contract to go in as their new storage admin. So the company is saved, or at least has been granted a one-year stay of execution.

Pillar claims SPC-1 supremacy

Pillar Data Systems has published its first system test benchmarks for the Axiom 600 disk array via the Storage Performance Council (SPC). It tested a system with 42 total TB, including 10 TB used and mirrored to allocate a total of about 20 TB. The results were 64,992 total IOPS.

These numbers come in ahead of competitive systems from IBM, NetApp and EMC. The EMC CX 3-40 was tested last January by archrival NetApp, making its SPC benchmarks controversial, but as listed a 22 TB system with 8.5 TB used and mirrored produced 24,997 SPC-1 IOPS. A NetApp 3170 with 32 TB and 19.6 used and mirrored resulted in 60,515 June 10. A 37.5 TB IBM 5300 with 13.7 TB used and mirrored produced 58,158 SPC-1 IOPS Sept. 25.

I found it interesting that with one 2,000-IOPS deviation between the IBM 5300 and NetApp 3170, the systems generally performed better according to which had been most recently tested. Note also how much of an outlier EMC is, both in terms of capacity used and total capacity. It was also an outlier in its free space, with just under 1 TB unused. IBM and NetApp both left approximately 5 TB of free space in their configurations, and Pillar had 16 TB of free space.

I do have to wonder how much weight users give to these industry benchmarks when selecting a product.  NetApp’s submitting EMC systems to SPC, a flap last summer over server virtualization benchmark testing, and  continued inconsistency among vendors as to who submits systems for benchmarking leaves a lot of potential reasons to take benchmarks with a grain of salt.

IDC report shows steady storage sales in third quarter, some warning signs

IDC’s quarterly tracker numbers for the third quarter of 2008 show disk storage and storage software sales holding steady at a time when many industries are feeling the effects of recession.

According to an IDC press release, “worldwide external disk storage systems factory revenues posted 8.8% year-over-year growth totaling $4.9 billion…total disk storage systems market grew to $6.6 billion in revenues, up 1.1% from the prior year’s third quarter, driven by softness in server systems sales.”

Meanwhile, the storage software market grew year-over-year for the 20th consecutive quarter with revenues of $3.1 billion, up 11.6% over last year’s third quarter.

On the disk side, companies with server businesses showed declines. IBM disk revenue declined 18.1%, Dell dropped 8.7% and HP was down 0.5% in overall disk system revenue (including servers).  Fujitsu Siemens and NEC also declined. But for external (networked) storage, HP increased 3.3% and Dell was up 8.6a% over last year. Storage system-only vendors EMC (16.2%) and NetApp (13.8%)  gained significantly over last year, as did Sun (up 25%).

Year-over-year numbers looked similar on the software side, with outliers like HP increasing revenue 106.2% in storage management software from one year to the next. However, nearly all the storage software vendors stumbled from the previous quarter. HP took a 19.7% sequential hit in storage management. Storage infrastructure software slipped 7.8% from the previous quarter, with NetApp revenue declining 18.3% in that category.

According to IDC’s software press release, storage software revenues in the third quarter are traditionally slower before a typically strong fourth quarter. However, the overall economy has been going in the opposite direction. Many industries look at the third quarter as the calm before the storm, with dire predictions of declines coming for next year. And having covered these trackers before, I can say anecdotally I don’t recall seeing quite such sharp declines one column (the quarterly comparison) affecting almost all companies and almost all categories.

Other industry experts have told SearchStorage.com that the worst is yet to come. According to a report issued in October by Forrester Research, the third quarter for IT companies remained relatively stable because most vendors are still working through a sales pipeline. But poor sales are predicted for all IT vendors in the fourth quarter.

In the meantime, however, while budget growth may be constrained next year, storage managers have said they aren’t expecting their daily tasks to change drastically because of the recession.

Users look increasingly to storage virtualization as data grows, analyst says

I had an interesting conversation today with TheInfoPro’s managing director of storage research Rob Stevenson about the results of his firm’s latest survey of 250 Fortune 1000 and midsize enterprise storage users. Fortune 1000 users surveyed by TIP cited block virtualization as having the biggest impact on their environment this year. TIP expects 50% of the Fortune 1000 to have virtualization in use by the end of 2009. All of this is in response to ongoing and relentless data growth.

“Impact” is difficult to define, as TIP doesn’t offer definitions or parameters to the open-ended question for users, instead letting the responses shape the definition. (Midrange users cited server virtualization as having the biggest impact, and we all know that there are good and bad impacts involved).

What really stood out to me, though, when I discussed the results (as well as the semantics of the word impact) with Stevenson, was how block virtualization is being used and for what purposes. My general impression has been that block storage virtualization has not lived up to its initial round of hype as a “silver bullet” for single-pane-of-glass management of an overall storage environment. I wondered, had that changed when I wasn’t looking?

According to Stevenson, while adoption for block virtualization has risen steadily even since this past February (number of respondents with the technology “in use” went from 21% in February’s Wave 10 to 23% in Wave 11), 23% said they’re using it with just 2% of the overall storage capacity.

Stevenson said the users in this case were petabyte-plus shops that in the past year or so have seen storage balloon from the single petabyte range to 2.5 petabytes or more, with no signs of stopping. These admins are scrambling to consolidate storage, move to new technologies that offer better utilization, and automate tasks. Where block virtualization comes in for most of them is performing data migration while moving to new technologies or systems — hence the relatively small proportion of data being managed by block virtualization devices from day to day.

Meanwhile, midrange enterprises are increasingly looking to maximize their resources on the server side.  Close behind that, though, come utilization improvement technologies for storage like thin provisioning and data deduplication.

It’s largely a matter of consolidating resources and improving utilization rates. “But the big Fortune 1000 shops have a bigger ‘legacy drag’ of data that they have to move,” Stevenson said. Hence the use of block virtualization tools.

Stevenson said continued data growth and an increasing amount of complexity to go along with it–storage managers are not only managing an average of 400 TB each compared to 200 TB each a year ago, but the number of LUNs to manage within that volume is also increasing. That drives a need for automated management. While the most popular use case for virtualization seems to be data migration, Stevenson said users are finding day-to-day data movement is also increasing, bringing these devices to the forefront once again for management.

Does this mean we could be seeing block virtualization tools proliferate once the midrange market reaches the petabyte level? (After all, as the old chestnut goes, a megabyte used to be a lot of data). That’s where Stevenson’s crystal ball grows, well, cloudier.

Right now, one tentative theory is that users whose data centers already have large amounts of data under management tend to also already have specialized staff. But as the midrange market comes up against a need to scale staff as well as technology, they may turn to service providers before turning to storage virtualization devices.

“In large data centers, we see the pooling of resources among multiple data center groups to balance workload, like moving storage networking to a networking team or data classification and archiving to server and application groups,” Stevenson said. “When it comes to midsize admins having to start ‘not doing things’, it’s probably not going to come with an increase in internal staffing–instead they may look to offshoring those tasks to cloud service providers.”

But that’s not to say large enterprises won’t be looking up at the clouds, too. ”We’re still working out the ‘competing futures’ if you will,” Stevenson said.

IDC: Unstructured data will become the primary task for storage

According to a new IDC Enterprise Disk Storage Consumption Model report released this week, transaction-intensive applications are giving way as the main segment of enterprise data to an expanded range of apps as well as a tendency to create more copies of data and records for business analytics including data mining and e-Discovery.

The report estimates that unstructured data in traditional data centers will eclipse the growth of transaction-based data that until recently has been the bulk of enterprise data processing. While transactional data is still projected to grow at a compound annual growth rate of 21.8%, it’s far outpaced by a 61.7% CAGR predicted for unstructured data in traditional data centers.

“In the very near future, the management and organization of file-based information will become the primary task for many storage administrators in corporate datacenters,” the report reads. “And this shift will have a significant impact on how companies assess storage solutions in terms of systems’ performance, operational efficiency, and file services intelligence.”

The IDC report also builds on research first highlighted in an IDC blog last week concerning the cloud. According to the report, the sharpest growth in storage capacity will come from new organizations described as “content depots.” IDC estimates storage consumption from these organizations will grow at a compound annual growth rate of 91.8% through 2012. Examples of content depots  include the usual cloud suspects: Google, Amazon, Flickr, and YouTube.

These content depots have different IT requirements and infrastructures than traditional enterprise data centers. We’re seeing examples of these new  infrastructures pop up in the market, including systems with logical abstraction between the hardware and software elements; the use of commodity servers as a hardware basis for storage platforms; and the use of clustered file systems.

Some in the industry have compared this “serverization” of storage to the transition between proprietary workstations and PCs in the 1980’s. But IDC analyst Rick Villars says this isn’t a zero-sum game. “This isn’t going to replace traditional IT,” he said. “Ninety-five percent of what people are developing and building in the storage industry today is irrelevant to what the cloud is building. You could take that as a negative, but it also translates into opportunity. These are new market spaces and new storage consumers that weren’t around five years ago.”

There’s been a lot of discussion lately about the role the cloud will play as the global economy softens. There is a difference of opinion between those who see a capital-strapped storage market as an even more conservative and risk-averse one and those who argue the opportunity to avoid capital expenditures will nudge traditional IT applications into the cloud. Still others point out the hurdles to cloud computing that remain, including network scalability and bandwidth constraints.

For example, when it comes to storage applications such as archiving, analyst reports from Forrester Research this year cited  latency in accessing off-site archived messages and searching them for e-discovery as major barriers to adoption for archiving software-as-a-service (SaaS) offerings.

Cloud computing “definitely exposes weaknesses in networking,” Villars said, but “the closest point to the end user is the cloud, if you want to distribute content to end users spread around the world.”

Other challenges include the growing pains major cloud infrastructures such as Amazon’s S3 have experienced over the last 18 months, and the potential risk of putting more enterprise data eggs in one service provider’s cloud data center basket. Villars points out, “I doubt Amazon has had more problems than a typical large enteprise, and they offer backup with geographic distribution for free.”

However, geographic distribution brings with it its own challenges, such as varying regulations among different countries. “There are regulatory problems with Europe,” Villars said. “Laws there say that if you have data on a European customer, yhou can’t move it out of Europe. If you want your cloud provider to spread copies between the U.S., Asia and Europe for global redundancy, that becomes an issue.”