software as a service

Emerging vendors take a SaaS approach to storage reporting

Two startups are taking a software-as-a-service (SaaS) approach to reporting on storage assets.

Storage Fusion Ltd, a UK company spun off by a private investment firm last year, claims to be reporting on storage environments of up to 60 PB, and signed a licensing agreement with GlassHouse Technologies last fall. According to managing director Graham Wood, the 15-person company is currently working with about 50 active customers, all with more than 50 TB, not counting “one-off” analytics done with some partners. The company was not able to provide an end user for an interview.

Storage Fusion’s product, Storage Resource Analysis (SRA), consists of a series of scripts which customers download and execute to collect data on EMC, Hitachi Data Systems, Hewlett-Packard, IBM, or NetApp arrays in their environment. The data generated by the scripts are then sent back to Storage Fusion’s data center, where Storage Fusion performs the analysis and provides results to the customer via the Web. According to Wood, if users get their data uploaded before 3 p.m., the analysis will be available the same day.

The company’s Web portal provides analysis according to gegraphical parameters, or the total resources, allocation and utilization at each data center location; a consumer view, which shows the hosts connected to the storage; a provider view, which normalizes the view of resources across heterogeneous storage providers into one report on the total storage environment; an environmental view, which provides energy consumption statistics according to published power and cooling specs from vendors; and an optional add-on business view, which translates storage capacity data into business-relevant statistics like dollars and cents. The reporting tool also looks for “exceptions” and provides error warnings and information on “orphan” reclaimable storage. The tool can decompose virtualization layers and supports thin provisioned arrays under its tiering tab.

Some customers, particularly large ones, might be wary of a third party peeking into their environment or sending data about their environment out of their data center. But Storage Fusion sales and operations director Peter White said “from a security perspective, our scripts are completely open — we hid nothing, and prior to running them, the user can look at them and see they’re just service log commands, the kind of command line utilities they execute all day.” The Web portal is also accessed via an SSL connection.

On the other side of the pond, and the other side of the customer-size spectrum, is Waltham, Mass.-based Aprigo, whose Ninja product has gotten several hundred free-version downloads since August. This first free version of the product collects file metadata regardless of hardware vendor on common file attributes such as name, type, size, and date modified. Aprigo compliles those attributes in a single view, and presnets them along with a cost calculator to show the dollar value of storing information on a yearly basis.

“It can be used for archiving or tiered storage business justification,” Aprigo CEO Gill Zimmerman said. Customers can also store up to 500 GB or 5 previous historical scans for trending reports. Aprigo is also working to put together a “community intelligence” report where users can compare themselves anonymously against other Aprigo customers.

Aprigo has a midmarket focus and doens’t use the term SRM because it’s reporting on file data rather than physical devices, according to Zimmerman. It’s working on a collector for other SaaS-based file systems like Google Docs. The company plans a Nov. 15 release that will also report on access control lists for file systems.

While some analysts have said the SRM space, which has seen its share of ups and downs, won’t mature until services and help for customers interpreting analytics results are more widely available, the SaaS or services-delivery model of SRM tools is not a new idea – Aptare has sold its backup and storage reporting tools to service providers for years; similarly, IBM’s Storage Enterprise Research Planner (SERP) storage resource management tools are deployed through IBM Global Services. CommVault began offering a SaaS-based backup reporting service in 2008; Dell has pledged a SaaS approach to services; and Continuity Software also offers a SaaS option for its disaster recovery change management tool.

IDC: Unstructured data will become the primary task for storage

According to a new IDC Enterprise Disk Storage Consumption Model report released this week, transaction-intensive applications are giving way as the main segment of enterprise data to an expanded range of apps as well as a tendency to create more copies of data and records for business analytics including data mining and e-Discovery.

The report estimates that unstructured data in traditional data centers will eclipse the growth of transaction-based data that until recently has been the bulk of enterprise data processing. While transactional data is still projected to grow at a compound annual growth rate of 21.8%, it’s far outpaced by a 61.7% CAGR predicted for unstructured data in traditional data centers.

“In the very near future, the management and organization of file-based information will become the primary task for many storage administrators in corporate datacenters,” the report reads. “And this shift will have a significant impact on how companies assess storage solutions in terms of systems’ performance, operational efficiency, and file services intelligence.”

The IDC report also builds on research first highlighted in an IDC blog last week concerning the cloud. According to the report, the sharpest growth in storage capacity will come from new organizations described as “content depots.” IDC estimates storage consumption from these organizations will grow at a compound annual growth rate of 91.8% through 2012. Examples of content depots  include the usual cloud suspects: Google, Amazon, Flickr, and YouTube.

These content depots have different IT requirements and infrastructures than traditional enterprise data centers. We’re seeing examples of these new  infrastructures pop up in the market, including systems with logical abstraction between the hardware and software elements; the use of commodity servers as a hardware basis for storage platforms; and the use of clustered file systems.

Some in the industry have compared this “serverization” of storage to the transition between proprietary workstations and PCs in the 1980’s. But IDC analyst Rick Villars says this isn’t a zero-sum game. “This isn’t going to replace traditional IT,” he said. “Ninety-five percent of what people are developing and building in the storage industry today is irrelevant to what the cloud is building. You could take that as a negative, but it also translates into opportunity. These are new market spaces and new storage consumers that weren’t around five years ago.”

There’s been a lot of discussion lately about the role the cloud will play as the global economy softens. There is a difference of opinion between those who see a capital-strapped storage market as an even more conservative and risk-averse one and those who argue the opportunity to avoid capital expenditures will nudge traditional IT applications into the cloud. Still others point out the hurdles to cloud computing that remain, including network scalability and bandwidth constraints.

For example, when it comes to storage applications such as archiving, analyst reports from Forrester Research this year cited  latency in accessing off-site archived messages and searching them for e-discovery as major barriers to adoption for archiving software-as-a-service (SaaS) offerings.

Cloud computing “definitely exposes weaknesses in networking,” Villars said, but “the closest point to the end user is the cloud, if you want to distribute content to end users spread around the world.”

Other challenges include the growing pains major cloud infrastructures such as Amazon’s S3 have experienced over the last 18 months, and the potential risk of putting more enterprise data eggs in one service provider’s cloud data center basket. Villars points out, “I doubt Amazon has had more problems than a typical large enteprise, and they offer backup with geographic distribution for free.”

However, geographic distribution brings with it its own challenges, such as varying regulations among different countries. “There are regulatory problems with Europe,” Villars said. “Laws there say that if you have data on a European customer, yhou can’t move it out of Europe. If you want your cloud provider to spread copies between the U.S., Asia and Europe for global redundancy, that becomes an issue.”

Bad week for cloud storage

Last week, we saw a good bit of rain falling on cloud storage’s parade. First there was another Amazon outage. Then it came to light that a cloud storage site called The Linkup (nee MediaMax) has completely failed, because of an apparent problem with data migration. At least that’s what it sounds like from their blog post about going out of business:

It was not possible to satisfactorily complete the move of files from MediaMax to The Linkup as we had expected, and as a result cannot offer a service that meets your expectations and our business requirements. This is a very disappointing outcome for us, and we know it has been a frustrating experience for many of our customers.

Maybe the owners of The Linkup could bounce back by taking Xdrive off AOL’s hands for the bargain price of $5 million and starting over.

Generally, I reserve judgment on the ultimate fate of cloud storage services. I know that online storage had a brief period of interest during the tech bubble but never went anywhere, and some believe this is more of the same. But I can be convinced, for now, that this time might be different. Small outfits such as The Linkup get trampled during the gold rush toward any new technology, and perhaps established service providers such as Amazon are going through growing pains. It’s still too early for these events to be anything other than a possible warning sign.

But things have sure looked ugly lately.

Amazon’s S3 crashes again; Web 2.0 goes Boom!

The biggest difference between the last time S3 crashed and this time, in my observation, is that there was a much, much bigger chain reaction this time around. Last time, I knew of only a few companies using S3, like photo hosting site SmugMug, and startups that offer online backup services using their own interfaces on the front-end and Amazon’s hardware infrastructure on the back-end.

This time, not only were those types of Web 2.0 companies affected, but much bigger fish also felt the sting: no less than Web 2.0 microblogging phenom Twitter and some iPhone applications crashed along with S3.

The last Amazon outage was attributed to “growing pains” as the service gained popularity. I’d imagine adding popular apps like Twitter and the iPhone constituted another wave of painful growth. This is a new medium, and users of very new storage media accept some level of risk. But two major outages in six months is obviously raising some questions.

“Skype has crashed and stopped responding, Twitter, Tumblr and other major websites are barely working, most aren’t displaying images, widgets or static material that was outsourced to Amazon S3 services,” reported blogger LinkFog as the outage occurred. “It’s kinda funny how this goes against the very nature of the web, in each networks are interconnected in several ways to ensure that a major breakdown won’t happen.”

Others, like a blogger at Web Worker Daily, were not happy with Amazon’s SLAs:

Amazon does offer an SLA for the S3 service, guaranteeing 99.9% uptime or part of your money back. With .1% of a month being around 45 minutes, that means they owe people money. The requirements for claiming a refund, though, are onerous enough that no one except large users will bother (hey, Amazon, how about an automatic refund when you know your servers are down?).

Recent reports suggest that this is actually what will happen.

Clearly it’s not a major disaster for people not to be able to Twitter for a few hours. But when it comes to things like the backup services attached to S3, it might be time for people to rethink whether one cloud back-end is the same as another. Amazon’s appeal is that it’s cheap and relatively unrestricted for Web developers–but I hope the backup companies basing their hardware infrastructure on S3 at least inform their end users what the back end is, so they can make an informed decision about service provider reliability.

Yahoo, I hardly knew ye

I’ve watched the story unfold about Microsoft and Yahoo, but from a removed perspective because it has little to do with the storage industry and when it comes to most things Web-based and search or email related, I’m a Google user. Still, it’s been a good story to sit back with some popcorn and watch develop.

Recently, though, it’s hit home a little more for me. First, I saw that the New York Times/AP reported that the co-founders of Flickr, a photo sharing service bought by Yahoo in 2005, have left the company. Then I found out that the founder of Del.icio.us is also leaving Yahoo–which was the first time I even realized Del.icio.us was a Yahoo property.

Now I wonder two things–1) How many other staples of my Web 2.o life are part of Yahoo and I didn’t know it? (One helpful resource for this question: TechCrunch has posted a big table to keep track of the Yahoo exodus). 2) What’s going to happen to them?

It’s as close as I’ll ever come to the experience my enterprise storage audience must have regularly when dealing with the effects of mergers and acquisitions. Anxiety frequently accompanies these events, causing people to wonder how the user experience will change with the product, how support might change, how well might the company keep up with features…

It’s not like products can’t survive without their original innovators, and for the moment, Yahoo does still exist as we know it (though there’s speculation that’s not for long). But I have seen in the storage industry how innovation diminishes after the guys who first built the machine in the garage leave the company, innovation diminishes, and the company itself is more likely to move on to the next shiny object.

That’s what I’m afraid will happen now to Flickr and Del.icio.us, and then I’d have to face another nightmare commong among enterprise folks–how to get my 8,000-plus photos and 2,000-plus bookmarks migrated over to another service.

Symantec’s SwapDrive: $500 a year for 2 GB?

I wasn’t convinced at first when an alert blog reader flagged an error in my previous posts about Symantec and SwapDrive: a comment from  “kataar” pointed out that yearly, SwapDrive actually charges $500 (five hundred) for 2 GB, not $50 (fifty).

That couldn’t possibly be right, I thought. I clicked the site, saw the same price list, read down the column for individual users–ah! 2 GB, $50. I was all ready to post a reply when I went back and checked it one more time, just to be sure. That’s when I noticed “Monthly” over the cost I was looking at. Under “Yearly” was, indeed, $500. For 2 GB of storage per year. For multi-user plans of up to 10 GB, the yearly cost is $2,800.

My bad. And thanks to kataar!

EMC, of course, is having a field day with this. Even comparing a relatively modest price of $49.50 a year (you’ll notice Mark Twomey made the same mistake I did), they are only too happy point out that you can get 2 GB of storage free from Mozy (I’ll let the irony of EMC gloating about another vendor’s pricing pass for now). Meanwhile, you can get up to 5 GB free from Windows SkyDrive, GMail will give you a 2 GB inbox for free, and Carbonite will let you back up unlimited capacity to its cloud for $49.95 per year.

I’ve heard of some of the older data hosting services, like certain specialized deals with Iron Mountain, costing in the neighborhood of SwapDrive’s quoted price, but I haven’t heard of too many in the consumer/SOHO/SMB space charging on that scale.

When I asked Symantec about the pricing, this was the response: “SwapDrive’s current online pricing will keep pace with the market and the value derived. Our service is more robust and redundant than many others offered in the market today.” The spokesperson added that 2 GB of online storage comes included with Norton 360 for an MSRP of $79.99.

I’d really like to learn more about exactly what makes SwapDrive hundreds of dollars more robust and redundant per year.  And what makes it worth $500 standalone but worth some percentage of $80 with Norton 360? That seems like a big swing to me.

Vendors are buddying up in the cloud

So I come back from a day off and what do I find? IBM and Google, Sun and Amazon reportedly pairing up in the cloud.

I have to say that cloud computing has made the growing IBM/EMC rivalry that much more interesting. EMC threw one of the first punches with the rollout of Fortress and acquisition of Pi–it seems EMC will probably stick to building its own infrastructure rather than partnering. But then IBM went for a partnership with one of the other most recognizable brand names in the world (aside from its own) in Google, which consumers are already comfortable using in the real world. Meanwhile, PiWorx remains in stealth. It will be interesting to see where the next leapfrog move comes from.

Incidentally, how long before Sun acquires Zmanda? They’ve already acquired MySQL, for which Zmanda offers open-source backup and now they’re buddying up with Amazon, for which Zmanda offers an interface (Amazon still requires you to roll your own GUI or get one from a partner). It could link Sun’s open storage products–via open source software!–to the cloud with Amazon. It would be just one big happy open-source conflagration…I’m still watching for it.

Meanwhile, the other tizzy lighting up my Google Reader is over the lack of a deal between Microsoft and Yahoo. Rob Enderle has an interesting post up on Google’s role in that situation. I’m wondering, as IT and cloud vendors keep pairing up, if we shouldn’t be looking for familiar faces among those next in line to be Yahoo’s dance partner.

IBM starts showing its cloud arsenal

Hardly a day goes by without a new storage service rolling out. On Monday, it was IBM’s turn to launch two storage services as part of its portfolio of services for midsized customers - organizations with 100 to 400 employees and a handful of Windows servers.

The interesting thing about IBM’s Remote Data Protection Express and E-Mail Management Express offerings is they are the first new services IBM has launched from its Arsenal Digital Solutions acquisition in December. The Arsenal brand is gone but the remote data protection service is the same one that Arsenal offered, even still including data deduplication from EMC’s Avamar.

The email archiving service is something Arsenal was working on at the time of the acquisition, pushed to market quicker with IBM technology.

“Email management is a new offering [for Arsenal],” said Arsenal alum Brian Reagan, now an IBM Information Protection Services executive. “Under the covers we’ve started to adopt and integrate more IBM offerings.”

The email service covers Exchange and Lotus Notes, and Reagan said database and unstructured data archiving services are in the works.

Since Arsenal was into managed storage services long before anybody talked of clouds and SaaS (software as a service), I asked Reagan if he thought IBM should have kept the Arsenal brand. He said Arsenal did have a name for itself and partnerships with AT&T and other large providers, but IBM is a pretty recognizable name too.

“We get the benefit of IBM’s brand,” he said. “As Arsenal, we would have to spend twice as much money to get the attention of customers because they didn’t know who we were.” Reagan pointed out that IBM ran advertisements for its services during the PGA Masters broadcast. “That was something Arsenal could only dream of,” Reagan added.

Then again, you don’t have to be IBM to attract attention for storage services these days. Everybody’s getting into the cloud act, and Reagan says the glut of offerings have served mostly to confuse customers.

“There’s a tremendous amount of customer interest,” he said. “The downside is, it’s created confusion. Some of the really low end players that only service the consumer end of the market have clouded the picture. They’ve confused people wondering what the difference is between low end service that’s priced too good to be true and real resilient service.”

In other words, it will take awhile before enough sun shines on cloud computing so we can really know what to expect.

Blog dialogue: Online vs. traditional backup

I was very happy to see one of my regular blog-stops, Anil Gupta’s Network Storage, pick up on a recent post I wrote–the one about HP’s new online storage services.

In his response post, Gupta picks up on this graf in particular:

Like most online storage offerings to date, this offering is small in scale and limited in its features when compared with on-premise products. Most analysts and vendors say online storage will be limited by bandwidth constraints and security concerns to the low end of the market, with most services on the market looking a lot like HP Upline.

And responds:

there is nothing unique in most Online Backup Services that couldn’t be in traditional backup for laptop/desktop. At least traditional backup also come with peace of mind that all backups are stored on company’s own infrastructure. In last few years, I tried over a dozen online backup services in addition to putting up with traditional backup clients for laptop/desktop and I don’t see much difference among the two.

IMO, most online backup services are just taking existing on-premise backup strategy for laptops/desktops and repackaging it to run backups to somebody else’s infrastructure instead of your own.

I see what he’s saying, but in my opinion Gupta probably has “too much” experience with backup clients to necessarily see things from the SMB customer’s point of view. For him, installing a backup client isn’t a big deal–for some, it might be enough of a reason to let somebody else deal with it. Or at least, backup SaaS vendors are hoping so.

A storage reporter’s shameful secret comes to an end

I feel the need to make a confession here. Up until yesterday, despite spending a generous portion of my waking hours covering data backup, disaster recovery and data protection, I myself did not have a backup plan.

I do digital photography in my spare time, and creative writing outside work, and I’ve been a digital music addict since the advent of Napster. So I have about 100 GB on two IDE drives inside a Windows XP machine custom-built for me by a highly geeky friend. And it’s just been sitting there, waiting to be snatched away into the ether.

Then another friend of mine told me about how his MacBook hard drive crashed. On his birthday. While he also had the flu.

He told me how his entire visual design portfolio, an important part of his resume for the business he’s in, has been lost, along with all of his digital photographs, many of which he didn’t have posted on Flickr or stored anywhere else.

He went on to tell me that his costs for trying to recover the data from the drive are going to run him upwards of $2,000–if he’s lucky. It could be cheaper, but that would mean less of his data has been recovered, and so now he finds himself in the position of hoping he’ll have to spend more money.

It’s a bittersweet subject for him that so many people he knows, myself included, have credited his experience with finally getting them off their butts and backing up. But that’s the reality.

I ended up going with the 500 GB Western Digital MyBook, because that’s what my friend also ordered once he learned his lesson the hard way, and he’s far more technical than me, so I trust his judgment. The MyBook came with Memeo’s AutoBackup and AutoSync software, of which I’m only using the former. It also came with a bunch of Google software including Google Desktop, which I found rather odd.

Having covered data storage for the enterprise, I’ve had a chuckle whenever I’ve checked on the initial backup job’s progress. Granted, it’s got a QoS feature that cedes system resources to the PC, but let’s just say I’m not seeing the kind of data transfer rates with this thing I’m used to hearing about. It’s been funny, after being immersed in systems that perform at 8 Gbit or 10 Gbit for a few years, to watch my little PC poke along at what seems like 1 MB/hr, if that.

But still. At least I have a backup. Finally. And I can finally rid my closet of that skeleton.

Now my issue becomes off-site disaster recovery. It’s far more likely that my hard drive(s) will crash than that my house will be napalmed or something (knock on wood), but no sooner had I told Tory that he could stop bugging me about backup, than he started bugging me about taking the drive to my office once the data transfer is done.

But the AutoBackup software, like so many low-end and consumer backup offerings, is set to automatically backup changed files, and what I told Tory was, I like having a low RPO over here. And I made that napalm comment, I’ll admit (I can just feel karma coming to get me). So I’m thinking about some kind of backup SaaS for off-site DR, but capacity with those services is at a much higher premium than it is in 3.5 inch external SATA. And so you know what that means…data classification!

I may be poking along at 1 MB/hr, but it all feels like a slow-motion, small-scale version of the issues I cover every day. It’s interesting to see firsthand how ”Digital Life ™” is, in fact, blurring the boundaries between home and business computing.