Storage

Too much data center traffic throws off MozyHome clients

Mozy says a high volume of traffic is to blame for a backup monitoring glitch flagged by a MozyHome user on his blog, but the online backup service says it has not lost any customer data.

Dan Frith, a technical consultant in Australia, wrote on his blog Penguin Punk earlier this week that he was seeing some quirkiness in Mozy’s monitoring interface. He’s been using MozyHome to backup an iMac with approximately 32 GB of data, he wrote, but last weekend the Mozy interface was only showing him 10.3 GB backed up at its data center.

Frith’s posts (which include screenshots) also detail his interactions with Mozy support to get the problem sorted out, including the full text of an email Mozy sent him saying that if he initiates a full backup again, Mozy will “re-associate” the full data set with his account.

Frith indicated he’s unimpressed with the workaround Mozy has suggested. “The point is that if I needed to recover data from Mozy today I would only be able to get back 10.x GB,” he wrote. “That seems uncool. Very uncool.”

Mozy responded to my request for comment with a statement through a spokesperson:

recently, we experienced a high volume of data center traffic that prevented the Mozy client from adequately identifying files that were previously backed up. As a result, Mozy is sending third or fourth copies of the same files to our data centers.

Our development team is working right now to address the issue and expects to have this fixed soon. We want our customers to know, however, that we have not lost any of their information.

This is not the first complaint to surface about Mozy recently. In September, backup expert W. Curtis Preston also blogged about how his Mozy agent didn’t notify him it wasn’t backing up data. Last year, commercial users of MozyPro also said they were frustrated by long restore times with the service.

HP buys 3Com, not Brocade

Although Hewlett-Packard spent $2.7 billion to buy 3Com Wednesday largely to make it more competitive with Cisco on the Ethernet switching front, the deal will also have implications for Cisco’s main storage competitor Brocade.

First, the deal means HP won’t be buying Brocade – at least not any time soon. HP was considered the most likely company to buy Brocade after word leaked that Brocade was looking for a buyer last month. It appears that HP did consider it – HP executive VP Dave Donatelli said on a webcast explaining the 3Com acquisition that HP looked at all networking options – but decided 3Com’s Ethernet switches and routers were a better fit than the products that Brocade picked up from Foundry.

The 3Com acquisition also means HP won’t follow the lead of IBM and Dell and sign an OEM deal with Brocade for its Ethernet switches. With 3Com’s products and its own ProCurve platform, HP should have enough to fill out its Ethernet lineup.

The deal won’t impact Brocade’s core business – selling Fibre Channel switches. With Cisco and Brocade as its only options, HP will likely continue to lean heavily on Brocade for storage connectivity.

Still, the HP-3Com deal is seen as bad news for Brocade. Several Wall Street analysts downgraded its stock price today, and its shares dropped more than a dollar in early trading from Wednesday’s closing price of $9.25.

While talk of Brocade getting acquired has diminished, Wedbush Securities analyst Kaushik Roy raised the possibility that networking vendor Juniper might want Brocade to create “an even more formidable competitor to Cisco.”

“We think that neither Dell nor IBM would be interested in buying Brocade due to Brocade’s OEM model,” Roy wrote today in a note to clients. “Any purchase by one of the server vendors would lead to loss of revenue streams from the other server OEM vendors. We, however, think that Brocade might be a good acquisition target for Juniper.”

HP looks to entice SMBs, hypes Hyper-V bundles

HP made some storage updates today as part of a larger announcement aimed at SMBs looking to cut costs, including new Hyper-V bundles.

Storage-related updates include:

• New application-integrated snapshot option for LeftHand iSCSI SANs. LeftHand already had application-integrated snapshots that supported VSS for snapping the LeftHand SAN, but not with the ability to run inside and quiesce an application. This update fills that gap; competitors like Dell EqualLogic and NetApp already offer this capability.
• New NAS interface for D2D data deduplication products. SMBs no longer need to license virtual tape drives to use HP’s low-end data deduplication product.
• New DAT 320 tape drive. SMBs still using tape, on the other hand, have the option of doubling the capacity on DAT tape drives to 320GB. HP claims the new drives also offer up to a 75% performance increase with a 50% lower power draw. Tape, especially for SMBs, is frequently declared dead, but there are still pockets of tape use in this market, particularly in remote and branch offices.
• Six new HP Virtualization Smart Bundles for Microsoft Hyper-V. Full specs on the bundles, which range from an entry-level tower server form factor to rackmount servers with networked storage, are available at HP’s website. These bundles are similar to the ones HP previously rolled out this year for VMware environments.

LeftHand product marketing manager Chris McCall says the Hyper-V bundles are not a counter strike against the new joint venture between VMware, Cisco and EMC. He says HP is supporting both hypervisors, even though VMware is deeply aligned wiht HP HP competitors Cisco and EMC. ”We’ve done bundles for VMware already because we’ve targeted market size — we did VMware first because they’re the market leader,” he said. “If Microsoft was, we would’ve done Hyper-V first. There’s nothing to read into there.”

Emerging vendors take a SaaS approach to storage reporting

Two startups are taking a software-as-a-service (SaaS) approach to reporting on storage assets.

Storage Fusion Ltd, a UK company spun off by a private investment firm last year, claims to be reporting on storage environments of up to 60 PB, and signed a licensing agreement with GlassHouse Technologies last fall. According to managing director Graham Wood, the 15-person company is currently working with about 50 active customers, all with more than 50 TB, not counting “one-off” analytics done with some partners. The company was not able to provide an end user for an interview.

Storage Fusion’s product, Storage Resource Analysis (SRA), consists of a series of scripts which customers download and execute to collect data on EMC, Hitachi Data Systems, Hewlett-Packard, IBM, or NetApp arrays in their environment. The data generated by the scripts are then sent back to Storage Fusion’s data center, where Storage Fusion performs the analysis and provides results to the customer via the Web. According to Wood, if users get their data uploaded before 3 p.m., the analysis will be available the same day.

The company’s Web portal provides analysis according to gegraphical parameters, or the total resources, allocation and utilization at each data center location; a consumer view, which shows the hosts connected to the storage; a provider view, which normalizes the view of resources across heterogeneous storage providers into one report on the total storage environment; an environmental view, which provides energy consumption statistics according to published power and cooling specs from vendors; and an optional add-on business view, which translates storage capacity data into business-relevant statistics like dollars and cents. The reporting tool also looks for “exceptions” and provides error warnings and information on “orphan” reclaimable storage. The tool can decompose virtualization layers and supports thin provisioned arrays under its tiering tab.

Some customers, particularly large ones, might be wary of a third party peeking into their environment or sending data about their environment out of their data center. But Storage Fusion sales and operations director Peter White said “from a security perspective, our scripts are completely open — we hid nothing, and prior to running them, the user can look at them and see they’re just service log commands, the kind of command line utilities they execute all day.” The Web portal is also accessed via an SSL connection.

On the other side of the pond, and the other side of the customer-size spectrum, is Waltham, Mass.-based Aprigo, whose Ninja product has gotten several hundred free-version downloads since August. This first free version of the product collects file metadata regardless of hardware vendor on common file attributes such as name, type, size, and date modified. Aprigo compliles those attributes in a single view, and presnets them along with a cost calculator to show the dollar value of storing information on a yearly basis.

“It can be used for archiving or tiered storage business justification,” Aprigo CEO Gill Zimmerman said. Customers can also store up to 500 GB or 5 previous historical scans for trending reports. Aprigo is also working to put together a “community intelligence” report where users can compare themselves anonymously against other Aprigo customers.

Aprigo has a midmarket focus and doens’t use the term SRM because it’s reporting on file data rather than physical devices, according to Zimmerman. It’s working on a collector for other SaaS-based file systems like Google Docs. The company plans a Nov. 15 release that will also report on access control lists for file systems.

While some analysts have said the SRM space, which has seen its share of ups and downs, won’t mature until services and help for customers interpreting analytics results are more widely available, the SaaS or services-delivery model of SRM tools is not a new idea – Aptare has sold its backup and storage reporting tools to service providers for years; similarly, IBM’s Storage Enterprise Research Planner (SERP) storage resource management tools are deployed through IBM Global Services. CommVault began offering a SaaS-based backup reporting service in 2008; Dell has pledged a SaaS approach to services; and Continuity Software also offers a SaaS option for its disaster recovery change management tool.

Emulex leads its convergence strategy with Ethernet

Emulex used its analyst day today to officially roll out its OneConnect Universal Converged Network Adapters (UCNAs) and underscore its strategy of taking a 10-Gigabit Ethernet path to converged Fibre Channel and Ethernet networks.

Emulex said the OneConnect adapters it first talked about in February are available for partners to ship, and IBM has agreed to OEM its 10-GigE NIC and 16 Gbps Fibre Channel HBAs with the Power Systems server platform.

Emulex is taking a different path than its main rival QLogic, which already has several partners selling its single-chip 8100 Series CNAs. Emulex is releasing its 10-GigE adapter first with TCP/IP and TCP Chimney support, and hopes to deploy a pay-as-you-go strategy where customers will later upgrade with iSCSI and FCoE connectivity. Emulex gets its 10-GigE silicon through an OEM deal with ServerEngines.

Emulex is counting on 10-GigE and Intel’s Nehalem servers driving convergence, with storage connectivity to follow.

“We’ve taken an Ethernet approach rather than a storage-centric approach,” Emulex VP of corporate marketing Shaun Walsh told StorageSoup. He says the Emulex approach lets customers pay for only 10-GigE first, instead of having to pay for FC connectivity they might not use yet.

“Every NIC card has the potential to be an FCoE card,” Emulex CEO Jim McCluney said during his analyst day presentation.

Emulex customer Lars Linden, SVP of data center services for Royal Bank of Scotland, spoke at the analyst day to express his eagerness for a converged network. Linden said convergence will eventually help him simplify management, reduce cables and increase utilization, adding that it can’t arrive soon enough for him. He says he RBS spends about $500,000 a year on cabling.

“I have people on staff who do nothing but cabling all day long,” he said. “They’re very clever people and I would like to have them do higher value activity, but this is table stakes for running a data center. As soon as there is a commercially available set of capabilities and technologies supporting convergence, there will be a rapid adoption.”

Linden compared the eventual move to consolidated networks to virtualization as a “game changer” technology.

Nobody expects 16-gig FC any time soon, despite Emulex’s touted design win. Emulex executives acknowledged the market moved to 8-gig FC much slower than it went from 2-gig to 4-gig FC so there’s no hurry to push out 16-gig products.

“This is a future announcement,” McCluney said. “We’re not going to see any 16-gig revenue for quite some time.”

The suspicion here is that Emulex and IBM announced the 16-gig FC design win to end speculation that QLogic might replace Emulex FC HBAs on the Power Series after securing a CNA OEM win with IBM last week. Emulex is IBM’s exclusive partner for 4-gig and 8-gig HBAs on the Power Series.

“We believe investors may have questions regarding QLogic’s recent announcement that its FCoE CNAs had been qualified within IBM’s System p (Unix) server platforms given that Emulex has long been the sole-source provider of FC HBAs into these IBM server platforms,” Stifel Nicolaus Equity Research analyst Aaron Rakers wrote in a note to clients after the Emulex analyst day.

Industry bloggers debate dedupe to tape

It just wouldn’t be the storage industry if there weren’t technical debates popping up on a daily basis.

One that caught my eye today is an ongoing conversation between some storage bloggers about data deduplication to tape, and whether or not it’s a crazy idea. Or, more accurately, whether it’s “good crazy” or “bad crazy.”

Backup expert W. Curtis Preston got things started with a blog written after he visited CommVault’s headquarters in Oceanport, N.J., and discussed the concept of CommVault’s data deduplication to tape feature added in Simpana 8. “Dedupe to tape is definitely crazy.  But is it crazy good or crazy bad?” Preston wrote.

Everyone (including the CommVault folks) agrees that no one would want to do any significant portion of their restores from deduped tape.  But I also agree that if I typically do all my restores from within the last 30 days, and someone asks me for a 31 day-old file, it’s generally going to be the type of restore where the fact that it might take several minutes to complete is not going to be a huge deal.  (In the case that you did need to do a large restore from a deduped tape set, you could actually bring it back in to disk in its entirety before you initiate the restore.)

Now here’s the business case. Anyone who has done consulting in this business for a while has met the customer where everyone knows that 99% of the restores come from the last 30-60 days — and yet they keep their backups for 1-7 years.  What a waste of resources.  CommVault is saying, “Hey.  If you’re going to do that, at least dedupe the tapes.”  They showed me two business cases from two customers that doing this was saving them over $500K per year in their Iron Mountain bill.

Curtis made some declarative statements in that blog post, and when that happens you can expect someone in the storage blogosphere to write a post in opposition. EMC Networker data backup consultant Preston de Guise did the honors this time, with a reponse titled “Dedupe to tape is “crazy bad” if the architecture is crazy.”

Yes, it’s undoubtedly the case that the CommVault approach will reduce the amount of data stored on tape, which will result in some cost savings. However, penny pinching in backup environments has a tendency to result in recovery impacts – often significant recovery impacts. For example, NetBackup gives “media savings” by not enforcing dependencies. Yes, this can result in in saving money here and there on media, but can result in being unable to do complete filesystem recoveries approaching the end of a total retention period, which is plain dumb.

The CommVault approach while saving some money on tape will significantly expand recovery times (or require large cache areas and still take a lot of recovery time). Saving money is good. Wasting a little time during longer-term recoveries is likely to be perceived as being OK – until there’s a pressing need. Wasting a lot of time during longer-term recoveries is rarely going to be perceived as being OK.

An IT admin/blogger writing at Standalone Sysadmin picked up on de Guise’s post and had this to say:

My problem with this is tape failure. If one of the 50 individual backup tapes fails, it’s no problem. Sure, you lose that particular arrangement of the data, but it’s not that big of an issue. Unfortunate, sure, but not tragic. If you lose the 1 tape that contains the deduplicated data, though, then you immediately have a Bad Day(tm).

Essentially, you are betting on one tape not failing over the course of (in the argument of Mr Preston) 7+ years. And if something does happen in that 7 years, whether it’s degaussing, loss, theft, fire, water, or aliens, you don’t lose one backup set. You lose every backup that referenced that set of data.

So I would, if I could afford one, buy a deduplicated storage array in a heartbeat for my backup needs. But I would not trust a deduplcated archival system at all. The odds of loss are too great, and it’s not worth the savings. I’d rather cut the frequency of my backups than save money by making my archives co-dependent.

Of course, another user we talked to around the launch of Simpana 8 felt differently:

The global deduplication with Simpana 8 also extends to tape, making it the first product of its kind to allow for writes to physical tape libraries without requiring reinflation of deduplicated data. “That’s very appealing,” said Paul Spotts, system engineer for Geisinger Health, a network of hospitals and clinics in central Pennsylvania. “We added a VTL [virtual tape library] because we were running out of capacity in our physical tape libraries, but we lease the VTL, so we’re only allowed to grow so much per quarter.”

What say the rest of you?

LiveDrive looks to hop across the pond with online data backup

U.K.-based LiveDrive, a competitor to consumer online backup services like MozyHome and Carbonite, is getting U.S. distribution thanks to a new partnership with LifeBoat Distribution.

Online marketing manager Jamie Brown says LiveDrive has 300,000 unique accounts worldwide, 120,000 of them already located in the U.S. LiveDrive creates a network drive that shows up on users’ PCs. Any files sent to that L:\ drive will be backed up to LiveDrive’s cloud; data can be stored there for safekeeping or users can use LiveDrive to keep the L:\ drive synced and share data among multiple machines. Users can also access their data through LiveDrive’s Web portal, which also offers mini-applications that allow users to edit or play back photos and video.

The company has data center infrastructure in the United States through collocation, but currently all users access data through load balancers in the UK. Brown said there are plans to expand the US infrastructure organically, but won’t rush, saying currently users aren’t experiencing performance issues with the way the infrastructure is set up.

After a year, though, Brown said LiveDrive hopes to have an office in the US within 12 months, and may also add a business-level service to compete with services like MozyPro and i365’s EVault Small Business Edition. It currently does not offer service level agreements or geographic redundancy for consumer users.

Despite its claims about its client base, LiveDrive was unable to provide a public customer reference before the announcement this morning.

Enterprise Strategy Group analyst Lauren Whitehouse said this is becoming an increasingly tough space for new players to differentiate themselves in. “Mozy has more than a million customers, and for Symantec’s SwapDrive the number’s even greater,” she said. “LiveDrive has plenty of formidable competition.”

One factor that might hurt LiveDrive, at least in the beginning, is the fact that data must currently be accessed through the U.K. “Anyone who has discomfort sending data out to the cloud might have more discomfort knowing there’s a geographic distance there,” Whitehouse said. Even if performance isn’t bad, “there could be ramifications if there’s a dispute.”

As for the differentiation of being able to manipulate content within the cloud, Whitehouse said LiveDrive will also face competition from players like Memeo and Ricoh’s Quanp, to say nothing of photo-sharing sites like Flickr and Photobucket, boith of which offer small photo-editing software suites with their services. “It’s somewhat of a Wild West situation right now ith different companies trying to do a ‘land grab’, capturing customers and then building from there,” she said, including LiveDrive in that mix.

QLogic ’swipes’ another FCoE win

Another piece of the Fibre Channel over Ethernet (FCoE) puzzle was put in place today when IBM said it will ship QLogic’s 8142 converged network adapters (CNAs) inside the Power Systems server platform for native FCoE connectivity.

IBM’s p Series servers run Unix and Linux operating systems. QLogic reps are crowing over the design win because it comes in its biggest rival’s backyard. IBM ships Emulex Fibre Channel HBAs exclusively with its p series, but has gone to its rival for FCoE adapters. Satish Lakshmanan, QLogic’s director of product marketing for host solutions, says QLogic’s is the only CNA that will ship with the IBM p series.

The 8142 is part of QLogic’s 8100 Series single-chip CNA platform that handles 10-Gigabit Ethernet traffic and has an integrated FCoE offload engine.

“This gives them a foot in the door in the p series,” Enterprise Strategy Group analyst Bob Laliberte said of QLogic.

Laliberte says the long-time FC HBA rivals have taken different paths in the early days of converged networking, with QLogic racking FCoE design wins while Emulex picks up 10-Gigabit Ethernet wins with its universal converged network adapters (UCNAs).

“QLogic was first to market with a single-chip architecture for FCoE that’s dramatically lower in power consumption, footprint and heat,” Laliberte said. “Being first out of the gate gives them a chance to get design wins in FCoE. Emulex is going in a different direction. You see Emulex getting wins for 10-gigE and down the road partners have an option to turn on FCoE.”

“This isn’t the only customer we’ve swiped away,” Lakshmanan said. “There’s more that will be coming.”

NetApp said in August that it would rebrand QLogic’s 8152 CNA as a built-in adapter for its FAS storage arrays. NetApp also qualified Brocade’s 1020 CNA but not Emulex’s yet.

IBM also sells QLogic’s 8100 CNAs on its System x and BladeCenter servers, and EMC qualified QLogic’s CNA on its Symmetrix, Clariion and Celerra NS storage platforms.

These are still early days for FCoE and converged adapters, though. Widspread adoption of FCoE on servers isn’t expected before 2011 and it will likely take years after that for it to show up in any volume on the storage side.

CommVault fires back at EMC’s Slootman

Former Data Domain CEO Frank Slootman, now president of EMC’s data backup and recovery division, sat down for a Q&A with SearchDataBackup.com that’s been getting some attention from the industry, particularly other deduplication competitors.

Among those competitors, one with a contentious relationship with EMC/Data Domain is former partner CommVault, with whom Data Domain had a messy breakup after CommVault introduced its own deduplication with Simpana 8.

Here’s what Slootman had to say about them:

SearchDataBackup: Will you continue to work closely with Symantec Corp.’s OpenStorage (OST) API now that you’re EMC?

Slootman: Yes. I’m not throwing my partners under the bus. We’ll compete, but we’re all competitors and partners these days. We won’t screw them. We’ll screw other companies, like CommVault. We {Data Domain] treated them as a good partner and they came after us.

In an email to Storage Soup this week, CommVault vice president of marketing and business development Dave West had this response:

As I said back in June, I applaud Frank and Data Domain’s ability to create momentum for deduplication and a tremendous return for its shareholders. In the Dave Raffo piece, Frank calls out CommVault simply because we’re giving them a run for their money. Simpana, with built-in dedupe, works really well, and we are winning business. Now, I find it ludicrous to suggest a product vision that forces a customer to deploy 3 or more disparate products to achieve basic data protection. (Pile on more products for replication, encryption, archive and SRM).  At the end of the day, customers want less complexity, improved operational efficiency and ultimately, to spend less money. That means fewer, not more solutions. Less hardware and smarter software. EMC’s product portfolio is both complicated and costly for customers, so buyer beware. Also, in our opinion, this interview should raise some serious flags among the thousands of already nervous NetWorker customers out there looking for reassurance in the wake of the Data Domain acquisition.

I asked West to elaborate on the “red flags” about NetWorker, and he pointed to this statement by Slootman in another part of the interview:

SearchDataBackup: If Avamar is the future of data backup software, where does that leave NetWorker?

Slootman: Well, Avamar is augmenting NetWorker in a lot of places. People are moving a good part of their workload to Avamar, but not all. They’re still running applications like big, fat databases on traditional backup software. NetWorker can support conventional backup on tape and mixed media and people can integrate it with Data Domain.

“Former EMC customers are telling us that there is no real investment or innovation going into the Networker product and they’re tired of it,” West added.

This dedupe feud will get really interesting if CommVault partner Dell Inc. starts selling Data Domain, which is a likely scenario because Dell sells much of EMC’s storage products. CommVault’s Simpana is currently a big piece of Dell’s deduplication strategy.

Google DRAM study turns conventional wisdom on its head…again

Remember the research paper Google made a splash with two years ago on disk drive failure rates? The one that showed that most failed drives didn’t raise significant SMART flags, failed to find a correlation between temperature and utilizaation with failure rates, and instead established that failure rates are more correlated to drive manufacturer, model and age?

Well, there’s now a DRAM equivalent — and it doesn’t paint a much prettier picture than the one on hard drive failures.

According to a new paper, “DRAM Errors in the Wild: A Large-Scale Field Study“, engineers from Google and the University of Toronto found that once again, failure rates and patterns did not match the received wisdom in the industry about how Dual Inline Memory Modules (DIMMs) behave. According to the paper:

We find that DRAM error behavior in the field differs in many key aspects from commonly held assumptions. For example, we observe DRAM error rates that are orders of magnitude higher than previously reported, with 25,000 to 70,000 errors per billion device hours per Mbit and more than 8% of DIMMs affected by errors per year. We provide strong evidence that memory errors are dominated by hard errors, rather than soft errors, which previous work suspects to be the dominant error mode. We find that temperature, known to strongly impact DIMM error rates in lab conditions, has a surprisingly small effect on error behavior in the field, when taking all other factors into account. Finally, unlike commonly feared, we don’t observe any indication that newer generations of DIMMs have worse error behavior.

As with the disk drive study, temperature also doesn’t play a huge role in DRAM failures. Here, vendor and model didn’t make as much difference as in the disk drive study.

However, the study showed errors were more highly dependent on motherboard design than previously thought. And contrary to conventional wisdom about DRAM, more failures were hardware than software-based. According to an article analyzing the paper by Data Mobility Group’s Robin Harris,

This means that some popular [motherboards] have poor EMI hygiene. Route a memory trace too close to noisy component or shirk on grounding layers and instant error problems…For all platforms they found that 20% of the machines with errors make up more than 90% of all observed errors on that platform. There be lemons out there!

These two reports raise one common question, according to Harris — why didn’t we know about these things before? As he put it, “Big system vendors have scads of data on disk drives, DRAM, network adapters, OS and filesystem based on mortality and tech support calls, but do they share this with the consuming public? Nothing to see here folks, just move along.”