Previous version of SANsymphony-V targeted the midmarket. With version 9, DataCore is going after large data centers, companies looking to build private cloud,s and cloud service providers with private, public or hybrid cloud offerings.

“We are trying to take it up a higher level,” DataCore CEO George Teixeira said. “We have automated tasks to make it simple, so you don’t have to focus on the details. Most of the commands and features have been made more adaptive.”

SANsymphony-V, which DataCore bills as a storage hypervisor, allows data on disk, solid state drives (SSDs), and Google cloud storage to be managed as a single pool. Auto-tiering can be applied so that administrators can put higher performing applications in memory, while archiving data into the cloud, Teixeira said.

DataCore also automated its N+1 scaling feature, allowing administrators to scale capacity and processors by adding one node at a time. The extra node can take over if any node in the cluster is lost.

Snapshots of multiple drives now can be done with a single click, and one-to-many bidirectional replication has been automated. Load balancing among multiple drives has also been automated.

DataCore is also adding reporting for chargeback and a DataCore Cloud Service Provider Program that offers new licensing options allowing CSPs to license the storage virtualization at a fixed monthly, per-Terabyte rate.

Not surprisingly, EMC claims its virtualization features go beyond those of Hitachi’s, and HDS claims EMC is off base with those claims.

Brian Gallagher, EMC president of enterprise storage, made the case for EMC’s virtualization Monday during the opening of EMC World.

“We’ve extended Symmetrix’s data integrity to non-EMC devices. Hitachi does not do that,” Gallagher said. “Also, our technology is free of charge. You can virtualize any amount of non-EMC storage behind Symmetrix. Hitachi gives you a certain amount of terabytes for free, and then they charge when you go beyond that. Hitachi will also tell you not to use virtualization for databases, we don’t say that. We also extend FAST to other arrays. They [Hitachi] don’t extend auto tiering [to VSP].”

Claus Mikkelsen, chief scientist at HDS, disputed EMC’s points in an e-mail to Storage Soup. He said the HDS VSP supports “full inheritance” on externally virtualized storage devices from HDS or other vendors.

As for EMC’s free claim, Mikkelsen said “EMC states that software license enablement of FTS is a no-charge feature to customers, but fails to mention the future impact on software maintenance costs for the FTS license and any other EMC software license that charges maintenance based on installed capacity. With the Switch It On program from HDS, virtualization is free and third-party capacity is deeply discounted.”

Mikkelsen said HDS offers “prudent” advice on using virtualization with databases.

“We state that that Hitachi Dynamic Tiering on VSP will intelligently place data pages based on an application’s I/O access pattern,” he said. “Additionally, we recommend that customers do not immediately place OLTP database environments with high I/O transaction rates and low average response time requirements as externally virtualized storage. This is prudent advice that any intelligent storage vendor would recommend.”

Finally, Mikkelson said HDS’ technology does extend its Dynamic Tiering to third-party virtualized arrays.

EMC’s FTS will require time for users to kick the tires before it can be accurately judged, but the virtualization features are welcome additions.

“This is at least a step down the path to VSP-style virtualization,” said Ray Lucchesi, president of Silverton Consulting.

You can expect more steps – and more spats with HDS – before EMC’s array virtualization story is finished.

Today, DataCore addressed that omission through a partnership with cloud storage gateway vendor TwinStrata.

SANsymphony-V virtualizes storage across pools of heterogeneous systems, adding management features such as thin provisioning, RAID striping, asynchronous replication, and snapshots. The new tiering feature lets customers dynamically move disk blocks among different pools of storage devices.

Beginning in late October, when a customer purchases SANsymphony-V — which DataCore calls a “storage hypervisor” — it will include a 1 TB version of TwinStrata’s CloudArray virtual appliance at no extra cost. That lets DataCore customers move data off to the cloud, although they need a subscription with a cloud storage provider such as Amazon S3 or Nirvanix. A DataCore customer can also go beyond a 1 TB gateway by upgrading the appliance through TwinStrata, which charges $4,995 for unlimited capacity. The CloudArray software deduplicates, compresses and encrypts data before moving it to the cloud.

TwinStrata also sells its cloud gateway as an appliance. DataCore CEO George Teixeira said SANsymphony-V will also work with the hardware appliance, using it as cache to speed backups.

“Now we’ve allowed a cloud tier to be part of our storage hypervisor,” Teixeira said. “When data gets to a lower tier, it can be put on an iSCSI device that is actually a cloud disk.”

There are other cloud gateway products on the market, but Teixeira said he picked TwinStrata because it supports iSCSI while most of the others are for files and backup. As the name implies, SANsymphony-V is a SAN application, so it required a block storage gateway.

“Most of them [gateways] are doing things at the file system,” Teixeira said. “These guys [TwinStrata] present an iSCSI disk. We can use storage virtualization across all disk and this looks like another disk we are auto-tiering, so it plays into our model. This doesn’t require a lot of thinking on the part of the customer. It’s just another tier, and you can choose to pick a pay-as-you-go model.”

Although TwinStrata can handle primary storage, Teixeira said he expects his customers to use the cloud mostly for backup and archiving. “I think we have some ways to go to get to primary storage in the cloud,” he said. “We’re looking mostly at backup, archiving and scratch storage. We think most production data will stay on-premise, but why not put non-production and backup data on the cloud?”

TwinStrata is looking to use partnerships to get its gateway into the market, even if it has to give it away at first. Last month the startup began offering a free 1 TB CloudArray appliance to Veeam Backup & Replication customers who want to back up to the cloud.

Gartner research director Gene Ruth said organizations are interested in moving to the cloud, but are still looking for the best way to go about it. He said storage virtualization and gateways are two potential starting points.

“It’s hard for people to get their arms around the idea that they’ll put all their data out on the cloud, but they know they have some data they can put out there,” he said. “I see storage virtualization in general as one of those building blocks to help consolidate a cloud storage environment around a common provisioning point. It’s not the end-all, but it’s a good start.”

Ruth said he also sees gateways as key enablers of cloud storage, and not just as standalone devices.

“It seems a pretty obvious step for major vendors to put that [gateway] functionality into disk arrays and file servers,” he said. “I don’t think it’s that difficult to add a gateway to their arrays.”

Cloverleaf’s Intelligent Storage Network (iSN) products are designed to manage heterogeneous storage environments as well as disaster recovery under one user interface. The transaction will consist of $2.5 million in cash, plus $9.5 million in Dot Hill stock.

According to a Dot Hill press release, Cloverleaf was formed in 2001 out of Elta Systems/Israel Aerospace Industries and was VC funded to the tune of $43 million. Dot Hill CEO Dana Kammersgard said in the release that the move signals a shift for Dot Hill, up to this point primarily a provider of storage hardware systems to OEMs, to intensify its focus on the storage software market.

“This acquisition is in line with our previously stated objectives to transform Dot Hill into a storage software and solutions focused company,” Kammersgard was further quoted as saying in the release. “Cloverleaf’s products accelerate this plan by as much as two years and with the breadth of features they provide, we believe we can compete very well with both the virtualization appliance companies as well as with the newer storage companies who bundle similar features that operate solely on their own array products.”

Dot Hill sells its storage arrays through OEM deals with Fujitsu, Hewlett-Packard, NetApp, Sun, and others, so it probably senses an industry wide shift to add virtualization features to its systems. But you have to wonder how much of an opportunity Cloverleaf management and investors believed existed for their software. They sold out for barely one quarter of the amount invested, and most of what they got is tied into the value of Dot Hill stock.

Dot Hill will hold a conference call Thursday to discuss the acquisition, which is expected to close mid-month.

The expansion of logical volume support from 2 TB to up to 1 PB is made possible by DataCore’s move to 64-bit support in the latest major release of its storage virtualization software. The 64-bit support lets software address much more storage capacity in a single volume than previous versions.

DataCore director of product marketing Augie Gonzalez said that as with last year’s 1 TB “mega-cache” support, the logical limit is beyond where most customers will be looking to stretch today. But the previous 2 TB limit had grown impractical for making RAID sets out of the latest 1 TB and 2 TB SATA disks.

“The logical volume expansion and thin provisioning allows users to say, ‘I don’t care how big the volume will be in the future’,” Gonzalez said. “Rather than defining LUNs up front and then having to make changes later, you can immediately set up a large volume and expand the storage with no applicvation or infrastructure changes.”

A DataCore service provider customers says adding ALUA support will improve management in his storage environment. Joseph Stedler, director of data center engineering for cloud computing and managed IT service provider OS33, said he uses DataCore’s SANSymphony software to host back-end storage for his SMB customers. Right now SANSymphony is running on IBM System x servers in front of IBM DS3400 arrays and Xiotech Emprise 5000 storage devices, mirroring between redundant sets of the tiered hardware. The logical volume expansion will be especially helpful in cutting down on backup administration overhead, Stedler said.

“With two terabyte volumes, we had to present things in 2 terabyte chunks to our Veeam [backup] server,” he said. “With a larger primary volume we could have fewer backup targets to manage.”

Stedler said the addition of ALUA support will be even more important for creating multipath I/O in OS33′s VMware environment. “ALUA solves a major pain for everybody running DataCore with VMware,” he said. “The way VMware understood it before was active-passive only. DataCore was able to do active-active failover but with VMware you’d have to run multipathing with the most recently used path. The new release is fully compliant with ALUA, so VMware can view it as active-active.”

Stedler said he’s looking forward to the addition of more granular scripting capabilities for the software in future releases.

Sanrad VP of product management Allon Cohen, “Storage-intensive I/O can be a barrier to virtualization of some applications.” The company’s quality of service aims to overcome that by letting users throttle and reserve bandwidth for the most performance-intensive applications. This is similar to features offered by array vendors such as EMC and Pillar Data Systems.

However, unlike those storage arrays, Sanrad’s first QoS release will allow for two levels of service only, according to type of disk. A SATA disk pool will have one quality of service and SAS another (arguably the difference in speed between these drives also provides service differences). In the next release, expected in a month or so, Cohen said, Sanrad will add the ability to prioritize LUN by LUN. Sanrad is also releasing an API so users can set policies and thresholds for dynamic monitoring systems. 

The product is currently in beta with partners and will become available at the end of the quarter. Sanrad is formally announcing it in a couple of weeks. QoS will be a free feature included with Sanrad products and a free upgrade for existing users.

This is the follow-on to NetApp’s announcement last month that it planned to offer Flash-as-disk to go along with its Flash-as-Cache and DRAM-based Performance Acceleration Module (PAM).

A common issue with deploying solid state drives, analysts have said since EMC first announced support for STEC Inc. SSDs in Symmetrix last year, is integrating them with storage management software tools. Until recently, provisioning SSDs could be like provisioning hard disks used to be before storage virtualization–complex, slow and fairly rigid.

According to Bercovici, while the RamSan acts as the high IOPS storage behind the V-Series, the V-Series gives it storage management features through NetApp’s WAFL operating system:

WAFL’s log-structured architecture implements native load-balancing of write operations via write-aggregation to solid state NVRAM. This includes an innovative data layout engine which enables WAFL to “write anywhere” in order to optimize the placement of data across the appropriate media. For flash, that means native built-in wear-leveling optimized to spread writes over as many flash cells as possible in parallel, with minimum wear to each individual flash cell.

According to NetApp chief marketing officer Jay Kidd:

[The V-Series and RamSan] effectively [create] the industry’s only Enterprise Flash storage system that supports thin provisioning, fast snapshots, remote mirroring, and data deduplication

So far, though, this approach to Flash-as-disk isn’t really flying with storage admins. “This would be the most expensive way of doing SSD,” Tom Becchetti, storage admin for a manufacturing company and NetApp customer, wrote SearchStorage.com today in an email. “What I would like to see is just how EMC implemented their SSD. They have SSD that is physical and logically the same form factor of the hard drive. It would give you the most flexibility and as more SSD vendors show up on the scene, the cost will dramatically fall.”

Denizens of the storage blogosphere were even more outspoken. “Is that it??” was the title of a post on U.K. storage end user Martin Glassborow’s blog, Storagebod. “I expected more, I expected something which was going to force EMC to raise the bar on their SSD implementation.”

Right now, in the wake of VMworld Europe, there’s quite a bit of chatter going on about developing conflicts between VMware and its partners, especially in storage.

Hmm, now where have I heard that idea before?

Still, where earlier discussions on this blog have been purely speculative, some new articles and posts have surfaced that push the observation further toward reality. There are those who continue to pooh-pooh the idea, but Burton Group analyst Chris Wolf, whom I interviewed for the post linked above, came away from discussions with partners at VMworld Europe seeing Storage VMotion as more disruptive to VMware’s alliances than ever:

…it did not take long for me to realize that storage vendors were not exactly singing Storage VMotion’s praises. Instead, many storage vendors were still feeling Storage VMotion’s sting. Why should they care about a new storage value-add in ESX 3.5? Vendors that offer storage virtualization as an integral part of their products have seen one of their key value-adds move to the ESX hypervisor and as a result see Storage VMotion as a threat to their bottom line.

Then, our SearchITChannelcom sister site published an article today about how channel partners, too, are feeling conflicted over VMware:

Some VMware partners are blaming the company’s rapid ascent and aggressive strategy in the server virtualization market for creating channel conflict.

VMware’s strategy, according to these value-added resellers (VARs) and independent software vendors (ISVs), has been to fill gaps in its market coverage by acquiring partners in those specific segments and integrating new technologies into its hypervisor. And the more niche markets the vendor enters, the more competition it creates with its partners.

“They have a go-it-alone approach,” said Erik Josowitz, vice president of product strategy for Surgient Inc., a VMware ISV partner in Austin, Texas. “They’re predatory in a certain sense.”

News writer Colin Steele, who reported that story, has some further tidbits on his blog as well (though he brings up the Patriots’ 18-1 season in that post, which stings a little for yours truly).

It’s still not much more than a matter of opinion and conjecture at this point, but it looks to me like when it comes to VMware and its partners, especially in the storage market, the plot is thickening. 

The problem, ESG analyst Brian Babineau points out, is that VMware grew 90% last year–it’s not that 50% growth is bad, it’s that 50% growth is relatively bad. “You’ll get the airbag on that stop,” is the expression I’ve heard used.

Meanwhile, the consensus is that EMC’s poor stock performance is a direct result of the VMware issue–even though EMC has achieved its goals of folding in a dizzying array of acquisitions, balancing its revenue streams across software, services and its core storage hardware business, and bringing its products up to speed with emerging technology trends. . .even getting out ahead of them with the first Tier 1 array, DMX-4, to support flash drives internally. “I don’t know that EMC’s business execution could have been much better,” is how Brian put it to me yesterday in my story on EMC’s earnings call.

And yet these companies both are in trouble on the stock market today, and more alarming is the underlying reason: a dramatic slowdown in revenue predicted for VMware. If this truly comes to pass, it could be the darkest omen yet in the chain-reaction brought about in the market by the subprime mortgage crisis, the culmination of fears about the tech market in general this year that began when Cisco revised predictions downward in November. The general vibe from the financial eggheads seems to be that if the highest-flying tech company on the market is forecasting a slowdown in spending, what’s next?

I’ll admit this scares me a little too. The way I understand it, greedy bankers gave loans to unqualified people, and then those shaky loans in turn were carved up into securities, meaning that when those unqualified debtors couldn’t manufacture money (lo and behold!) the whole house of cards started to tumble down. In a way, it’s satisfying to see the people who played on the dreams of low-income people to own homes by locking them into high-interest-rate deals, with no regard to how they were going to come up with the cash, get their comeuppance. But when it threatens the entire national economy, it’s hardly worth the last word.

But before I could step off the ledge into panic myself, this morning I had a chat with Andrew Reichman of Forrester Research, whose level-headed perspective is one I think will eventually shake out once the initial frenzy is over. Or, at least, I hope.

“This is typical of the financial world–overblown expectations,” he said. “I still see VMware’s product and outlook as very strong.” Even in a recession, Reichman pointed out, VMware still has a value proposition, since server virtualization is a consolidation and cost-cutting play. “They can still demonstrate to companies why they should spend money on their product even as they try to put the brakes on IT spending otherwise.”

Great point. Then there’s just the plain fact that 50% growth ain’t too shabby! Especially as the market braces for a spending slowdown–and especially when analysts say 60% of the market has already purchased and deployed VMware’s product. “It’s rare in the technology world to see such consensus around one piece of technology,” Reichman concurred. “They’ve built up a lot of momentum and there’s still a lot of room for them to take advantage of the ‘network effect’ and expand existing customers’ use of the product.”

However, Reichman had another suggestion about the VMware situation that piqued my interest, especially given my recent post on storage virtualization and VMware and the sometimes tense relationships between the two. “What they need to do to continue their growth is take the money they got in the IPO and they’ve built up in revenue and find the next frontier.”

Reichman’s prediction for that next frontier in the near future is business continuity/disaster recovery, which VMware has already said it’s working on. “There’s a high level of interest at a lot of companies in using VMware for BC/DR,” he said. But he added that the next horizon will probably be primary storage–or storage virtualization.

“The story of storage behind VMware has never been clear,” Reichman said. “There are a lot of issues that remain around storage virtualization, performance and compatibility and a lot of room to improve that picture.”

He continued, “They’ve played Switzerland for a long time. Now they need to get off the dime and make a call about how storage is going to work behind server virtualization.”

Of course, it’s hard to tell what the consequences of that might be. Other analysts such as the Burton Group’s Chris Wolf have pointed out that if hardware vendors don’t support VMware, end users won’t take the risk of using their product. But has VMware’s ascension into a Wall Street bellwether changed that equation? Has its ubiquity in IT shops turned the tables on the storage vendors–so that end users will instead be less inclined to use a storage technology if it’s not certified with VMware? How will this balance-of-power play out?

It’s like that expression about the old Chinese curse. We are living in interesting times.

FalconStor is walking a s tightrope here, because it’s also partnered with Virtual Iron’s large rival VMware. But FalconStor has to come up with reasons to use Virtual Iron over VMware, (i.e. ways to promote that partnership). This led Alex to begin an  interesting conversation with FalconStor’s vice president of business development Bernie Wu about the pros and cons of virtualizing storage with VMware vs. Virtual Iron. Wu pointed out what he’d later reprise in a separate call with me: that the use case for FalconStor’s IPStor storage virtualization software is in many ways stronger with Virtual Iron, because VI doesn’t have its own file system, like VMware does.

As Burton Group senior analyst Chris Wolf patiently explained to me later, VMware’s file system means that its hypervisor (the software layer that controls the host server, guest OSes, and their interaction with the rest of the network) is handling virtual hard disk mapping on back-end storage systems. You can use VMware with raw device mapping (RDM), but then you turn off many of the features VMware users have come to like about VMware, such as VMotion. (RDM also has a slightly limited “virtual mode” as of 3.0, but that’s a tangential discussion.) This makes virtual hard disk mapping performed by storage virtualization products, whether appliances or software, at least somewhat redundant.

So I asked Wu, “what are users missing out on if they can’t use your storage virtualization software with VMware?”  His first answer was large-scale data migrations.

Up until VMware’s Virtual Infrastructure 3.5, VMware had no ability to move the data it managed in its virtual hard disks on back-end storage; hence storage virtualization devices stepped in to fill the gap. With Storage VMotion in 3.5, that gap was at least partially closed. Storage VMotion is still a difficult way to do a large-scale migration, however, because it migrates data one host at a time. So storage virtualization devices, which perform migrations en masse, still have that advantage. At least, until and unless Storage VMotion adds that capability.

Aside from large-scale migrations, Wu also told me that thin provisioning is another capability IPStor offers that VMware doesn’t. That’s a big deal–VMware’s best practices recommend that users allot twice the amount of disk space they actually plan to write to; the ability to expand capacity on the fly helps everyone avoid buying 2x the amount of storage they need.

The Burton Group’s Wolf pointed out plenty more gaps in VMware’s storage capabilities–heterogeneous array support; heterogeneous multiprotocol support (Storage VMotion doesn’t support iSCSI yet);  I/O caching; and heterogeneous replication support.

Some of these gaps will likely be filled by VMware or the storage industry. For instance, when it comes to multiprotocol support, VMware’s MO with new features has always been to support Fibre Channel first and they usually get around to iSCSI soon after. And what happens to the need for heterogeneous multiprotocol support if FCoE ever takes off? What of I/O caching, when and if everybody’s working with 10 Gigabit Ethernet pipes? And VMware’s launching its own management software for heterogeneous replication support (even if it’s not doing the replication itself).

So it seems that storage virtualization players will have to start coming up with more value-adds for VMware environments as time goes on.

VMware has its own tightrope to walk, too. Take replication for example–VMware supports replication from its partners, saying it doesn’t want to reinvent the wheel. But that’s the kind of thing it said when users were asking for Storage VMotion back in 2006, too.

“Deep down, I believe that VMware isn’t going to push its partners out,” Wolf said. And indeed, VMware did make a good-faith gesture last fall with the announcement of a certification program for storage virtualization partners. Wolf also pointed out, “A lot of organizations are afraid to deploy something in the data path unless their hardware vendors will support and certify it–without the support of their partners, VMware would have a tough time playing ball.”

But that might not be the case as much now as back when EMC first bought VMware in 2003 and everybody in the storage world scrateched their heads and wondered why. Now, VMware has its own muscles to flex as its billion-dollar 2007 IPO for 10 percent of the company proved.

More and more analysts are telling me that the hypervisor will become the data center operating system of the future. Over in the server virtualization world, Wolf says VMware competitors argue that the hypervisor is a commodity, and VMware says it isn’t. “In order to keep the hypervisor from becoming commoditized, they have to keep adding new features,” he said.

Which suggests to me that storage virtualization vendors should probably be working on new features, too.