data reduction

Hifn adds speed and software to data reduction cards

Hifn (now part of Exar Corp.) is taking another crack at getting major OEMs to ship products integrated with its DR line of compression, encryption and deduplication hashing acceleration cards, which could potentially spur the development of primary storage data deduplication offerings.

Prior to its acquisition by Exar, Hifn began sampling Express DR 250 and 255 cards to OEMs, but they hadn’t made their way into any announced third-party products. At this spring’s SNW, Hifn launched its own product based on the DR 255.

It was unclear why the chip boards, which perform processor-intensive data reduction and encryption in silicon, hadn’t caught on with OEMs. Maybe Hifn’s announcement today of its new DR 1600 series may tacitly answer that question with new features such as high availability and boosted performance.

The DR 1600 line consists of six new models offering different levels of performance and combinations of compression, encryption, and dedupe. The Express DR 1600, 1610 and 1620 perform LZS compression and encryption only, at speeds of up to 300 MBps, 900 MBps, and 1800 MBps, respectively. The Express DR 1605, 1615, and 1625 run at the same three levels of throughput, but offer compression, encryption and hardware-based hashing for data deduplication (hash comparisons must still be performed by an OEM in software).

Hifn has also developed new software to go with the cards for this release, which includes a new API to standardize and ease integration of the cards into storage products to make it quicker for OEMs to take them to market. The 1600 series includes new high availability software for failover between cards, or to “pass through” traffic. That means if one card fails, the other can still perform compression, encryption, and dedupe in software.

According to Zack Mihalis, director of product marketing for Hifn, the new cards are sampling to OEMs and will become generally available at the end of July. Mihalis claimed that several large OEMs are considering the cards, potentially for primary storage dedupe. EMC, NetApp and Quantum are traditionally among Hifn’s OEMs, but Mihalis declined to disclose if any of them are sampling the DR 1600 cards.

Still, some industry analysts see this as the first step toward primary storage data reduction products becoming as ubiquitous as those for backup workloads. “Hifn has some very major OEMs as clients,” said IDC analyst Benjamin Woo. “This release is very timely - in this downturn we need to be more efficient with how we deal with data.”

However, Taneja Group analyst Jeff Boles pointed out that there’s still plenty of engineering work to be done to produce primary storage dedupe products, even with some of it already completed by Hifn. “Keep in mind that Hifn is hashing at 1,800 megabytes per second, but that’s not the speed of writing out to disk,” he said. “It’s still up to someone to make maximum use of this on disk, with caching, etc. Can you use this to service a random workload? That may be an engineering feat in itself.”

IBM virtual desktop storage update - sort of

Last week I wrote about some confusion I had regarding IBM’s virtual storage optimizer (VSO) for VMware Desktop Infrastructure (VDI), especially after I was told by a VMware official that the IBM product, credited to an internally-developed algorithm, was based on VMware’s Linked Clone API.

I wrote to one of the researchers involved and got a response through IBM’s PR spokesperson that:

• The IBM-developed algorithm is based on VMware API available in Virtual Infrastructure version 3, not the VMWare LinkedClone API. Specifically, the algorithm uses VMware Infrastructure SDK 2.5.0 as documented at https://secure.techtarget.com/exchweb/bin/redir.asp?URL=http://www.vmware.com/support/developer/vc-sdk/ and file system level access on ESX servers.
• We developed the algorithm based on the API that was publicly available and supported at the time that we began development efforts
• VMware can provide detail on the differences between the APIs in Virtual Infrastructure version 3 and VMware LinkedClone API

So far no response from VMware.

Regardless of what API was or was not used, what I am trying to get at is the functional difference between these two products, if any. If there is one, it’s important for users to know about. If there isn’t one, it speaks to the growing convergence between VMware’s virtual infrastructure and storage vendors’ value-add software.

the bottom line right now seems to be that IBM’s product is for existing IBM customers, since it requires professional services through IGS. There are some shops that need the IBM label before they buy, and so VSO could at least be a fit for them.

Appreciate weigh-ins from IBM, VDI, and / or VMware experts.

Much ado about virtual desktop data storage

Last week at VMWorld, IBM announced the Virtual Storage Optimizer (VSO), ESX-based software that reduces virtual desktop storage by creating a “golden image” of the desktop’s operating system and other static files, while saving the changes users might make to that image.  It’s a concept akin to NetApp’s space-efficient snapshots, but because it’s delivered in software at the ESX level, IBM said, it can be applied to any storage system.

The next day, at a keynote, VMware officials demonstrated a new concept they’re rolling out in the next version of VMware Infrastructure called LinkedClones. LinkedClones create a “golden image” of virtual desktop files, as well as incremental changes;. The golden images, VMware demonstrated, can be updated with patches that automatically proliferate to all virtual machines based on the image to simplify rollouts and updates.  

From the briefing I’d had with IBM the day before and this keynote demonstration, these products seemed similar. Since the VMware demo last Wednesday, I’ve been trying to assess what might be different about them. VMware officials I spoke with Wednesday and Thursday said they didn’t know enough about IBM’s product to comment on its differentiation and IBM spokespeople were unavailable.  IBM’s press release about VSO had stated that it was “based on an algorithm developed by IBM Research,” but VMware said LinkedClones  wasn’t based on anything from IBM.

Today I spoke with VMware director of enterprise desktop Jerry Chen, who told IBM’s VSO is based on the LinkedClones API. I asked about the algorithm developed by IBM. “There are things our partners can do to further optimize LinkedClone,” Chen said. “For example, there are different settings for the number of LinkedClones each master virtual machine can copy.”

Chen said he wasn’t familiar enough with the IBM product to say what VSO adds on top of LinkedClones.  Meanwhile, IBM has been coy on this one. Since last week, I’ve put in numerous requests for comment by phone and email, including a fresh round of requests today after speaking with Chen. So far, no comments have been forthcoming.

Ocarina will pay you 10 grand to beat it at data compression

Ocarina Networks, which came out of stealth in April, claims its compression appliance will reduce file data on primary storage systems. Its main competitor, StorWize, applies standard (2:1) compression to files, but Ocarina claims 10:1 compression and the ability to compress already compressed objects, such as video and photos. The company even claims its algorithms can be used to create a 3-D cube of numeric values to represent a photo or video image, so it can recognize elements that it has “seen” before.  Pretty interesting, albeit ice-cream-headache-inducing stuff.

So it was puzzling to see the announcement of the Ocarina Compression Prize, a $1 million fund that will be doled out in $10,000 increments for each submission that advances the current best scoring compressor by at least 3%. Isn’t the idea supposed to be that Ocarina has the most compression expertise in the market?

“A lot of our compression work is already based on prior art,” CEO Murli Thirumale told me. The idea, he explained, is to make this contest a “category builder,” raising interest in the subject of primary storage compression. “A lot of compression work is focused around transmission of files, rather than reducing them for storage. We want to encourage the best minds in compression to address it in that context.”

So I guess it doesn’t matter how many cool algorithms you can bring to the table if there isn’t really a market yet. “As there’s more widespread adoption [for products], clearly [vendors] with a leadership stance will benefit more,” Thirumale said.

“Good compression has a history of coming from independent researchers, open source or anywhere that can foster easy standardization and non-proprietary code,” Taneja Group analyst Jeff Boles says. “So this seems like a pretty good approach to me. Interesting stunt to boot.”  

The initial prize fund will include awards for three categories: JPEG 2000 recompression, h.264 video recompression and an industry file mix for engineering CAD file types. Maybe Riverbed, Silver Peak and Autodesk will jump in on that last one.