» VIEW ALL POSTS Jan 4 2010   11:07AM GMT

Data deduping goes beyond backup

Posted by: Eric Slack
data deduping, Eric Slack, Storage Channel

In an article on SearchStorage.com, “Top 10 enterprise data storage news stories of 2009,” Beth Pariseau does a great job of recapping some of the interesting developments of this past year. In this blog and the next few, I’m going to mention some of these developments and the opportunities they present for VARs. First up on my list is data deduping.

 

When data deduping first came out, I was working as a storage VAR. Data Domain had this new technology that got us appointments all over town but was a little difficult for users to grasp. We’d spend the first meeting and a good part of subsequent meetings explaining the technology, originally called “commonality factoring.” They did grasp the effective data reduction rate of about 20x, but didn’t like the fact that it was being presented for backup only. We’d explain how the technology relied on the inherent redundancy in backup data and that using it for regular files would net only a fraction of these data reduction percentages.

 

A much stickier term than commonality factoring, data deduping is now expanding beyond backup and being applied to storage systems. Single-digit data reduction has replaced the numbers we saw with backups, but as a technology enhancement to (lower tier) primary storage, dedupe is still attractive. Consequently, there are a growing number of products offering deduplication.

 

Permabit and NEC Hydrastor both have scalable clustered architectures that feature data deduping to improve storage capacity and reduce the price per gigabyte. Ocarina’s technology is geared toward digital media archiving and, unlike “traditional” dedupe, it can actually reduce image file formats such as JPEG, TIFF, GIF, etc. NetApp includes deduplication as a feature in its OnTap operating system. FalconStor’s dedupe technology is included in a number of vendors’ products (Nexsan, Spectralogic nTier) that are sold for backup but also used for non-backup applications as well.

 

The point is, data deduping is a storage optimizing technology that can make your tiered storage solution more attractive, especially compared with traditional disk solutions that don’t offer dedupe. In this era of low budgets for new capacity and high requirements for storage utilization, incremental improvements like deduplication can make a difference.

 

Follow me on Twitter: EricSSwiss.