The Storage Networking Industry Association (SNIA) released the results of a survey of its members on long-term archiving, which indicated that yes, there is an archiving problem in the industry: of 276 “long-term archive practitioners” who responded to the survey, 80 percent said they have information they must keep over 50 years, and 68 percent said they must keep this data more than 100 years. 30 percent said they were migrating information at regular intervals. Around 40 percent of respondents are keeping e-mail records over 10 years. 70 percent said they are ‘highly dissatisfied’ with their ability to read their retained information in 50 years.
One caveat is that the survey, which was sent to more than 10,000 members of the SNIA and associated organizations, was “self-selecting”, and though 80 percent sounds like a high number, the number of SNIA members concerned enough about long-term archiving to take a half hour to complete the survey was a much smaller percentage.
Perhaps more interesting were the vertical markets where the survey drew the most responses; according to SNIA they include education, manufacturing and chemical processing, power and energy companies and banking in addition to the obvious ones like libraries and government.
In response, SNIA’s 100-Year Archive Task Force (a name which conjures up an image of SNIA Ninjas crashing in through the window…) will be creating an archiving-infrastructure reference model and adding an archiving information field based on the Open Archiving Information System standard into its own standards including SMI-S and XAM.
Speaking of which, XAM proof of concept demos will be taking place at fall SNW, based on version 0.6 of the standard. This doesn’t jibe with what SNIA told us last October about the timing for the standard in a story for Storage Magazine–back then, those demos were supposed to be happening in San Diego this past April. Now, SNIA says that at the time of that story’s publication, the XAM Initiative hadn’t been officially launched. This, according to a SNIA spokesperson, has now officially, formally occurred as of Spring SNW 2007, and since that announcement, at least, the timing on proof of concept demos hasn’t changed. Hence, “there has been no delay” in XAM’s progress. (Clears things right up, doesn’t it?)
At any rate, whenever it actually gets here, XAM, which stands for Extensible Access Method Interface, would define a single access method for archiving devices like EMC’s Centera, Hewlett-Packard’s (HP) Reference Information Storage System (RISS) and the HDS HCAP product. The SNIA’s also calling for “self-healing systems” available today to be modified for use in archiving and storage, automating the migration process between old and new data formats.
We don’t necessarily need the SNIA to tell us that archiving’s a big deal right now, and they’re certainly not alone in pointing out that, hey, all this technology’s great, but even a cave drawing kicks its butt when it comes to readability longer than a few months after its creation. However, there’s one aspect of long-term data preservation that isn’t addressed by XAM and other software-based standards: storage media itself.
SNIA reps say software’s a bigger concern now with new optical media that’s claiming a 30 to 50 year shelf life, but other experts feel that most storage media is further behind the long-term viability curve than that. From my point of view, I can’t help but wonder why more attention isn’t being paid to the development of new storage media in this industry, given the double-whammy of power and cooling and long-term preservation issues that have lately become hot topics. After all, even if every vendor adopted the XAM standard tomorrow, without better long-term media, there won’t be any data available for it to access in a couple of decades, anyway.