I don’t hear from Avid much, mainly because the vendor is mostly focused on video-editing software. So I took their invitation to tour their demo data center at their Tewksbury headquarters as yet another sign that storage has gone far more mainstream than it was when I first joined the industry.
That’s a pattern I’ve been noticing a lot lately–storage has emerged at the center of everything, and become much more interdisciplinary. It’s partly because people are starting to think more about the overall data center than its separate spheres, but also because the continued digitization of data and the enrichment of data formats, has naturally brought more attention to the problem of where we’re going to put it.
Avid storage senior product manager Bill Moren agreed when I met with him yesterday. “It’s not so much an accessory as it used to be,” he said. “It’s a big part of the end to end workflow that users are concerned about.”
The video processing world in particular has been among the verticals that have seen data absolutely explode, thanks not only to new mechanisms for digital delivery but the advent of ever-bigger HD formats for film and video. Avid’s networked storage system, Unity ISIS, was updated with the goal of addressing ballooning capacity Dec. 1 with support for a 10 GbE connection directly to the desktop workstation for faster access (previous versions supported 10 GbE to the switch), support for 1 TB drives which bring the total system capacity to 384 TB from 192 TB, improved internal bandwidth of 400 MBps per storage chassis (up from 250-300 MBps), and LDAP/ActiveDirectory integration.
The ISIS architecture consists of chassis called Storage Engines. Within each engine is a series of blades containing CPU, memory, and two disk drives. I’m sure you can see where this is going: the blades are parallelized, and divide up pieces of files among them according to software algorithms. Data is reassembled for playback using keys issued by a metadata server. Moren said the metadata server is usually a piece of white-box commodity hardware. Some in the industry argue that the metadata node can become a bottleneck. I asked Moren about that, and it kicked off a conversation about what Avid’s trying to do vs. what the data center vendors I usually talk to are aiming for.
For people looking to work on realtime media files, Moren said, the amount of storage you can cram into one box is less important than it is for users running big shared repositories of files in the enterprise. What’s more important isn’t the burst-rate speed or peak performance of the box either, but rather the its ability to deliver data in an almost metered fashion so frames of video aren’t dropped. Moren described the process of Avid’s parallel file system feeding data from the parallelized blades through the metadata node to the client as being “like a metronome.” It’s the predictable steadiness, not the raw speed, that’s most important, he said.
I found that interesting, especially because of all the storage vendors out there looking to capture the video content delivery market. Many of those vendors, however, reference customers in the 3D rendering space, or special effects houses for live-action films, which often work on single frames rather than live streams. This isn’t a distinction I’ve considered before.