From a VAR’s perspective, these are the kinds of “pain” situations to look for. Solving a data deluge problem can have a number of solutions and involve multiple vendors’ products — the ideal integration opportunity. One solution is to use a content indexing application to create a searchable index of all this unstructured and unorganized data that people can’t seem to get rid of.

There are number of ways to implement file indexing. One approach is to integrate archiving with other data applications, like email and backup. Symantec’s Enterprise Vault and CommVault’s Simpana offer data archiving with storage tiering and content indexing for e-discovery and other requirements. They integrate archive with backup, email and data management to help companies get their arms around their unstructured data across the enterprise.

Digitiliti takes a little different approach, combining active archiving storage and content indexing to provide what it calls a Universal Archive Platform (UAP). This solution consists of an “information director” and an “archive store,” with a software agent for each client computer or server. When first implemented, data stores — typically file servers, NAS appliances and email servers — are “ingested” into the archive, whereby a content-aware index is created. Then, subsequent changes or new files are added to the archive as the files are saved, keeping the archive current without impacting users or performance, similar to the way a continuous data protection (CDP) process works. A scale-out, modular architecture with an object-based file system allows the “archive store” to be geographically dispersed or to be located in Digitiliti’s cloud. As data is moved off the primary storage locations, the UAP can free up this premium space, and moving data to the archive tier can take it out of the backup rotation, if so desired.

A more specific use case, dealing with legal discovery, uses an application that crawls all file sources on the network and creates an index that can be used for e-discovery, as well as data mining and overall organization and productivity. This type of solution can be implemented as a network-based application or as a cloud-based service, or it can be offered on an engagement basis by service providers. We’ll take a look at some of these in a future post.

Follow me on Twitter: EricSSwiss

Aside from the obvious operating expense costs, like power, cooling, data center space and admin time consumed for each terabyte of storage, there are some other hidden costs to storing data that can change the arithmetic of the earlier equation. When finding the data we need, at the time it’s needed, becomes more difficult, the cost goes up. This is a productivity issue for all users, not just an overhead item for IT admins. I’m reminded of those creepy reality TV shows about hoarders, people who literally can’t walk in their houses because of all the stuff. This is a pretty extreme example, and I’m not suggesting we’re really data hoarders, but the concept is valid. Increasing the time it takes to find the information we need affects productivity. And besides making tasks and projects take longer, an inability to find data easily can result in people not finding data that’s needed, impacting the quality of their work, not to mention their frustration levels.

There are technologies available that can help address this problem. Aside from organizational tools like SharePoint, there are a number of file and content indexing engines on the market. Many of these products are used for e-discovery purposes, either implemented by IT or by contractors who are engaged in response to pending legal action. But these solutions can also help companies get their arms around the data sets they’ve accumulated. A content index can put every piece of data at users’ fingertips, eliminating the time and frustration formerly involved with searching for a document or piece of information. In the process they can improve storage utilization, especially primary storage as the file and content indices they create are ideal for driving a tiered storage or archive strategy.

For VARs this should be fertile ground, as most companies suffer from some form of this problem. Also, it’s not a one-and-done engagement. Many solutions involve multistep projects that can entail professional services as well as hardware and software. In the next blog we’ll detail some of the solutions available in this space.

Follow me on Twitter: EricSSwiss