This really begs the question, “Why don’t we delete data when we don’t need it anymore?” When I asked this question to IT people, they had several reasons for keeping data.

Government regulation was the most common reason. Many of these regulations are in regard to email and associated with corporate accountability. People in vertical markets such as bio-pharmaceuticals and healthcare have extra industry-specific retention requirements.

Business policy was another top reason for not deleting information. There were three underlying reasons for this category. In some cases, the corporate counsel had not examined the information being retained and had issued orders to keep everything until a policy was developed. Others keep data because their executives feel the information represents business or organization records with future value. (It was not really clear what this meant.). In other cases, IT staff was operating off a policy written when records were still primarily on paper and had not received new direction for digital retention.

Another common response was that IT staff had no time to manage the data and make retention decisions or to involve other groups within the organization. In this case, it is simpler to keep data rather than make decisions and take on the task of implementing a policy.

The other reason was probably more of a personal response – some people are pack rats for data and keep everything. I call this data hoarding.

Rather than only listing the problems, the discussion about data retention should always include ways to address the situation. Data retention really is a project. To be done effectively, it usually requires outside assistance and the purchase of software tools. In every case, an initiative must be undertaken. This includes calculating ROI based on the payback in capacity made available and reduced data protection costs. The project requires someone from IT to:

• Understand government regulations. Most are specific about the type of data and circumstances, and almost all of the regulations have a specific time limit or condition for when the data can be deleted.
• Examine the current business policies and update them with current information from executives and corporate counsel. Present the costs of retaining the data along with the magnitude and growth demands as part of the need to review the business policies.
• Add system tools to examine data, move it based on value or time, and delete it when thresholds or conditions are met.
• Get a grip. Data hoarding is costing money and making a mess. The person who replaces the data hoarder has to clean it up.

Knowing when data can be deleted is good operations practice in IT. It is a key component of storage efficiency. The Evaluator Group has more on storage efficiency here and here.

(Randy Kerns is Senior Strategist at Evaluator Group, an IT analyst firm).