The increase in capacity demands come from many sources. These include business expansion, the need to retain more information for longer periods of time, data types such as rich media that are more voluminous than in the past, and an avalanche of machine-to-machine data used in big data analytics.

The data increase requires more storage systems, which are usually funded through capital expense. Often these are paid for as part of a project with one-time project funds.

The increase in data also changes the backup process. The amount of time required to protect the information may extend beyond what is practical from a business operations standpoint. The amount of data to protect may require more backup systems than can physically be accommodated.

It is common for new projects to budget for the capital expenses required. Unfortunately, the increase in operational expenses is rarely enough to support the data protection impact. The administration expenses from increased time spent by the staff in handling the data can be estimated, but is difficult to add to the project budget because it is an ongoing expense rather than a one-time expense.

Unexpected data growth can exceed capacity-based licensing thresholds and turn into an unpleasant budget-buster. Even expenses related to external resources such as disaster recovery copies of information may ratchet up past thresholds.

There are new approaches to data protection. However, there is usually not enough funding available to implement alternative data protection approaches. Changing procedures in IT is also difficult because of the training required and the amount of risk that is introduced.

Vendors see the opportunities, and address them with approaches that make the most economic sense for them. The most common approach is to enhance existing products, improving their speed and effective capability. Another vendor approach is to introduce new data protection appliances combining software and hardware to simplify operations. Whether these are long-term solutions or merely incremental improvements depends on the specific environment.

Another approach evolving with vendors is to include data protection as an integral part of a storage system. This involves adding a set of policy controls for protection and data movers for automated data protection. These come in the form of block storage systems with the ability to selectively replicate delta changes to volumes and in network attached storage systems that can migrate or copy data based on rules to another storage system. Implementing this type of protection requires software to manage recovery and retention of the protected data.

A change must be made to continue the IT mandate for protecting information. However, the fundamental problem with data protection addressing capacity demand is economics. For most IT operations, the solution cannot represent a major investment and it must be administratively cost-neutral to a great extent. Current data protection solutions that meet those requirements are hard to find.

(Randy Kerns is Senior Strategist at Evaluator Group, an IT analyst firm).

Like storage systems, IT organizations tend to get more complex over time. The complexity affects the decision-making process involving the storage architects/administrators and the business owners responsible for the applications and information. There may be layers of groups with varying responsibilities between the staff that needs to develop and implement the storage technologies, and those who truly understand the requirements.

Having one or two levels of filtering makes it much more difficult to understand the needs of the “customer,” who in this case is the business owners or their staffs. Much of the technology optimization process involves understanding what is required and includes byplay between architects and the actual customer. The lack of that interaction and base understanding of customer needs often results in a solution that fails to address key needs and planning for the changes that will occur.

Optimizing organizational structure is perceived to be a difficult task. The changes affect many influential people and groups. The need is obvious, but the complex organization structures that have developed over time may be deep-rooted and require a commitment and direction from the most senior levels in IT. Working towards data center optimization and improved storage efficiency is much more difficult without this commitment and direction. Often, the process brings compromises that reduce the project’s effective value.

Understanding the problems in working toward an optimized environment means understanding the organization structure and those limitations. Any storage optimization strategy must take into account the structure as well as technology and products. This brings about more work, but it is the current situation in many IT operations.

(Randy Kerns is Senior Strategist at Evaluator Group, an IT analyst firm).

The growth in data that they will have to deal with comes from several areas:

• The natural increase of the amount of data required for business continuance and expansion of current operations. This data represents the normal business requirements.

• New applications or business opportunities. While this is a positive indicator for the business, it represents a potentially significant increase in the amount of data under management.

• The machine-to-machine data from pervasive computing generates an overwhelming amount of data that most IT people have not had to deal with before. The data is used for “big data” analytics or business intelligence, and it will be left to IT to manage for the data scientists.

The problem is really one of scale. Because operational expenses typically are not scaled properly to address the management required for that amount of data, there is insufficient budget to handle the onslaught of data.

Storage professionals are looking at different approaches to address the increased demands. These include more efficient storage systems. Greater capacity efficiently – making better use of capacity – is a big help. So are storage systems that support consolidation of workloads onto one platform.

Data protection is a continuing problem. The process is viewed as a necessary requirement but not as a revenue-enhancing area. Consequently, data protection needs are dramatic but often lack the financial investment to accommodate the capacity increases. This means storage pros must either find products that can be more effective while fitting within the financial constraints or re-examining the entire data protection strategy by using technologies such as automated, policy-controlled archiving and data reduction.

Exploiting point-in-time (snapshot) copies on storage platforms for immediate retrieval demands, implementing backup to disk, and reducing the schedule for backups on removable media to monthly or less frequently are considerations for stretching backup budgets.

Storage professionals need to be open to new ideas for dealing with the massive influx of data. Without addressing the greatly increasing capacity demand, managed storage becomes an oxymoron.

(Randy Kerns is Senior Strategist at Evaluator Group, an IT analyst firm).

The goals for optimization are reducing cost and increasing the operational efficiency. Capital cost savings come from getting more effective use of what has been purchased and operational cost savings come from reducing administration and physical resources such as space, power, and cooling. Optimized operations make IT staffs more capable of addressing the demands for business expansion or consolidation.

Along with server virtualization, storage efficiency is a major focus area for data center optimization (DCO) initiatives because of the opportunity for major savings. This Evaluator Group article provides an IT perspective on measuring efficiency. Storage efficiency can be accomplished in the following ways:

• Making greater use of storage capacity through data reduction technologies (compression and deduplication) and allocation of capacity as needed (thin provisioning).
• Supporting more physical capacity for a storage controller by enabling greater performance from the controller.
• Increasing performance and responsiveness of a storage system with storage tiering and intelligent caching using solid-state technology.
• Improving data protection with advanced snapshot and replication technologies and data reduction prior to transferring data.
• Scaling of capacity and performance in equal proportion (scale out) to support greater consolidation and growth.
• Providing greater automation to minimize administrative requirements.

DCO requires a strong overall strategy. Storage has a regular cadence of technology transition and product replacement, and DCO requires adding products and upgrading systems already in place. Evaluating the best product to meet requirements is a major part of the execution of the plan. There are many complex factors to consider and the decisions are not straightforward.

As DCO initiatives continue, storage efficiency will remain a competitive battleground for vendors and an opportunity for customers.

(Randy Kerns is Senior Strategist at Evaluator Group, an IT analyst firm).

This really begs the question, “Why don’t we delete data when we don’t need it anymore?” When I asked this question to IT people, they had several reasons for keeping data.

Government regulation was the most common reason. Many of these regulations are in regard to email and associated with corporate accountability. People in vertical markets such as bio-pharmaceuticals and healthcare have extra industry-specific retention requirements.

Business policy was another top reason for not deleting information. There were three underlying reasons for this category. In some cases, the corporate counsel had not examined the information being retained and had issued orders to keep everything until a policy was developed. Others keep data because their executives feel the information represents business or organization records with future value. (It was not really clear what this meant.). In other cases, IT staff was operating off a policy written when records were still primarily on paper and had not received new direction for digital retention.

Another common response was that IT staff had no time to manage the data and make retention decisions or to involve other groups within the organization. In this case, it is simpler to keep data rather than make decisions and take on the task of implementing a policy.

The other reason was probably more of a personal response – some people are pack rats for data and keep everything. I call this data hoarding.

Rather than only listing the problems, the discussion about data retention should always include ways to address the situation. Data retention really is a project. To be done effectively, it usually requires outside assistance and the purchase of software tools. In every case, an initiative must be undertaken. This includes calculating ROI based on the payback in capacity made available and reduced data protection costs. The project requires someone from IT to:

• Understand government regulations. Most are specific about the type of data and circumstances, and almost all of the regulations have a specific time limit or condition for when the data can be deleted.
• Examine the current business policies and update them with current information from executives and corporate counsel. Present the costs of retaining the data along with the magnitude and growth demands as part of the need to review the business policies.
• Add system tools to examine data, move it based on value or time, and delete it when thresholds or conditions are met.
• Get a grip. Data hoarding is costing money and making a mess. The person who replaces the data hoarder has to clean it up.

Knowing when data can be deleted is good operations practice in IT. It is a key component of storage efficiency. The Evaluator Group has more on storage efficiency here and here.

(Randy Kerns is Senior Strategist at Evaluator Group, an IT analyst firm).

Efficient storage systems control the placement of data within the storage system and the movement of data based upon a set of rules. The systems maximize capacity and performance in several ways:

• Data reduction through data deduplication or compression
• Tiering with intelligent algorithms to move data between physical tiers such as solid state drives (SSDs) and high capacity disk drives
• Caching to maintain a transient copy of highly active data in a high speed cache
• Controlling data placement based on quality of service settings for performance guarantees.

Efficient data management requires dynamically changing the data’s location. This may involve moving data beyond a single storage system. The initial data placement and subsequent movement is based on information about the data that determines its value. This information determines performance needs and frequency of access, data protection requirements including disaster recovery and business continuance demands, and the volume and projected growth of the data. Most importantly, the process takes into account that these factors change over time.

Managing data efficiently presumes that there are classes of storage with different performance and cost attributes, and a variable data protection strategy that can be adapted according to requirements.

When data value changes, it must be moved to a more optimal location with a different set of data protection rules. The movement must be seamless and transparent so the accessing applications are not aware of the location transitions.

Data protection changes must also be transparent so that recovery from a disaster or operational problem always involves the correct copy. Efficient data management must be automated to operate effectively without introducing additional administration costs.

This type of data management existed in the mainframe world for a long time as Data Facility Systems Managed Storage (DFSMS) before moving into open systems.

An interesting area that should be watched closely is migration capabilities built into storage systems that can move data across systems based on policies administrators set up. The IBM Storwize V7000 Active Cloud Engine, Hitachi Data Systems BlueArc Data Migrator and EMC VMAX Federated Live Migration are a few examples of these. The EMC Cloud Tiering Appliance also does this, but is not built into the storage system.

This will be a competitive area because there is great economic value in managing data more efficiently. Watch this area for significant developments in the future.

(Randy Kerns is Senior Strategist at Evaluator Group, an IT analyst firm).

Toigo defined storage efficiency from engineering, business and operational perspectives, but stressed the business perspective as the most effective gauge when measuring storage efficiency. “How are our investments in storage going to increase our productivity and our competitiveness?” he asked. “That’s the bottom line. What is it doing for us? Are we just hosting a bunch of data … that doesn’t really deliver any value and recovery?”

Toigo told attendees that Gartner recently predicted that the popularity of server virtualization technology would add to the problem — increasing storage capacity needs by 600%. The way for storage pros to counteract intimidating numbers like that, he said, was to use a broad definition when considering storage efficiency — but pay close attention to specific metrics.

From the engineering standpoint, he said, efficiency is the ratio of the output to the input of any system; from the business perspective, it’s a comparison of what is produced with what can be achieved with the same consumption of resources; and from the operational perspective, efficiency is defined as the skillfulness in avoiding wasted time and effort.

It’s also important to collect baseline data about storage, he said, but that’s a challenge for storage managers because of the wide range of storage systems running in data centers. “There are lots of different configurations for storage, a lot of different storage products, a lot of different standards for storage,” Toigo said.

Toigo advises storage admins to measure efficiency using five metrics: capacity allocation, capacity utilization, storage performance (I/O throughput), data protection (downtime avoidance) and storage energy.

Collecting these metrics is becoming increasingly more vital as the “non-trivial” challenges to storage efficiency continue to pile up. Toigo listed factors such as the neglect of data management, the narrow interpretation of storage management as capacity management, yielding to vendor “marketecture” over architecture, and storage administrators’ tendency to address problems by buying more hardware instead of addressing the source of the problem.

Instead of succumbing to these “tactics from the trenches,” Toigo advises developing a strategic storage plan, which involves a three-part, measurable process. First comes an analysis of the current state of company requirements, as well as current and future market and technology trends. Then, Toigo said, it’s important to assess the options to meet these requirements, in terms of time, budget and other business parameters. Finally, implement the plan in a manner that allows for ongoing testing. This strategy building process, Toigo contended, will ultimately enhance storage efficiency.

Several IT administrators at the seminar said they are evaluating ways to improve their storage efficiency. Keith Price, system administrator at Johnson & Wales University in Providence, R.I., said his IT team is looking to buy a new SAN to replace a system coming off support.

“We’re just trying to figure out how to figure out what we want,” he explained. “We’re doing that by doing what Jon said, benchmarking item by item.” Price’s department manages an extensive collection of databases on its SAN – Exchange and customer relationship management (CRM), for instance – as well as file systems.

System programmers Edith Allison and Michael Orcutt make up the enterprise storage team for the University of Connecticut, and are seeking ways to improve their storage from a price/performance standpoint as the university centralizes its IT operation.

“UConn is at a crossroads,” Allison said. “We have central IT, and the university has lots of little pockets of IT, and we’ve all just come together under one IT leader for the first time.”

The university has a Fibre Channel SAN, and the team manages 300 TB of data across all the academic units of the university. “We’re looking at how we are going to become a more efficient organization, how we’re going to save money. We’re a state agency, we have no money,” Allison said, laughing. “We’re a state and a public university, so it’s a double whammy.”

These events really do display the results of the hard work of talented people. Some products are organically developed while others result from the acquisitions of other companies and their talented people.

The messages are typically about new product capabilities, which may be iterative refinements to the current products or new products entirely. It is easy to get enamored with these – the result of both effective marketing and our insatiable desire to keep up with the next new thing.

But for IT organizations, the focus needs to be dialed back a bit to look at the requirements – both now and in the next three to five years. Some of the new things fit into this category while others will not. The importance is moderated by meeting specific requirements and putting a set of evaluation criteria in place with a decision process.

Vendor materials –marketing collaterals, specification sheets, case studies, and other realistic information — are useful in providing a foundation for understanding the product or solution. There really is no substitute for having a detailed knowledge base to work from, however.

There are two elementary starting points when looking at new products or technologies to meet the needs of an IT operation. You must ask if there is a capability that can only be done by using this product/technology, and if there is an economic advantage with a particular solution. There are unique products that hold their place because of what they bring. But for other products, there are competitive alternatives and sometimes tradeoffs that bring more considerations into the decision.

Economics for storage systems, including evaluating storage efficiency of the individual elements (see an explanation of this at Evaluator Group’s web site), are the most important considerations besides the fundamental requirements of storing information and providing access. Economics include the gains from doing things faster or more efficiently. If they can be quantified, then the decisions can be made and defended with more confidence.

It is necessary at times to remember that the focus on making storage decisions needs to be rooted in sound judgment with the best information and counsel available. The value provided by the products and the new generations and new technology improve the choices.

(Randy Kerns is Senior Strategist at Evaluator Group, an IT analyst firm).