Server-side caching or tiering systems essentially augment hard disk drive (HDD) performance; they don’t replace disk drive arrays. This means they need to run a process to analyze and determine which data sets should be on flash and then move those data at the appropriate time. The net effect of these processes can be increased system complexity and reduced overall performance.

For tiering systems, this analysis is typically applied only after data’s been written to the HDD tier and can involve a “warming” period where multiple accesses are analyzed. Moving those data at the appropriate time also creates storage controller overhead, especially on the hard disk array that it’s supporting.

But there’s another issue: flash endurance. As we covered in the first blog post of this series, “Server-side flash implementation explainer,” flash memory can support a finite number of write and erase cycles. The data turnover of caching or tiering consumes more of these cycles, shortening the effective useful life of flash devices.

All-flash arrays are an alternative to caching and tiering systems that can address these issues.

All-flash arrays are complete storage systems with storage controllers, modular flash capacity and storage services (RAID, snapshots, cloning, replication, etc.). They also can support multiple storage protocols — either block storage or both block and file storage. They use storage controllers that are designed for flash’s performance, breaking free from the limitations of putting flash in legacy disk arrays that used HDD controllers designed around the latency of spinning disk drives.

These modular systems implement like traditional disk arrays, providing data centers with flash performance without the complexity and potential side effects of caching or tiering. They also leverage flash’s performance to bring acquisition costs closer to that of the high-performance HDD arrays they’re replacing.

All-flash arrays also include data reduction technologies like thin provisioning, deduplication and compression to decrease the amount of capacity they need to store a given data set. Unlike HDD arrays, flash-only systems have plenty of performance to accommodate the CPU load of these processes. This can bring typical data reduction rates up to 10 times or more, driving down their effective cost per gigabyte. When compared with the disk drive arrays they’re replacing, typically high-performance SAS or Fibre Channel systems in high-spindle-count configurations, flash-only arrays can be more than cost-competitive.

Nimbus Data Systems was one of the first on the market with an all-flash array but has been joined more recently by others, like Pure Storage. All-flash systems are not for every environment but are certainly a product that should get serious line-card consideration for VARs interested in embracing flash as part of their storage go-to-market strategies. As a disruptive alternative to legacy high-performance storage arrays, they can be an ideal way to get into a new account or unseat an incumbent vendor.

Follow me on Twitter: EricSSwiss

Flash is getting more affordable but is still several times the cost per gigabyte of hard disk drives (HDDs). Because of the cost disparity, it’s often used to augment HDD performance: The more performance-critical data sets are placed on flash, sometimes temporarily, to take advantage of its orders-of-magnitude better performance, especially IOPS.

Read caching involves placing a copy of the most frequently used data objects into SSD to speed access times by applications or users. When data’s changing, it’s more complicated (write caching) since the primary data set must be kept updated and writes must be protected against system failures until they’re committed to nonvolatile storage. But there are algorithms that can do this too, as part of the OS or application or as a part of a PCIe card that houses the flash storage itself.

Caching can be the simplest to implement, since it leaves the primary copy of data intact on existing disk storage, often operating transparently to the application. It can also be implemented with a relatively small amount of SSD capacity, since data can be moved into and out of the cache area rapidly.

Tiering is like caching except it involves moving an entire application or data set into cache and then copying back to primary storage when the period of high activity is over. For this reason, tiering typically requires more SSD capacity than caching and may involve configuration changes to the application.

Server-side flash implementations can be done with PCIe flash devices, which can have up to a terabyte or more of flash capacity and may include caching software as well. Flash can also be in SAS or SATA drive form-factor packages, which plug into 3.5-inch, 2.5-inch or 1.8-inch drive slots. There are also SSDs that plug into an empty DDR3 memory slot on the motherboard and connect via a SATA cable.

Server-side flash is dedicated to the server it’s installed in, meaning less flash capacity is required than with array- or network-based flash devices and implementation is simpler than in those shared storage scenarios. Although flash capacity is significantly more expensive than HDDs, SSDs caching or tiering can actually provide a lower-cost alternative, with better performance.

For VARs, server-side SSD implementation can be an ideal way to break into a new account or capture new business in an existing account that’s currently going to an array vendor. Whether implemented as a cache or tier or just a high-performance storage area, server-side flash can provide an immediate solution for a slow application.  For other use cases, an all-flash array or flash appliance may be a better alternative. We’ll look at those in another post.

Follow me on Twitter: EricSSwiss

This last entry in the top 10% list of storage-related technologies from 2011 will be devoted to solid-state storage devices. I’m going to refer to this broad collection of NAND flash storage products as SSDs, although that term technically stands for “solid-state drives.” SSDs continued to grow as a segment during 2011, with additional vendors, products, customers and use cases. The links in this blog go to write-ups we did from briefings at the Flash Memory Summit, VMworld and SNW this past year.  

SSDs

As the name suggests, SSDs originally appeared as plug replacements for spinning disk drives. Flash memory chips in disk drive packages were put into existing disk array chassis by all of the major storage vendors. Users also put SSDs into servers to replace boot drives or to provide a local performance boost. But the area we’ve seen the most growth is at the board level, especially PCIe-based products and one innovative implementation using DIMM modules.

Putting flash into the server is appealing because it moves this fast storage area closer to the action (the CPU), important for performance, and can be easier to implement since it’s not shared between multiple servers. It also eliminates potential network problems. Finally, it’s ideal for caching (see below), the “killer app” for implementing flash as a storage product.

Cache is king

Placement is the question that users have to ask when looking at SSDs. Fast storage only benefits data that’s actually on it (duh), and flash is still expensive enough to keep it in relatively short supply in the storage infrastructure (another duh). So the method for getting the most appropriate data onto the flash device and keeping it there is a key determinant of the performance benefit users will see.

Tiering’s focus on larger blocks of data and less frequent moves fits the disk drive industry it was developed for. Caching, on the other hand, involves smaller data objects and much faster movement, more fitting of flash memory. You could say that tiering is the way humans would tackle the data placement issue and caching is how computers would do it.

Caching is a software process that can be run almost anywhere there are CPU cycles and available memory. Currently available products run caching on PCIe cards, in the host server, in the storage array, on the array controller card or (obviously) in the controller of a dedicated caching appliance that’s connected to the network.

Read caching refers to the practice of copying data objects to a faster storage area with the intent of making them available for read operations in the shortest time possible. Write caching serves to stage data from a write operation on faster storage and acknowledge the write so that the application can move on instead of waiting for a disk system to complete the write.

Read caching represents the low-hanging fruit with regard to performance impact, since most applications do more reads than writes. It’s also much easier to implement, which helps explain why every caching product we’ve looked at offers read caching, and far fewer do write caching. That said, write operations are significantly slower than reads, so the potential for performance improvement is there.

Caching is often implemented within an SSD solution, and I’d expect that trend to continue. At some point caching will probably become like deduplication, thin provisioning or one of the other embedded storage services. An understanding of how it works and what options are available will be important as part of the evaluation process for solid-state storage.

Follow me on Twitter: EricSSwiss

The old standby, storage tiering, is certainly one way to do this, and most major storage systems allow users to create a “Tier 0” for SSDs. But this method may not be able to keep up with how quickly data access requirements change, resulting in underutilized SSD investments and disappointed users. SSD caching may provide the answer.

A flash storage caching solution monitors data traffic between the CPU and storage devices and determines which subsets of data are the most active. It then copies those data blocks or files to the high-speed cache area, giving applications data access at silicon speeds. Read caching is the most common and provides the majority of performance improvements since most applications generate a much higher percentage of read than write transactions. Write caching is more complicated and requires separate steps to ensure that data is protected throughout the write process.

Flash SSD caching is typically implemented as a piece of software that runs on the host, on a PCIe card or on the RAID controller card. There are also caching solutions built into networked storage devices and even into applications. Server-based solutions typically allow any internal or server-attached SSD capacity to be used as the cache pool, making this an easier product to implement than caching solutions that only use dedicated storage capacity.

This technology has a number of advantages over tiering solutions for solving the data placement issue. Caching typically monitors data access continuously, making it a good fit for dynamic data sets that may need SSD performance for only a short period of time. Also, caching solutions are easy to implement since they’re usually application-agnostic. Most move data at the block level and don’t require APIs or integration with existing software.

At the Flash Memory Summit in Santa Clara, Calif., last week, it was all about SSD cache. Storage Switzerland was briefed by five companies that had read and write caching products. For VARs, caching can be the piece that’s been missing from their SSD solution sets. As many early users have found out, just putting SSDs into a server or storage array doesn’t magically improve performance—at least not to the extent most have expected. They need to solve the data placement question. This is one their VAR can answer, with an appropriate SSD caching solution.

Follow me on Twitter: EricSSwiss

Increasing storage system performance is usually the reason solid-state storage devices are first considered. Performance, especially IOPS, for SSDs is typically an order of magnitude (at least) greater than HDDs for writes and even better on reads. This read/write differential is due to the fact that SSDs must erase blocks of storage space ahead of each write, a process called garbage collection, which adds a significant amount of time to the write cycle. When an SSD is first used, referred to as FOB, or “fresh out of box,” it can accept writes without running this erase cycle, since it’s essentially empty. After the device has had all its NAND cells filled, it must run garbage collection before each write and is then in a “steady state” condition. A write saturation curve can show this graphically, how performance degrades as the device moves from FOB to steady state. Obviously, it’s essential to measure performance only after reaching steady state.

SSD endurance is the other main factor to consider when comparing SSDs. Unlike spinning disk media, NAND flash has a finite number of writes it will accept before reliability suffers. Called program/erase, or P/E, cycles, this statistic can give an accurate assessment of when an SSD will wear out. Manufacturers know the maximum number of P/E cycles their devices can sustain and use this to compute a total bytes written (TBW) spec. When comparing SSDs, TBW figures should be comparable.

There are some other factors to consider when comparing SSDs, like the knowledge of the vendor and its ability to provide reliable information that, interestingly enough, can help users make good product choices. But generally, there’s a lack of standards in the industry, in the terminology used for different specs as well as in the data reported. This means that published specs offer only a starting point, a way to narrow the list of candidates. In-house testing with an environment similar to that used in production is usually warranted, something we’ll address in another post.

Follow me on Twitter: EricSSwiss