With flash, writes are made to an entire cell of memory; flash devices can’t overwrite a few hundred bytes like a hard disk drive can. When a flash device is new (referred to as the “fresh out of the box,” or “FOB,” state), this fact is transparent since each new write goes to an already-empty flash cell. But after it’s been filled to capacity, in which case all the cells have been written to once, it’s a different story. This condition (known as “steady state”) requires that before each new write occurs, the controller must erase a cell to make room for it. In this process, called “garbage collection,” the controller copies segments of data that need to be saved and then erases the entire cell.

The result of this process is that steady-state performance is much slower than FOB, although it’s still orders of magnitude better than hard drives. But when two flash devices are being tested, it’s imperative that the steady-state condition be established, since this is where the device will operate for most of its life. Aside from impacting performance, this extra copying process also increases the number of write cycles or “program/erase” (P/E) cycles the device experiences, which impacts endurance.

Originally designed for consumer use in memory sticks and cameras, etc. (by Toshiba, we learned on a briefing call recently), there was no need for flash to be written and rewritten to a large number of times. But when it started being used in the enterprise space, flash endurance became an issue. Enterprise applications, especially the ones that are performance-critical enough to justify the expense of flash storage, generate a lot of read/write activity. Database transaction logs and caching, for example, generate a lot of IOPS, the performance metric most often associated with flash applications.

Flash designers have addressed this problem in a number of ways, but it’s imperative that users (and their VARs) understand how much write traffic they expect to generate on a daily basis. Then they can compare that with a standard endurance metric called “total bytes written” (TBW) to calculate how long that flash device will last in their environment.

Ask your flash vendors about their products’ performance and endurance. See how much they know about these numbers and what they mean and find out if the products you’re selling or looking to sell are going to make your customers happy in the long term. You can also set customers’ expectations around flash as a technology and demonstrate your value as a trusted storage advisor in the process.

We’ll talk more about flash in an upcoming blog. In the meantime, here are some video links on Storage Switzerland about flash performance and endurance.

Follow me on Twitter: EricSSwiss

This last entry in the top 10% list of storage-related technologies from 2011 will be devoted to solid-state storage devices. I’m going to refer to this broad collection of NAND flash storage products as SSDs, although that term technically stands for “solid-state drives.” SSDs continued to grow as a segment during 2011, with additional vendors, products, customers and use cases. The links in this blog go to write-ups we did from briefings at the Flash Memory Summit, VMworld and SNW this past year.  

SSDs

As the name suggests, SSDs originally appeared as plug replacements for spinning disk drives. Flash memory chips in disk drive packages were put into existing disk array chassis by all of the major storage vendors. Users also put SSDs into servers to replace boot drives or to provide a local performance boost. But the area we’ve seen the most growth is at the board level, especially PCIe-based products and one innovative implementation using DIMM modules.

Putting flash into the server is appealing because it moves this fast storage area closer to the action (the CPU), important for performance, and can be easier to implement since it’s not shared between multiple servers. It also eliminates potential network problems. Finally, it’s ideal for caching (see below), the “killer app” for implementing flash as a storage product.

Cache is king

Placement is the question that users have to ask when looking at SSDs. Fast storage only benefits data that’s actually on it (duh), and flash is still expensive enough to keep it in relatively short supply in the storage infrastructure (another duh). So the method for getting the most appropriate data onto the flash device and keeping it there is a key determinant of the performance benefit users will see.

Tiering’s focus on larger blocks of data and less frequent moves fits the disk drive industry it was developed for. Caching, on the other hand, involves smaller data objects and much faster movement, more fitting of flash memory. You could say that tiering is the way humans would tackle the data placement issue and caching is how computers would do it.

Caching is a software process that can be run almost anywhere there are CPU cycles and available memory. Currently available products run caching on PCIe cards, in the host server, in the storage array, on the array controller card or (obviously) in the controller of a dedicated caching appliance that’s connected to the network.

Read caching refers to the practice of copying data objects to a faster storage area with the intent of making them available for read operations in the shortest time possible. Write caching serves to stage data from a write operation on faster storage and acknowledge the write so that the application can move on instead of waiting for a disk system to complete the write.

Read caching represents the low-hanging fruit with regard to performance impact, since most applications do more reads than writes. It’s also much easier to implement, which helps explain why every caching product we’ve looked at offers read caching, and far fewer do write caching. That said, write operations are significantly slower than reads, so the potential for performance improvement is there.

Caching is often implemented within an SSD solution, and I’d expect that trend to continue. At some point caching will probably become like deduplication, thin provisioning or one of the other embedded storage services. An understanding of how it works and what options are available will be important as part of the evaluation process for solid-state storage.

Follow me on Twitter: EricSSwiss

Unlike HDDs, the write process for SSDs is done a block at a time, and space must be cleared ahead of each write operation. This means a “garbage collection” step must occur before each block of data is committed to flash, adding to the overall write cycle time. When SSDs are new — in the “fresh out of box” state — they’re essentially empty and can conduct data writes without taking the garbage collection step. Obviously, performance results gathered during this initial period before the flash device is filled for the first time are not indicative of how it will perform over the long term.

To perform an SSD test, the SSDs must be in a “steady-state” condition, where the device has been filled and refilled enough so that every write operation requires a consistent garbage collection step. For more information about this “pre-conditioning” process, the SNIA website is a good resource. It’s also important to run the SSD test in an environment as similar to the expected production environment as possible, including servers, storage and applications. Also, test one drive at a time and test them continuously; don’t start and stop tests.

Drive reliability and endurance are also different on SSDs. The NAND substrate used in flash SSDs has a finite number of write cycles — called program/erase (P/E) cycles — it can endure. Each manufacturer rates SSDs for this endurance level, which can also be expressed as total bytes written and calculated as a product of maximum P/E cycles and the capacity of the drive. SSDs also leverage the self-monitoring, analysis and reporting technology (SMART) standard for drive diagnostics, which provides a number of drive attributes including the net number of P/E cycles remaining. Published drive endurance can be confirmed by comparing this remaining P/E cycle number with the rated drive maximum after a controlled test period.

For VARs, SSDs represent an opportunity to show customers their value. The disparity that exists between the quality of products and of vendors means that customers will be relying on their VARs to help them navigate the marketplace. Besides identifying the best vendors and products through an SSD test, VARs will need to help customers set reasonable expectations for SSD deployment.

Follow me on Twitter: EricSSwiss

Increasing storage system performance is usually the reason solid-state storage devices are first considered. Performance, especially IOPS, for SSDs is typically an order of magnitude (at least) greater than HDDs for writes and even better on reads. This read/write differential is due to the fact that SSDs must erase blocks of storage space ahead of each write, a process called garbage collection, which adds a significant amount of time to the write cycle. When an SSD is first used, referred to as FOB, or “fresh out of box,” it can accept writes without running this erase cycle, since it’s essentially empty. After the device has had all its NAND cells filled, it must run garbage collection before each write and is then in a “steady state” condition. A write saturation curve can show this graphically, how performance degrades as the device moves from FOB to steady state. Obviously, it’s essential to measure performance only after reaching steady state.

SSD endurance is the other main factor to consider when comparing SSDs. Unlike spinning disk media, NAND flash has a finite number of writes it will accept before reliability suffers. Called program/erase, or P/E, cycles, this statistic can give an accurate assessment of when an SSD will wear out. Manufacturers know the maximum number of P/E cycles their devices can sustain and use this to compute a total bytes written (TBW) spec. When comparing SSDs, TBW figures should be comparable.

There are some other factors to consider when comparing SSDs, like the knowledge of the vendor and its ability to provide reliable information that, interestingly enough, can help users make good product choices. But generally, there’s a lack of standards in the industry, in the terminology used for different specs as well as in the data reported. This means that published specs offer only a starting point, a way to narrow the list of candidates. In-house testing with an environment similar to that used in production is usually warranted, something we’ll address in another post.

Follow me on Twitter: EricSSwiss