By pairing SSDs in the drive capacity with automated tiered storage, users only purchase the hardware they need to house their active data. They get better performance and utilization benefits than by using SSDs for cache. Automated tiered storage increases performance by keeping frequently accessed blocks of data on tier 0 SSD storage, and moves less frequently used blocks to less expensive tiers like SATA. Without automated tiered storage, users would be required to purchase SSDs for entire volumes, driving up their costs and not full taking advantage of the performance benefits.

The answers to Barry’s questions are

1. NAND write performance is irrelevant for flash used as a NetApp read cache because NetApp’s architecture allows it to cache reads initially in its stable DRAM cache and then destage them to flash as convenient to free up the DRAM cache for new entries (that’s likely what NetApp meant when it characterized the flash as a ‘victim’ cache, though its choice of victims might not be the the conventional one if, for example, it decided to evict the data *most* likely to be read again in order to maximize flash longevity by minimizing the flow through it).

Interestingly, leveraging the existing stable DRAM cache was Barry’s own answer when applied to EMC’s use of flash for drives rather than cache - but not only did he ignore its applicability to NetApp’s case but he also ignored the fact that his original question seemed to be about *read* caching (which of course EMC’s drive-only flash approach does not support at all with flash: if you want the benefits of flash-level access performance from EMC, *all* the data that needs it must reside on its flash drives).

2. I just offered one hypothetical way that flash cache wear-out might be alleviated above. Beyond that, the victims can be destaged to flash cache in large contiguous blocks (in a manner very similar to that used by NetApp to write data efficiently to disk - see Patrick’s reference to ‘new wear-leveling algorithms’ above), thus further minimizing flash cache write activity. And for a read cache the worst thing that can happen if a portion of the flash *does* wear out is that the data must then be fetched from disk, since the NetApp architecture is already set up to detect any such corruption even if the flash itself does not.

Contrast this with conventional approaches such as EMC’s in which data must be destaged in chunks far smaller than the block-erase size (at least in the kinds of small-request-intensive environments under discussion here) to specific logical locations in the flash, wholly relying upon the flash’s internal wear-leveling algorithms to attempt to mitigate the resulting heavy erase/rewrite activity. In some situations, this could result in faster wear-out for the backing flash storage than NetApp’s use as a cache would.

3. It’s not clear why NetApp would want to use flash as a write-back cache given how well their existing DRAM cache plus bulk-write technology handles this already. It would be interesting, however, to know exactly how EMC professes to deal with the problem of data integrity in its flash *drives*, since it’s difficult to see how it can ensure it against the kinds of problems that Barry envisions for cache write-back flash use without performing the same kind of mirroring and read-after-write verification that would be required of a flash write-back cache.

If one doesn’t take (as you apparently did above) Barry’s comments as being specifically directed at NetApp’s use of flash cache, one can perhaps be at least slightly more forgiving of them: they could (though it’s hardly clear that they do) apply to some other vendor’s use of flash as cache if it were implemented sloppily (and indeed that’s a specialty of EMC spin: hypothesize the worst and let the opposition defend against it rather than make any attempt to evaluate competing products on their actual merits). However, the assumption that *anyone* will treat flash as an extension of ECC RAM rather than as a caching tier (even if it’s directly connected to the system bus rather than out in the storage stack) that needs to be treated with the same skepticism as other storage tiers seems pretty far-fetched to me.

Flash only modifies the existing set of trade-offs between price and performance in storage. Just as DRAM SSDs have had an enterprise niche for years, flash drives will have one for applications that need all the performance that they can get at almost any price (remove that ‘almost’ and they’ll still use DRAM SSDs). Just as large amounts of DRAM have served effectively as (usually read) cache, so will even larger amounts of flash allow more mainstream applications to leverage the performance advantages of cache without the cost of DRAM (though where the DRAM can be shared flexibly between caching and other system uses it has added value for its higher cost).

As long as flash costs over an order of magnitude more per GB than disk storage only a minority of back-end storage applications will exist for flash in large quantities (though for small installations where its cost is a relatively small proportion of the total it should enjoy increasing acceptance). So NetApp’s use of it for cache during this period (however long it may last) makes perfect sense.

- bill