Discussion on:

The brewing battle between hybrid and all-flash storage

Scott. You might want to check out Starboard Storage the pioneers of unified hybrid storage www.hybridstorage.com . Starboard provides a unified hybrid storage solution that is built from the grounds up to consolidate SAN and NAS. Unlike those you mention in the hybrid space in your article Starboard is not an OEM of a platform like ZFS and from a scope perspective addresses a broader market than the iSCSI and VMWare only platforms you mention. When you leverage a hybrid architecture the best results are gained from consolidating more workloads because the more workloads you virtualize your SSD and HDd across the more effciiently you are leveraging your valauble resources. to back up your piece here is a blog I wrote of the subject http://blog.starboardstorage.com/blog/bid/189631/Unified-Hybrid-Storage-The-best-use-of-SSD . Full disclosure. I work for Starboard Storage Systems.

I appreciate Scott's perspective and agree that hybrid storage solutions will work fine for many applications that are not performance (IOPS) constrained. Where I would tend to disagree is along 2 fronts. One is that the need for all-flash is an "outlier" case. I agree that few environments need 1M+ IOPS, but the number of environments that can significantly benefit from a solution that delivers 50K, 100K or even 200K IOPS are not outliers and represent a very large segment of the storage market. Today's hybrids can't provide this level of performance. A good example is VDI. If you only are enabling less than 200 VDI users, yes a hybrid may work ok, but to guarantee performance for 200+ user environments, an all-flash solution may be mandatory, especially for persistent desktop environments. The 2nd area of disagreement is on cost per GB. All-flash devices with significant useable capacity for the mid-market can be acquired for well under $50K. The prices of flash devices are coming down dramatically, much faster than HDDs. Some flash vendors already claim they offer the same $/GB of HDDs. It's blatantly clear that HDDs will be going the way of vacuum tubes for I/O-intensive apps. We can debate the timing, but hybrid solutions are interim solutions at best. A better and less disruptive approach is to add an all-flash device (like ViSX from Astute Networks where I work) into an existing I/O intensive environment as a complementary solution to an existing traditional storage environment. Just move those applications or virtual machine datastores to the all-flash device (typically in a matter of minutes with Storage vMotion) and users can see an immediate 5X-10X performance boost without having to replace their existing storage systems.

Hybrid Arrays are designed for the mainstream use cases where customers want to consolidate multiple workloads. They happen to use Memory and SSD as cache today and HDD behind because that is the sweet spot for building the best balance of $/GB and $/IOP.

All-Flash arrays believe that SSD takes over the world are are making a bet on technology for high performance.

The reality is that Hybrid arrays do not have to use Flash and SSD. They could use other media. It all depends what enables you to hit the customer sweetspot.

If all capacity moved to Flash tomorrow and all write amplification issues solved their would still be high speed, high cost flash and lower cost higher capacity flash. Good system design would need to design a hybrid solution with caching to satisfy the market for consolidation and centralization of storage at the right price. It would be a hybrid System that was all flash.

The fact is that today Hybrid Storage vendors are focussed on mainstram IT needs and are leveraging the appropriate technology to satisfy the use case and all Flash vendors are focussed on the high performance use case.

Unfortunately it looks like the all-flash guys thought that they would be more mainstream and only competing with traditional stroage vendors and because hybrid solutions are spoiling their party they are trying to sell a religious technology argument for mainstream IT use cases.

And as far as the cost per GB being equal that is baloney. The all flash vendors discount two things:-

1) Data reduction technologies can also work on HDD
2) Many workloads do not dedupe or compress well. In fact those workloads are the fasted growing in capacity

What this means is HDD will be around for a good while and even when/if it disappears you still need hybrid architectures to satisfy the broad set of technology needs.

When you attempt to clone to a hybrid drive, you will find frequent failures because the cloning program doesn't handle the hybrid well. Copying files to it is a breeze, just cloning has problems. So either find a program that can clone to a hybrid drive (let me know if you do), or just create your partitions and copy to them.
w7hd@msn.com

NAND wears out. MLC wears out faster than SLC. The vendors obfuscate which type they use. New wear leveling algorithms help, but cannot eliminate this issue.
NAND can only block erase, this is the reason for write performance issues.
Do not go into Flash based storage without this understanding.
http://www.eetimes.com/design/memory-design/4390427/-SLC-vs-MLC--Which-works-best-for-high-reliability-applications-

P.S. Cobalt doped gamma hematite drives have been gone for over a decade so "Spinning Rust" no longer applies.

I agree with Appaccelerator1, the HDD as we know it will go the way of the dinosaur. It will hang around for quite some time, just like the 1.44" diskettes. But at some point the "mega-cache" will be cheaper than the mechanical parts, and even hybrids will become extinct. Although with a closed Helium case for hd's you might extend the hybrids a few more years.

But it's not only price and performance that matter, it's also physical size. With smartphones, pads and laptops taking over the consumer market, and PC's as we know them are phased out; size will matter! The traditional HD will probably never get smaller, but SSDs will.

makes more sense when it provides real world examples. What does a chain restaurant need? What does a large bank need? What does a university need? What about a company like Google? Or a company focused on salvage with a state-wide presence? How about a big city school district? And county, state, or federal agencies?

Because space limitations require a short article, the choices should be limited to two, one that is likely to choose HDD but would benefit from the hybrid choice, and one that might think it should choose flash but would actually benefit from the hybrid.

While the comparisons are nicely presented by Scott, I have to agree with appaccelerator on key points about flash vs. HDD. If you just compare Moore's law to Kreider's law (which defines the capacity growth of HDD), there isn't much doubt that in the end, semiconductors will win this fight. Just like tape, there will always be a few applications where flash doesn't make sense.

Here's a link to a blog by Pure Storage's CEO that explains how to understand what happens with performance on hybrid arrays:
http://www.purestorage.com/blog/the-risk-of-over-promising-and-under-delivering-with-hybrid-storage-arrays/

Appreciate not every application in the world "needs" the performance of flash, but if you can get it for the same price or close, why would you choose HDD?

I understand that there are a ton of variables and the 'right' solution would have to be carefully thought out, but I'd still like to know some relative differences between the various technologies. How much faster is SSD versus HDD? How close is the hybrid to pure SSD speedwise? An entry level solution of 200,000 for an SSD array was discussed in the article (and disputed in the comments). How much would it cost for comparable capacity in both HDD and hybrid?