Storage

The brewing battle between hybrid and all-flash storage

Scott Lowe breaks down the advantages and disadvantages of solid state and hard disk drives for storage and explains why a hybrid solution brings you the best of both methods.

If I've learned anything in the past few years, it's that the storage market is a fluid one, particularly as flash-based options have grown in prominence, resulting in a number of new players in two primary market segments: Hybrid storage and all-flash storage.

  1. Hybrid storage. Those vendors selling hybrid storage devices include newcomers Nimble, Tegile, Tintri and others. In a hybrid storage system, the array includes both solid state and rotational storage. In such scenarios, the solid state storage is often used as a mega cache which provides significant acceleration of all reads and writes from and to the array. There are a number of established players that produce product in the hybrid space, too. However, in many cases, support for solid state storage is added to a legacy platform. The newcomers generally have the greenfield advantage here as they can build systems with flash in mind and can sometimes take more advantage of flash's unique characteristics. Note that there are other vendors in this space, but the space changes every day!
  2. All-flash storage. An all-flash storage system is exactly what it sounds like. It's an array that contains no rotational storage whatsoever. All data is stored on solid state drives. Upstart vendors in this space include Pure Storage, Nimbus, and Whiptail.

Solid state drives (SSDs) versus hard disk drives (HDDs)

Before we start discussing the pros and cons of hybrid vs. all-flash arrays, let's make sure that the differences between flash drives and solid state is understood.

HDDs

First of all, many people refer to HDDs as "spinning rust" due to the nature of the drives. In short, they sit around and spin all day long, accepting instructions to read and write data as needs warrant. HDDs have been around for decades and have undergone a ton of growth, now reaching capacities of 3 TB and more. However, in the past decade, HDDs have not kept pace from a performance perspective. After all, 15K RPM drives have been around for a decade and they're the fastest HDDs out there right now.

As applications and new ways for doing IT (read: virtualization) began to come into play, IT departments found themselves having to constantly add new disks, not to meet capacity needs, but to meet needs for an ever-growing number of IOPS.  Individual hard drives simply do not provide much in the way of performance. It takes many, many drives to achieve necessary performance levels for many organizations.

From a capacity perspective, however, HDDs are just fine. Modern drives are massive and can hold a ton of data, but they just can't do it quickly.

If you'd like to learn more about hard drive performance, RAID and arrays, visit this article that I wrote a couple of years ago here at TechRepublic.

SSDs

After topping out at 15,000 RPMs, rotating hard drives simply sat still on performance for almost a decade. In recent years, the storage industry has turned increasingly to solid state drives in order to solve what was becoming a crisis with regard to storage performance. Whereas HDDs provide massive capacity with relatively poor performance, solid state drives offer massive performance with relatively low capacity. The table below helps explain where HDDs and SSDs are different from a metrics perspective:

HDD

SSD

Cost per gigabyte ($)

Very low

Very high

Cost per IOPS ($)

Very high

Very low

As you can see, there isn't much of a middle ground. You either buy for performance (SSDs) or you buy for capacity (HDDs).

If you'd like to learn more about solid state storage, please visit this article.

An array of solutions

Most companies don't buy single hard drives for storage. Instead, they buy arrays that house a lot of drives and then combine these drives together using RAID or some other mechanism.  With the advent of solid state storage, a storage buyer now has a number of new opportunities.  No longer does he simply need to buy more rotating spindles to meet his performance goals.  Now, he can buy fewer solid state drive arrays to meet those challenges.  Except that doing so comes at a cost: If he buys an all flash array, he's buying a lot of expensive capacity for applications that may not need expensive capacity.

HDD only

Hybrid (HDD + SSD)

SSD only

Cost per gigabyte ($)

Very low

Medium

Very high

Cost per IOPS ($)

Very high

Medium

Very low

Today, there are data centers that are clamoring for better storage performance. However, even though some vendors may hype the fact that they can do a million IOPS in 2U of rack space, how often do most organizations need that kind of performance? Unless you consider niche cases (i.e. analytics, business intelligence), most organizations don't need millions of IOPS.

What's emerging are really three distinct markets.

Traditional

A traditional environment consists of just rotational storage and offers amazing capacity, but with the tradeoff being low levels of performance. These solutions remain popular today as many CIOs focus on cost per gigabyte at the expense of low IOPS. While this kind of solution may be viable for "cheap and deep" storage, it's increasingly unsuitable for real workloads, particularly as organizations begin to look at trends like VDI.

These kinds of arrays are often priced at the low end of the market.

Flash only

At the other end of the spectrum, we have players that are building arrays that have nothing but solid state drives. These devices forsake capacity for massive performance and there are a number of good reasons that businesses need the IOPS, but it becomes more likely that they will run out of disk space at some point.

In order to combat the space issue since SSDs are much smaller than HDDs, many emerging all-flash vendors have turned to data reduction technologies, including compression and deduplication. With such technologies, vendors are seeing significant success in stretching the capacity of their all flash arrays beyond the hard capacity of the units, and offering customers a deduplicated lower price per GB than would be possible without the data reduction techniques being used.

However, I see two serious challenges:

  1. Most CIOs simply don't need the levels of performance offered by all-flash arrays. The broad swath of the market simply needs affordable storage that runs at very good levels of performance. While all flash supports the performance need, it still remains much more expensive than HDDs, even when data reduction is taken into consideration.
  2. The initial acquisition cost of an all-flash array can be staggering and is often well into the six figures and above $200,000. For many midmarket companies, it's not possible to get this kind of capital. And, still, the cost per GB is still higher than necessary.

Hybrid

So, what's the answer?

In my opinion, the answer is to combine the best of what both the SSD and HDD worlds have to offer and create an array that leverages the benefits of solid state storage for performance while continuing to be able to rely on the massive capacity benefits inherent in HDD-based solutions.

That's exactly what the growing hybrid market thinks, too.

There are a number of emerging storage companies that have entered the market in recent years and each brings with them a "secret sauce" or a differentiator that makes their solution unique. At the same time, these players are focusing on the balance that must come when one looks at the two primary storage metrics: $/GB and $/IOPS. A hybrid array very nicely balances these two metrics and can satisfy a wide swath of business needs. Sure, there may still be outlier cases that demand all SSD or all HDD solutions, but these are outlier cases. The cost per GB for a hybrid solution might be a little more than it is for an all-HDD solution, but will be much less than an all-flash solution.

Today's storage buyers are always looking for solutions that provide both capacity and performance value and do so in a way that makes budgetary sense. Because most of the new hybrid players rely on commodity hardware and differentiate themselves in software and because they don't require as many expensive SSDs, hybrid arrays carry a price tag that is a fraction of what one would find with an all-SSD product. While it won't beat the all-SSD array in a head to head test, when one measures the performance based on true needs, hybrid storage is often the perfect solution.

Take, for example, VDI.  A hybrid array in this scenario is a perfect fit. Because commonly accessed files are cached on super-fast SSDs, boot and login storms are a thing of the past. Further, deduplication savings in VDI is incredibly high, so you may find yourself using much less disk space than you need .In addition, because there is ample capacity in a hybrid device, an organization can consider allowing users to create persistent, customizable desktops. There are simply a number of opportunities that become available.

Summary

As you can tell, I'm pretty passionate about the hybrid market and have been for quite some time. While all-flash arrays are really, really cool and I'd love one for my lab, in challenging budget times, CIOs must think practically about the solutions they implement and I'd urge them to take one of the newer market players for a test drive. Some of the companies will provide loaner/test gear for potential new customers, so that you can take one of units for a spin.

About

Since 1994, Scott Lowe has been providing technology solutions to a variety of organizations. After spending 10 years in multiple CIO roles, Scott is now an independent consultant, blogger, author, owner of The 1610 Group, and a Senior IT Executive w...

13 comments
Fairbs
Fairbs

I understand that there are a ton of variables and the 'right' solution would have to be carefully thought out, but I'd still like to know some relative differences between the various technologies. How much faster is SSD versus HDD? How close is the hybrid to pure SSD speedwise? An entry level solution of 200,000 for an SSD array was discussed in the article (and disputed in the comments). How much would it cost for comparable capacity in both HDD and hybrid?

ludauber
ludauber

While the comparisons are nicely presented by Scott, I have to agree with appaccelerator on key points about flash vs. HDD. If you just compare Moore's law to Kreider's law (which defines the capacity growth of HDD), there isn't much doubt that in the end, semiconductors will win this fight. Just like tape, there will always be a few applications where flash doesn't make sense. Here's a link to a blog by Pure Storage's CEO that explains how to understand what happens with performance on hybrid arrays: http://www.purestorage.com/blog/the-risk-of-over-promising-and-under-delivering-with-hybrid-storage-arrays/ Appreciate not every application in the world "needs" the performance of flash, but if you can get it for the same price or close, why would you choose HDD?

jdm12
jdm12

makes more sense when it provides real world examples. What does a chain restaurant need? What does a large bank need? What does a university need? What about a company like Google? Or a company focused on salvage with a state-wide presence? How about a big city school district? And county, state, or federal agencies? Because space limitations require a short article, the choices should be limited to two, one that is likely to choose HDD but would benefit from the hybrid choice, and one that might think it should choose flash but would actually benefit from the hybrid.

TylerD75
TylerD75

I agree with Appaccelerator1, the HDD as we know it will go the way of the dinosaur. It will hang around for quite some time, just like the 1.44" diskettes. But at some point the "mega-cache" will be cheaper than the mechanical parts, and even hybrids will become extinct. Although with a closed Helium case for hd's you might extend the hybrids a few more years. But it's not only price and performance that matter, it's also physical size. With smartphones, pads and laptops taking over the consumer market, and PC's as we know them are phased out; size will matter! The traditional HD will probably never get smaller, but SSDs will.

paul.willy
paul.willy

NAND wears out. MLC wears out faster than SLC. The vendors obfuscate which type they use. New wear leveling algorithms help, but cannot eliminate this issue. NAND can only block erase, this is the reason for write performance issues. Do not go into Flash based storage without this understanding. http://www.eetimes.com/design/memory-design/4390427/-SLC-vs-MLC--Which-works-best-for-high-reliability-applications- P.S. Cobalt doped gamma hematite drives have been gone for over a decade so "Spinning Rust" no longer applies.

w7hd
w7hd

When you attempt to clone to a hybrid drive, you will find frequent failures because the cloning program doesn't handle the hybrid well. Copying files to it is a breeze, just cloning has problems. So either find a program that can clone to a hybrid drive (let me know if you do), or just create your partitions and copy to them. w7hd@msn.com

appaccelerator1
appaccelerator1

I appreciate Scott's perspective and agree that hybrid storage solutions will work fine for many applications that are not performance (IOPS) constrained. Where I would tend to disagree is along 2 fronts. One is that the need for all-flash is an "outlier" case. I agree that few environments need 1M+ IOPS, but the number of environments that can significantly benefit from a solution that delivers 50K, 100K or even 200K IOPS are not outliers and represent a very large segment of the storage market. Today's hybrids can't provide this level of performance. A good example is VDI. If you only are enabling less than 200 VDI users, yes a hybrid may work ok, but to guarantee performance for 200+ user environments, an all-flash solution may be mandatory, especially for persistent desktop environments. The 2nd area of disagreement is on cost per GB. All-flash devices with significant useable capacity for the mid-market can be acquired for well under $50K. The prices of flash devices are coming down dramatically, much faster than HDDs. Some flash vendors already claim they offer the same $/GB of HDDs. It's blatantly clear that HDDs will be going the way of vacuum tubes for I/O-intensive apps. We can debate the timing, but hybrid solutions are interim solutions at best. A better and less disruptive approach is to add an all-flash device (like ViSX from Astute Networks where I work) into an existing I/O intensive environment as a complementary solution to an existing traditional storage environment. Just move those applications or virtual machine datastores to the all-flash device (typically in a matter of minutes with Storage vMotion) and users can see an immediate 5X-10X performance boost without having to replace their existing storage systems.

Storageologist
Storageologist

Scott. You might want to check out Starboard Storage the pioneers of unified hybrid storage www.hybridstorage.com . Starboard provides a unified hybrid storage solution that is built from the grounds up to consolidate SAN and NAS. Unlike those you mention in the hybrid space in your article Starboard is not an OEM of a platform like ZFS and from a scope perspective addresses a broader market than the iSCSI and VMWare only platforms you mention. When you leverage a hybrid architecture the best results are gained from consolidating more workloads because the more workloads you virtualize your SSD and HDd across the more effciiently you are leveraging your valauble resources. to back up your piece here is a blog I wrote of the subject http://blog.starboardstorage.com/blog/bid/189631/Unified-Hybrid-Storage-The-best-use-of-SSD . Full disclosure. I work for Starboard Storage Systems.

Storageologist
Storageologist

Fairbs. In some respects the discussion is not relevant. As you say each implementation needs to be carefully looked at in context. SSD speed vs HDD speed is not necessarily relevant because the internal connection to the SSD may limit it's actually performance and transfer rates. The reality is you need to make sure you have enough IOPS and capacity for your applications needs. What Hybrid storage does is it virtualizes SSD resources and share them across applications that sit at rest on HDD. In that way it handles the performance of each application based on the specific real time need of the application in your business. You of course need to look at the total from a systems standpoint and ensure that it fits in your budget. Hybrid Systems are designed to give you up to 10 times the performance of traditional storage systems without increasing the raw $/GB (In general for systems over 25TB). In fact because they allow effective use of larger HDDs and have all inclusive SW they can have a TCO 75% less over three years than traditional systems. In starboard Storage's case we also have a unified SAN and NAS capability enduring that you can virtualize your SSD across more workloads for greater efficiency and all workloads on the HDD are virtualized as well into a single pool that can accept any size of hard drive without worrying about RAID group management.

Storageologist
Storageologist

We have a saying in the UK. " All-Flash, No Cache." OK so I have corrupted it a little here. The information in the pure blog on hybrid array performance is just that PURE FUD. I do have to ask myself why all flash vendors seem to want to attack hybrid storage arrays. Surely their enemy is the traditional storage platforms running high end databases applications. The only reason I can think they want to attack hybrid arrays is that they wish they were hybrid so they really did have the compelling cost /GB argument. The simple fact of the matter is that HDD is cheaper per GB and will be around for a long time. Many large workloads do not play well with compression and deduplication so if you are trying to consolidate storage and create an internal cloud HDD has to be in there to handle the bulk of the data. Hybrid Storage simply uses flash-based caching for performance. It is not esoteric, it is a proven technology that has been around for forever but is now applied using larger SSD drives. How many workload combinations let alone stand alone workloads really need more than 50K IOPS today? Very few. you might say in the future more will and to that I say "Clearly Hybrid Storage platforms will continue to push up in performance but will never sacrifice $/GB for that performance". Hybrid storage is aimed at offering the best combination of $/GB and S/IOP. That combination is simply not achievable with all flash. I know a customer who has a Pure storage system for a high performance application and likes it but they have 3 Starboard Storage Hybrid Arrays for the bulk of their workloads. The simple fact is that All-Flash and Hybrid Arrays are not targetting the same workloads. You can read more about Hybris arrays on our blog. www.starboardstorage.com/blog

Storageologist
Storageologist

Starboard Storage has a seperate read and write Cache SSD architecture and we use SLC drives for write cache because it is the best way to optimize the write cache today for the endurance we require. It is also mirrored for integrity. Customers get a 3 year warranty and can extend it to 5 years.

wenster
wenster

200+ user environments needing all flash? For what kind of user? Power, knowledge or task workers? For someone that requires 25-100 IOPS on their desktop, would they even let you replace their laptop/desktop equipped with 8 cores & 8GB of memory? VDI is not the answer for all users/all use cases, the key is to have central management of the corporate image + data + access to applications for ALL use cases (people that have to have their own device/laptop, people that are knowledge workers that only use specific apps, contractors that come in and out). Care to elaborate more on complementing an all flash array with a traditional HDD one for IO intensive applications (in addition to svmotion migrate the critical apps to a flash datastore)? Apps that need intensive level of IOPS are most likely biz critical apps - for such apps, data protection and business continuity are also key. What are your suggestions for data protection & business continuity? Also, can you highlight what are important factors to consider in an all flash solution, to address write amplification effect on flash? Please don't just say write leveling by the controllers - that's not sufficient.

Storageologist
Storageologist

Hybrid Arrays are designed for the mainstream use cases where customers want to consolidate multiple workloads. They happen to use Memory and SSD as cache today and HDD behind because that is the sweet spot for building the best balance of $/GB and $/IOP. All-Flash arrays believe that SSD takes over the world are are making a bet on technology for high performance. The reality is that Hybrid arrays do not have to use Flash and SSD. They could use other media. It all depends what enables you to hit the customer sweetspot. If all capacity moved to Flash tomorrow and all write amplification issues solved their would still be high speed, high cost flash and lower cost higher capacity flash. Good system design would need to design a hybrid solution with caching to satisfy the market for consolidation and centralization of storage at the right price. It would be a hybrid System that was all flash. The fact is that today Hybrid Storage vendors are focussed on mainstram IT needs and are leveraging the appropriate technology to satisfy the use case and all Flash vendors are focussed on the high performance use case. Unfortunately it looks like the all-flash guys thought that they would be more mainstream and only competing with traditional stroage vendors and because hybrid solutions are spoiling their party they are trying to sell a religious technology argument for mainstream IT use cases. And as far as the cost per GB being equal that is baloney. The all flash vendors discount two things:- 1) Data reduction technologies can also work on HDD 2) Many workloads do not dedupe or compress well. In fact those workloads are the fasted growing in capacity What this means is HDD will be around for a good while and even when/if it disappears you still need hybrid architectures to satisfy the broad set of technology needs.

Editor's Picks