If I’ve learned anything in the past few years, it’s that the storage market is a fluid one, particularly as flash-based options have grown in prominence, resulting in a number of new players in two primary market segments: Hybrid storage and all-flash storage.

  1. Hybrid storage. Those vendors selling hybrid storage devices include newcomers Nimble, Tegile, Tintri and others. In a hybrid storage system, the array includes both solid state and rotational storage. In such scenarios, the solid state storage is often used as a mega cache which provides significant acceleration of all reads and writes from and to the array. There are a number of established players that produce product in the hybrid space, too. However, in many cases, support for solid state storage is added to a legacy platform. The newcomers generally have the greenfield advantage here as they can build systems with flash in mind and can sometimes take more advantage of flash’s unique characteristics. Note that there are other vendors in this space, but the space changes every day!
  2. All-flash storage. An all-flash storage system is exactly what it sounds like. It’s an array that contains no rotational storage whatsoever. All data is stored on solid state drives. Upstart vendors in this space include Pure Storage, Nimbus, and Whiptail.

Solid state drives (SSDs) versus hard disk drives (HDDs)

Before we start discussing the pros and cons of hybrid vs. all-flash arrays, let’s make sure that the differences between flash drives and solid state is understood.

HDDs

First of all, many people refer to HDDs as “spinning rust” due to the nature of the drives. In short, they sit around and spin all day long, accepting instructions to read and write data as needs warrant. HDDs have been around for decades and have undergone a ton of growth, now reaching capacities of 3 TB and more. However, in the past decade, HDDs have not kept pace from a performance perspective. After all, 15K RPM drives have been around for a decade and they’re the fastest HDDs out there right now.

As applications and new ways for doing IT (read: virtualization) began to come into play, IT departments found themselves having to constantly add new disks, not to meet capacity needs, but to meet needs for an ever-growing number of IOPS.  Individual hard drives simply do not provide much in the way of performance. It takes many, many drives to achieve necessary performance levels for many organizations.

From a capacity perspective, however, HDDs are just fine. Modern drives are massive and can hold a ton of data, but they just can’t do it quickly.

If you’d like to learn more about hard drive performance, RAID and arrays, visit this article that I wrote a couple of years ago here at TechRepublic.

SSDs

After topping out at 15,000 RPMs, rotating hard drives simply sat still on performance for almost a decade. In recent years, the storage industry has turned increasingly to solid state drives in order to solve what was becoming a crisis with regard to storage performance. Whereas HDDs provide massive capacity with relatively poor performance, solid state drives offer massive performance with relatively low capacity. The table below helps explain where HDDs and SSDs are different from a metrics perspective:

HDD

SSD
Cost per gigabyte ($)

Very low

Very high
Cost per IOPS ($)

Very high

Very low

As you can see, there isn’t much of a middle ground. You either buy for performance (SSDs) or you buy for capacity (HDDs).

If you’d like to learn more about solid state storage, please visit this article.

An array of solutions

Most companies don’t buy single hard drives for storage. Instead, they buy arrays that house a lot of drives and then combine these drives together using RAID or some other mechanism.  With the advent of solid state storage, a storage buyer now has a number of new opportunities.  No longer does he simply need to buy more rotating spindles to meet his performance goals.  Now, he can buy fewer solid state drive arrays to meet those challenges.  Except that doing so comes at a cost: If he buys an all flash array, he’s buying a lot of expensive capacity for applications that may not need expensive capacity.

HDD only

Hybrid (HDD + SSD)

SSD only
Cost per gigabyte ($)

Very low

Medium

Very high
Cost per IOPS ($)

Very high

Medium

Very low

Today, there are data centers that are clamoring for better storage performance. However, even though some vendors may hype the fact that they can do a million IOPS in 2U of rack space, how often do most organizations need that kind of performance? Unless you consider niche cases (i.e. analytics, business intelligence), most organizations don’t need millions of IOPS.

What’s emerging are really three distinct markets.

Traditional

A traditional environment consists of just rotational storage and offers amazing capacity, but with the tradeoff being low levels of performance. These solutions remain popular today as many CIOs focus on cost per gigabyte at the expense of low IOPS. While this kind of solution may be viable for “cheap and deep” storage, it’s increasingly unsuitable for real workloads, particularly as organizations begin to look at trends like VDI.

These kinds of arrays are often priced at the low end of the market.

Flash only

At the other end of the spectrum, we have players that are building arrays that have nothing but solid state drives. These devices forsake capacity for massive performance and there are a number of good reasons that businesses need the IOPS, but it becomes more likely that they will run out of disk space at some point.

In order to combat the space issue since SSDs are much smaller than HDDs, many emerging all-flash vendors have turned to data reduction technologies, including compression and deduplication. With such technologies, vendors are seeing significant success in stretching the capacity of their all flash arrays beyond the hard capacity of the units, and offering customers a deduplicated lower price per GB than would be possible without the data reduction techniques being used.

However, I see two serious challenges:

  1. Most CIOs simply don’t need the levels of performance offered by all-flash arrays. The broad swath of the market simply needs affordable storage that runs at very good levels of performance. While all flash supports the performance need, it still remains much more expensive than HDDs, even when data reduction is taken into consideration.
  2. The initial acquisition cost of an all-flash array can be staggering and is often well into the six figures and above $200,000. For many midmarket companies, it’s not possible to get this kind of capital. And, still, the cost per GB is still higher than necessary.

Hybrid

So, what’s the answer?

In my opinion, the answer is to combine the best of what both the SSD and HDD worlds have to offer and create an array that leverages the benefits of solid state storage for performance while continuing to be able to rely on the massive capacity benefits inherent in HDD-based solutions.

That’s exactly what the growing hybrid market thinks, too.

There are a number of emerging storage companies that have entered the market in recent years and each brings with them a “secret sauce” or a differentiator that makes their solution unique. At the same time, these players are focusing on the balance that must come when one looks at the two primary storage metrics: $/GB and $/IOPS. A hybrid array very nicely balances these two metrics and can satisfy a wide swath of business needs. Sure, there may still be outlier cases that demand all SSD or all HDD solutions, but these are outlier cases. The cost per GB for a hybrid solution might be a little more than it is for an all-HDD solution, but will be much less than an all-flash solution.

Today’s storage buyers are always looking for solutions that provide both capacity and performance value and do so in a way that makes budgetary sense. Because most of the new hybrid players rely on commodity hardware and differentiate themselves in software and because they don’t require as many expensive SSDs, hybrid arrays carry a price tag that is a fraction of what one would find with an all-SSD product. While it won’t beat the all-SSD array in a head to head test, when one measures the performance based on true needs, hybrid storage is often the perfect solution.

Take, for example, VDI.  A hybrid array in this scenario is a perfect fit. Because commonly accessed files are cached on super-fast SSDs, boot and login storms are a thing of the past. Further, deduplication savings in VDI is incredibly high, so you may find yourself using much less disk space than you need .In addition, because there is ample capacity in a hybrid device, an organization can consider allowing users to create persistent, customizable desktops. There are simply a number of opportunities that become available.

Summary

As you can tell, I’m pretty passionate about the hybrid market and have been for quite some time. While all-flash arrays are really, really cool and I’d love one for my lab, in challenging budget times, CIOs must think practically about the solutions they implement and I’d urge them to take one of the newer market players for a test drive. Some of the companies will provide loaner/test gear for potential new customers, so that you can take one of units for a spin.