EMC's CTO, PureStorage's CTO, and an IDC analyst reveal how all-flash arrays are impacting storage budget allotments and software design.
Potential customers of all-flash storage arrays can rest assured that the market is nearing technological maturity, although the latest systems are having an interesting ripple effect on other aspects of enterprise IT departments.
Sales alone are startling. All-flash array revenue exceeded $2.5 billion for 2015 and will reach $5.5 billion by 2019, which should be 60-70% of all primary storage purchases, IDC analyst Eric Burgener said in a report last month.
The 2015 figures saw EMC grab the largest slice of pie, IBM and a specialist called PureStorage roughly tied for second place, HP Enterprise following, and NetApp (which recently purchased SolidFire for $870 million) rounding out the top five. (Burgener said he cannot publish detailed shares because many storage companies provided incomplete data.)
Flash is fast, so the back-end spending gives users a list of advantages that reads like alphabet soup. Backups (and restorations) take minutes instead of hours; cloud services react as if they're local applications; databases respond sooner; and e-commerce sites are more reliable. There are also some unanticipated side effects.
Burgener, EMC chief technology officer John Roese, and PureStorage chief technology officer Chadd Kenney, discussed with TechRepublic how all-flash arrays (as opposed to hybrid disk-flash arrays) are having unique impacts on storage budget allotments and even software design.
SEE: Data storage: Preferred vendors, demands, challenges (Tech Pro Research download)
Side effects on storage budget allotments
On the budget front, solid-state drives are rated for a very specific amount of writes before they fail, unlike traditional spinning hard disks in which mechanical parts such as bearings and arms can wear out gradually or sometimes even break quite unpredictably. That means an IT department can accurately plan the flash drive replacement purchases and replacement schedule by looking forward at drive specifications rather than looking backward at failure trends.
Drive manufacturers know this and usually offer multi-year free replacement plans, Burgener said. Longevity can be improved by having drive software that considers factors such as where it hasn't put data lately, as opposed to traditional disks that usually just put data in the next available spot.
However, flash drive costs remain much higher per gigabyte than regular drives because demand is exceeding production, EMC's Roese explained. "There is nowhere near the fabrication capacity in the world -- we can't produce enough NAND or enough of this non-volatile memory anytime soon," he noted. That is a big reason why flash array capacity currently ends at about one petabyte, while disk arrays have much more, he said.
Side effects on software design
Non-storage application developers are also gradually starting to consider flash memory's attributes. Some modern programs are developed specifically to have faster and fewer write cycles, while others are reducing intentional input-output delays that are sometimes used to make a CPU wait for traditional hard drives to catch up.
Such tweaks are a good way to make a general-purpose application or a storage management system work more appropriately for where the data resides, Burgener and Kenney both observed. Storage array companies like EMC and PureStorage are already delivering some of their own programs in flash-specific versions, but whether that means independent software vendors may start shipping a new class of "for flash" versions of their products remains to be seen.
Storage advancements and upcoming products
The future holds other intriguing products. For example, Roese said EMC is close to shipping products from its spring 2014 acquisition of startup DSSD. That group has an ultra-dense flash system that plugs directly into servers. DSSD is led largely by ex-Sun Microsystems engineers, including visionary Sun co-founder Andreas Bechtolsheim. Industry-wide advancements in new kinds of memory such as those using carbon nanotubes or memristors can further move the bar in flash memory's favor.
- Storage in 2016: Flash is in, disk is out, and tape's coming back (TechRepublic)
- EMC and Backblaze storage systems will talk to the cloud (TechRepublic)
- Inside The Machine: Hewlett Packard Labs mission to remake computing (TechRepublic)
- Ghosts of tech past: Photos of data storage from the 1950s - 1980s (TechRepublic)