Over the past couple years, I’ve been battling with the same explosion in growth of storage occurring across many industries. A recent podcast reminded me of the incredible challenges associated with maintaining data online and nearline. System administrator Julian Firminger, a recent guest on The Datanauts, shared how he uses pallets of LTO tapes to ship data from site to site. The LTO aren’t for backup retention, but primary data transfer.
The amount of data generated in the media and entertainment (M&E) industry requires data transfer via tape. Due to the incredible amount of data generated, the most efficient way to transfer that data is via LTO tape. Although, it had me asking the question of what’s the modern way to backup the ever-growing data sets.
Tape rules backup
From a pure cost perspective, tape is still the king of backup media. Tape has kept pace, and therefore its price and capacity advantage, with the price drops of disk for backup. Tape also has several technical advantages over hard drive medium. Archival is one of the most obvious benefits. Tape can be stored for years and reliably recovered. Tape is also more resilient for shipping as demonstrated in the above M&E use case. Shipping logistics are relatively simple vs. that for shipping hard drives. No special packaging is required, and X-rays don’t impact tape during shipment.
Tape has weaknesses vs. hard drive-based backup, though. Random access speed is especially a challenge. If frequent recovery is required, the logistics of managing data recovery is difficult, to say the least. Object-based storage looks to mitigate some of the issues of hard disk-based backup while offering tape-like portability.
In another post, I introduced a few options for scale-out backup solutions. While the technologies allow for some incredible data limits, there are practical limits to these technologies. One such limit is cost. While the price points of solutions such as Rubrik and Cohesity are lower than those of traditional hard drive-based solutions, the cost pales in comparison to tape at scale. Object-based storage provides a second layer of long term storage for a cost-efficient and higher performing medium.
There’s currently no widely accepted standard for writing file objects to disk. Filesystems are physically portable from one operating system to another operating system. The most obvious example of the portability of filesystems is a USB drive. A USB drive formatted in FAT32 is mountable on Linux, Unix, Windows, and macOS. Without a consistent disk format standard for object stores, an access standard provides similar portability as a filesystem.
The default standard for reading, writing and editing to object stores is Amazon’s S3. Most backup solutions allow customers to use S3 and S3-compatible cloud services to store archived data. Cloud-based backups enable offsite storage of backups and a high level of resiliency as cloud services allow replication of objects across cloud data centers.
Many data security policies still do not permit the storage of enterprise data in cloud services. For these customers, there are solutions on the market that allow S3-compatible solutions. I recently sat down and discussed an S3-based solution by Cloudian. According to Cloudian, their solution is the most S3 compliant solution on the market.
The market for S3-compatible storage is a crowded space. I’ve gotten briefed by several object storage companies looking for traction in the enterprise. Competitors include Swiftstack, Scality, and Dell EMC. According to Cloudian representatives, their extensive S3 compatibility ensures mobility of object storage between backup storage solutions and applications.
Object storage is growing in usefulness and maturity for enterprise use cases. Backup is an example of a massive service for object storage.
What do you think?
What enterprise storage use cases have you come across of object storage? Share your thoughts in the comments section.