Storage

Stop buying storage

During the current economic downturn, IT organizations must take steps to optimize existing assets. Storage managers no longer have the luxury of cutting management corners. However, the situation presents an opportunity to complete existing projects while implementing processes, procedures, and simple technologies to improve the storage cost profile.

In the face of a challenging economy, IT organizations are under unprecedented pressure to cut costs. At the same time, they are tasked with the smooth operation of the data center, which is becoming more complex all the time as a result of continual consolidation, changes, escalating virtualization, and exploding data volumes.

Understandably, organizations are reluctant to spend any more money than absolutely necessary, which has limited or delayed the acquisition of IT equipment that otherwise might have been routine. Forecasts vary widely for the depth and length of any recession, but it is clear that 2009/2010 budgets are uncertain at best.

For most IT organizations, the focus will be on optimizing existing systems and extending their usual life. The good news is that many organizations have 50 percent or more available storage capacity that may accommodate the organization's needs for the foreseeable future. Plus, many technologies exist that can help optimize existing systems.

During periods of economic growth, organizations may be tempted to take the "quick fix" to storage management problems. The incremental cost of adding storage is relatively small and can be absorbed by the budget. Such a short-cut may facilitate faster project roll-out, but it also leads to underutilized storage. Many organizations operate at only 30 to 40 percent utilization. According to InfoPro, the average is 35 percent.

Accurate storage allocation is difficult because data growth rate information is incomplete or unavailable. Consequently, storage allocation does not correlate to consumption. New applications, with no historical trend data, receive storage allocation on a "best estimate" basis. If the allocated capacity is too high, then the excess capacity may languish unused for the life of the array.

Needless spending is the primary consequence of benign neglect. Having an array only 50 percent utilized is like paying twice as much for the storage needed. Idle capacity also consumes power, increases cooling costs, and unnecessarily consumes floor space and maintenance dollars with no return on the investment. Moreover, storage array software licenses are typically based on total (or raw) capacity, not utilized capacity, thereby needlessly driving up the cost of software.

Storage Management: Turning things around

To make better use of storage resources, organizations can leverage storage management technologies. Storage resource management (SRM), for example, enables IT to navigate the storage environment and identify old or non-critical data that can be moved to less expensive storage. These tools can also be used to predict future capacity requirements.

Managing storage without an SRM tool is like going on a journey without a map. Having a clear plan and objective before taking action is the best assurance of rapid progress and success. Storage managers should ask some questions before cost-cutting:

  • What is the average utilization rate?
  • What is the utilization rate by application?
  • Which applications are growing fastest? Slowest?

SRM technology can help companies make an assessment and provide an enterprise-wide view of the storage environment, which helps identify problem areas, consolidation opportunities, and to create a priority list of solutions.

In addition, thin provisioning can be used to improve storage capacity utilization. These tools allow space to be easily allocated to servers on a just-enough and just-in-time basis. Thin provisioning can enable higher capacity utilization by allowing applications to share a pool of available storage that reduces the amount needed for any individual application. Storage is allocated to applications dynamically as needed, resulting in higher utilization.

Thin provisioning also eliminates the guesswork in new application provisioning because rapidly-growing applications can access space as needed, while low-growth applications will not hoard empty space.

Furthermore, thin provisioning can reduce capital expenses because it requires less up-front storage than a "stove pipe" environment and permits "just in time" storage allocation.

Data deduplication: Don't store it in the first place

Data deduplication is another technology that has gained wide acceptance as a tool to streamline the backup process. Deduplication eliminates duplicate data even when such data is unrelated, greatly reducing the data multiplier effect on data.

For example, if a Microsoft PowerPoint presentation is stored on different file servers multiple times, deduplication ensures that only one copy is stored no matter how many full or incremental backups occur. Organizations may consider specialized appliances to provide backup-to-disk and deduplication functions. However, these appliances add complexity to the data center with more devices to manage and actually add capacity to the environment rather than using what already exists more efficiently.

Data archiving: Out with the old

Thin provisioning and data deduplication are strategies for reducing the growth rate and space consumption of new data or finding more efficient ways of storing it. These strategies must be combined with addressing unnecessary data storage in order to fully utilize existing assets. The largest container of unnecessary and obsolete data is unstructured data.

E-mail is the biggest unstructured information pain point today and a top target for data reduction via archiving. The Radicati Group estimates that the volume of email will increase by 30 percent from 2006 to 2010. Although storage costs continue to fall on a per-unit basis, email is often stored many times in the email server, on the user's PC, in a Microsoft Exchange or IBM Lotus Notes file, on file servers, saved in SharePoint, and in backups. Because of the excessive storage consumed, the cost of power and cooling is also commensurately higher.

Across all business industries and public sector organizations, IT professionals are being called on to address the common management concerns around email and unstructured information, which is resource management. Archiving technology will act as an online archive for older items that are moved from primary application storage according to company-defined policies. It also leverages optimized single instance storage and compression technologies to further reduce the data footprint.

By controlling the size of the message store, the applications and servers hosting them remain focused on real-time transactions. The online archive also enables organizations to rationalize their storage resources and dedicate primary storage to dynamic and transactional data. Older, less frequently accessed content can be moved to a secondary or tertiary storage device, saving money for more strategic purposes.

During the current economic downturn, IT organizations must take steps to optimize existing assets. Storage managers no longer have the luxury of cutting management corners. However, the situation presents an opportunity to complete existing projects while implementing processes, procedures, and simple technologies to improve the storage cost profile.

-------------------------------------

Phil Goodwin is the secior manager of storage management, Symatec Corporation.

13 comments
jered
jered

Customers absolutely need to make better use of their storage, but as you point out that's not always the only problem. Customers need to stop buying expensive storage, their tier 1 platform that they buy huge amounts of every quarter by rote. The old adage "nobody ever got fired for buying (three letter company)" doesn't hold true anymore. Instead of buying $30/GB high-end SAN, customers need to consider more appropriate places to put their data -- places that are far, far cheaper and also provide superior reliability to purely performance-oriented storage. One area I'd like to comment on is deduplication. At the backup level dedupe can see savings of up to 50x or more, but that's generally based on doing it wrong -- backing up the same thing time after time. Right now data is being written to primary storage at $30 to $50/GB, and then backed up at an aggregate total cost of maybe $5 to $10/GB more. Deduplicating VTL or backup can shave a few dollars off this, at best. Instead, much of that data can be moved to an archive tier at $3 to $5/GB, and with an effective cost even lower with deduplication. Effective replication can eliminate the need for backup entirely. This path saves much more money, though isn't as seamless as merely optimizing backup. With economic pressures, any business would be remiss not to look at deploying an effective archive tier. At Permabit we've developed our Enterprise Archive product specifically to serve these needs, and believe we have developed the only truly scalable deduplication solution for archive data, while also providing levels of data protection far beyond what is available with RAID. I talk a little more about the underlying economics over at my blog in the article at http://permabit.wordpress.com/2008/12/09/cutting-costs-with-enterprise-archive/ Regards, Jered Floyd CTO, Permabit

hheuts05
hheuts05

Since 1 TB harddrives nowadays cost about 80 to 100 euros (so at most 140 dollars or so) depending on the brand and type I'm wondering about the pricetag of 30 to 50 dollars per GB that you are calculating ? Storage nowadays is in my opinion only a very small portion of the total cost in the IT - no ? It is the management that should be deminished since they mostly are overpayed and sometimes even uncapable of understanding their own decisions !

brian
brian

Search for "Bluearc" for a quick idea of what these things look like. Those guys publish photos and Google Image usually returns something quickly. As mentioned 1TB drives are cheap per gigabyte but not necessarily usable beyond a certain point. You can build cheap small office systems but even a small office will feel the effects of using 7200rpm drives in RAID. The technology to look at is SAS, SCSI, Fiber Channel equipment etc. which is much more expensive. Other points to consider are (1) for each drive whose capacity is used for storage, at least one other drive is mirroring it. Probably more than one. (2) For each set of mirrored drives there is likely a drive just sitting there empty, ready to power on and begin rebuilding if another drive fails. (3) Some larger businesses will have mirrored duplicates of the entire storage system, so that if the controller board goes down they can still operate. (4) It is not always possible to purchase the newest drives with the best dollar-per-gig ratio. When maintaining a huge system like that, the drive capacities have to match or the extra space just gets wasted. If you have a three year old array and some drives are starting to fail, you can't just buy 300GB SAS drives. You probably have to match the 76GB SCSI drives. (5) The cost of the chassis and the controller hardware is not small. Just the metal rack that holds the thing costs hundreds. Purchased parts-wise the hot-swap bays are $100 to $200 per every 3 to 5 drives. A system board capable of connecting 100+ hard drives and managing them as a single volume is very specialized and expensive hardware too. ... Not that I didn't take full advantage of the cost of 1TB drives when my boss needed 4TB's of safe storage, real cheap, right now, a couple months back! (But, sadly, even with five people hitting it, the performance is not nearly what I'd expect from SCSI.)

danyohene
danyohene

the new pcs you are advertising is really nice and powerful? can i have one for free? i am ohene from ghana and also a student. may be i can do adverts for the pc in my school and ghana as a whole. danyohene@yahoo.com

amendonca
amendonca

The $30 to $50 are not for the 1TB hard drives you refer. These are prices for enterprise storage, with higher throughput and better reliability, like raid systems. The 1TB hard drives are not prepared for server intensive usage.

reisen55
reisen55

The DataCenter I commented on was using, as mentioned, LT03 level tapes for backing up about 3 terabutes of data nightly either full in some cases or differential in others. Backup time was 15 hours end to end. Tape has about a 47% failure rate and restoration at 2 in the morning is not the time to discover that tape 3 is bad. A municipality, in PROCESSOR magazine, changed from a like scenario to a hard drive based format which cut backup time to 5 hours and also provides (a) certainty of restoration at a (b) higher speed as well. Secondly, nobody really tests. And that makes it all worthless. I test for my small business clients and have better DR plans for their offices than the DataCenter EVER could have. Yes it is smaller and such, but a small business has ONE location anyway so the impact of loss is, well, TOTAL. DataCenters tend to be linked to other servers where a modicum of corporate support will still reside in any situation.

reisen55
reisen55

The price of storage, per se, has fallen through the basement and continues to be one of the best investments an IT department can make. It can also be the worst. Having an open door to ever increasing storage capacity is good but the user community loves to eat it up quickly and without regard for other issues, namely backup and restoration. If you are to limit storage, you have to educate the user community on it's limitations and the dramatic effect it can have on backup scenarios. I just came out of a DataCenter position wherein tape backups were run with enormous difficulty. LT02 tapes replaced with LT03 and possibly LT04 would be even better but restoration issues at an off-site center prevented LT04 capacity. Well, having an Overland tape changer for backups is a great thing but when you are in restore mode -- a single drive, a plan and a few gallons of coffee are what you need. The backup plan was constantly impacted by storage capacity issues and planning, which is to say total chaos. And nobody was willing to test restoration either. Having a good backup is one thing, knowing how to put it back together again is quite another. Without doing so means your backups are mostly useless to start with. Email requires quotas but here, too, in a disaster scenario a large email file has a virtue, namely it holds stuff. Do not stint on storage per se. Do not stint either on user education and that means in house programs and seminars. Do not forget the value of off-site storage retention. Do not forget to test.

mike_patburgess
mike_patburgess

Totally agree with your statements and I might add the following. 1) A lot of the current install base is running older disk drive 73GB, 146GB, and the issue with that is those drives are becoming obsolete (technology waits for no-one). Replacing a failed unit with newer tecnology can come with all sorts of operational issues. 2) Archiving to a secure and environmentally safe facility is very expensive; probably more expensive than adding more storage. 3) Duplicate files... well, I guess that there is a need for some sort of pre processor that sorts this issue out and I am not sure it is de-dupe.

reisen55
reisen55

Totally different animals. The vast amount of the WinTel base is indeed in the 80gb to 160gb range. I have maybe six of them myself in various systems. I hated SATA initially but it's simplicity of connect is a virtue. I carry a SimpleTech shell I purchased 2 years ago and while it contained a 320gb drive (which I murdered by dropping it off desks four times), I now keep a 500gb drive (in an IBM Softbag) with me on client calls. It is a delightful companion whenever I need it to backup my client's data for off-site storage. And MicroCenter has BEAUTIFUL hard drive prices these days. NONE OF WHICH APPLIES TO DATACENTER. Here we have armies of 78 gb or so drives in SANS and the real capacity issue here is on the outside, user education. Shared folders tank up rapidly and universal storage limits do not exist. Users need to know how this works, and here office seminars with coffee and doughnuts do wonders. Triad redundancy storage: one primary location, one on-site storage point (hard drive better than tape) and one off-site storage point (again, hard drive). Tape itself is dead - just see John Cleese as Dr. Harold Twainwreck.

mike_patburgess
mike_patburgess

Yes I am sure that you have a plethora of 73GB drives as most people do but as a reseller, we cannot get FC or SCSI 73GB, 146GB drives any more. They are obsolete. In fact, the 10K rpm drives have been replaced with 15K rpm drives. Do you know what kind of issues this causes when trying to integrate them into a solution. Now along comes SAS drives for the data center, vertically or horizontally aligned in a chassis, lower capacity drives but they say they are faster... the saving grace is .. database people love them because of the lower capacity and the ability to spread the DB over many spindels. Don't get me started on SATA and the spin down "features", which are implemented because, "THE SPINDLES BREAK". You might have been lucky with them in your systems but I am sure that you do not use them in an 80%+ duty cycle for if you did, you would be cursing them. Now along comes the SSD disks. Has anyone really looked at the technical specs on this stuff. Circular and concentric write/reads in order to spread the damage that could be caused when you write to one cell continuously. Oh yeah, which type of SDD are you using? Every vendor has a different flavour.. Enterprise or commercial. Single cell or multi-cell. Reserved space on the chip in case of failure vs writing to the whole thing. Keeping up with this nonsense is a full time job.. agreed?

b.lantz
b.lantz

I can't help but think your response is misguided. When Michael mentions 73 and 146 GB drives, I do believe he means your typical scsi/sas server implementations and not desktop systems. Anecdotally, a year ago we repurposed our file server (6x73GB) to be our SMS server. The capacity wasn't cutting it and the hardware is fairly dated. It's been replaced by an M530 G5 with 6x146GB and in the past year our users found a way to max out the new capacity. I can't help but think most of the usage is duplication as I keep stumbling across duplicate folders. Our users find a repository in a shared folder and feel the need to copy it to their own personal network folder.

mentalguitar343
mentalguitar343

Correct. The consistent problem is that home users expect to be allowed the same privileges at the workplace. A sympathetic User Support desk has to move past merely dealing with `today's `crises' and to find the time and dedication to implement procedures for storage and file creation. In my work at Corporation for Supportive Housing, we were able to implement many storage and size restrictions -- we did this in a friendly way (not always easy). We were blessed with a young IT director who actually helped sell the idea to various department heads.

johndecoville
johndecoville

This is not the time to waste on storage: Well written Riesen55 or training (better use of resources). IT staff are a HUGE lever and they leverage the productivity of the whole organization and its clients. The article was a total waste and posts mis-information! TechRepublic! Stop it! Thank you posters for setting them straight.

Editor's Picks