Discussion Locked

Backup tape useable life and archiving

0 helpful
25 replies
Closed to new repliesThis discussion is closed. It stays readable, but it takes no new replies.

The other day one of my clients asked me how long they should keep using the same data tapes. My inital thoughts were until they start to have errors. But then it occured to me that you might want to stop using then well before then.

So my question is, how do you tell if a tape is degrading?

Does the age of the tape and the number of time it has been used degrade its ability to store the data?

What archive cycles do other members use or suggest using?

DO

25 replies

by stress junkie

I was in charge of backups, among other duties, at my
last job. The company had archive tapes stored off site
going back up to a decade. I can tell you that the tapes
that had been stored 5 years or more often could not
be read by the tape drives. When I realised this it
seemed to me that the requirement to store the data 7
or 10 years meant that we would have to recall tapes
after, say, 4 years and copy them to another tape. I
don't think that the old tapes were mechanically bad.
Rather I think that the magnetic signal had degraded.

However, it seems to me that the ONLY reasonable
solution today is to use DVD-R and DVD-RW disks. The
cost/capacity is about the same as tape but the drives
are much less expensive and the data can reasonably
be expected to retain its integrity for 30 years. You can
even reduce the cost of the long term archives by using
write once DVDs. Backup software like Veritas can use
the DVD format. And the DVDs are not going to be
erased by magnetic fields. I've seen many people put
magnetic tapes on top of or beside huge CRT monitors
and then wonder why they couldn't recover data from
those tapes. Duh.

So tape stinks. DVD-R and DVD-RW are great. Price is
comparable between the two media formats.

by stress junkie

I realised that my resonse strayed a bit from your
question. I'll say this about tape life. I've used old DLT
tapes reliably for daily backups. These tapes would be
recycled about every three or four weeks. I almost
never had trouble recovering data from these tapes
although they were used frequently, not just for
recording the backups, but by also recovering data for
people that considered backups as near line storage.

I recommend using backup software that verifies data
by reading the tape as it writes the backups.

I suspect that the tapes' useful life was comparable to
the life of the tape drives. I know that's surprising and
counterintuitive because the tape has to contact the
drive heads to be used, but that's what I've seen. So I'll
say that the life of a DLT tape is probably greater than
5 years. I'd keep using a tape until I got errors creating
a backup. Don't forget to use a bulk tape eraser before
discarding old tapes.

by pgm554

HP drives write header info on the tapes and after so many uses ,will refuse to use the tape for archiving purposes.

by msg2612

At our office we still use DDS4 20/40. I have a 9 tape set in place -- one each for Monday thru Thursday, then five separate tapes for Fridays, so we can go back four weeks if necessary.

I use these sets for one year before pulling them and creating a new set. I found in the past that the DDS3 & DDS4 tapes I was using for a little over a year would become error prone.

A DVD loader would be great to have as a backup resource. Each backup solution has its drawbacks, though. I've been reading a lot lately about how easily CD/DVD is damaged, so...

by phil spiby

Although this is not answering the question asked, I thought it may be useful to add some insights on digital archiving based on Stress Junkies response.

If the customer was actually wanting long term data retention then your approach of using DVD media (because it has an expected integrity of 30 years) is not the whole answer. Which ever media (and even data format) you choose now will be obsolete in 30 years. Punched cards (in regular use 30 years ago) have a good data retention period, but how many people would consider trying to use a punched card reader now? The same will be true for DVD's in 30 years time.

To achieve long term data retention you need to accept the fact that media and data formats will change over the life of the archive.

An excelent standard in this area is "ISO 14721:2003 Reference Model for an Open Archival Information System (OAIS)". Goes into great detail discussing preservation planning and media migration strategies.

The Pharmecutical, Nuclear and Aerospace industries are addressing these issues now using OAIS, in the certain knowledge that if they don't then Digital Doomsday (when you cannot recover a piece of critical digital data) will not be far away.

by warren sever, mcp

Tape is ancient technology and, when compared to the alternatives, tape is clearly the worst choice.

I recommend and use hard drives. They're screaming fast and instantly accessible.

For archiving, DVDs are the answer.

I'll take this opportunity to go off topic a bit and take a shot at the entire electronics industry. Digital camcorders should have hard drives, not tapes. C'mon, people, wake up and give me the technology I want!

by ed_gooding

>> I recommend and use hard drives. They're screaming fast and instantly accessible. <<

Do you use hard drive caddies and store the backup hard drives off-site. If you don't take your backups off-site, what is your disaster recovery procedure if the building housing your system(s) and backup drives burns down?

by razz2

I will get into tape life shortly but first:

Warren,

Light bulbs are ancient as well but we still see their
purpose in life. Note that I say purpose and not
advantage.

For a very small business, CD-RW is fine but a large
network backup operator would spend his life doing a
full backup.

For a Small/medium size, DVD is fine but has serious
draw backs. While they have a long shelf life they are
prone to read/write errors like any optical device. This
may be a device issue or as simple as a scratch. using
DVD to archive data is great because they are burned
and then stored. Rarely handled.

Also for Small /Med: you have the option of DVD or an
inexpensive tape such as a 20/40 Travan. Not a great
tape but it works for a standard rotation of 9 or 18 tapes.
It solves the offsite issue too. And, it is easy to setup for
automated backup when no tech is employed by the
company. Drive and tape cost is within reach of a small
company. Less risk of damaged backup mediawith a
tape compared to DVD is a benifit.

For a server room with 5 or 6 servers, including
Exchange and an SQL server, then DVD is not an
option. Even if they held 9 GB. I doubt they would make
it through a few offsite transfers safely let alone a few
restores. EVERYONE has scratched a CD/DVD on
inserting it to the tray. For this a Tape Loader is the key
regardless of format. DLT etc.

Backups simply provide access to data in a condition
from the past. That is NOT a true Backup Plan for
disaster recovery. In the event of a building burning
down, the clients, billing, books etc. are still available
with offsite storage. Onsite storage I see so often put in
a standard fire safe. Useless too. Even paper will char,
for media you need a media safe.

OK, now the question...sorry about the wait. Most of the
comments here reference a 9 tape rotation. I too use
this as a minimum. With tapes averaging $35 to $45,
with a purchase of 10, that is a whole set for $350 to
$450. I change tapes yearly. Tapes will last 5 years in
my experience based on use, but the drive may stretch
them. heat may damage them. Shelf-life is only part of
it, and then only if they are handled correctly. For less
than $450 a year I can protect the whole company with
tape. I can have it automated and available offsite.
Compared to the cell phone bills now that is very little.
A cost of doing business.

Just an opinion, but this is a great discussion.

by jturbett

DVDs have major issues in enterprise back up systems.
1. Size, 1 LTO tape = 42.5 DVD disks
2. Tapes such as DLT and LTO are in protectected cartridges during the loading and unloading, where optical media may be scratched. Can you ensure that staff handle your data with care each time?
3. DVD's have a greater chance of error (coasters)during the write process than tapes. There are tape systems that can recover during a verification process. DVDs do not rewrite automaticly since they burn through sequentialy
4. The Industry is heading for hierachacal storage systems or as I like to call them cost based storaged systems. In which storage life cycle passes from SAN to SATA drive san attached arrays then to tape. Even some of the SATA arrays can mimic tape. It all ends in long term archiving to tape.

Any technology implentation must be tested to ensure it works in your environment. This means that you must perform operational readiness testing at randomly scheduled intervals. regardless if you choose tape, disk as tape or dvd technologies. Not to perform the tests leaves you unsure of something you should be sure about and that is your data storage.

FYI my company backs up 62 TB of storage each night. You do the math how many dvd's is that?

by jasonhiner

TechRepublic contributor Mike Talon has written extensively on tape backups and alternative backup systems. Here's an article that can help with planning tape backups:

The three tasks in building a tape backup
https://www.techrepublic.com.com/5100-6265-1040176.html

I'd also recommend subscribing to the Disaster Recovery newsletter that Talon writes on a regular basis because it deals with various backup and recovery issues. You can subscribe by clicking this link:
http://nl.com.com/MiniFormHandler?brand=builder&subs_channel=bldr_front_door&list_id=e061&tag=fb)

by paul

Brenden, on the sites that I manage I use DLT for normal backup routines, and DVD for archival. Generally speaking, I make it policy that a random restore test is conducted twice a week, so that all tapes are tested over a fairly short period. This is part of the Disaster Recovery regime I normally implement.

With the DVD side of things, I generally recommend that clients take a DVD "Snapshot" of their data structure at the end of each month, preferably on the day that accounts close off the month - these are stored in a secure location and provide a nice month by month historic snapshot of how the company looked. In order to maintain quick access, we burn the data to the DVD as it is, rather than using a backup system to generate a file - that way if an accountant or similar needs to look at the DVD at any point, they can simply browse the usual directory structure.

Bit more difficult when your data structure is above 4.7GB, but most of my clientele is in the SME space and therefore under this limit - although now I see the latest DVD technology increases the capacity greatly.

by jim_maclachlan

This was a great question to ask. I recently had to restore data off a DLT tape made in 97 using my SDLT drive. It worked (maybe because I had my fingers crossed? ) So that's a 7 year old tape on a slightly different technology. One of the reasons I went with the SDLT was that it would be backward compatible.

I don't think DVD drives would work in my situation since I'm backing up well over 100gb every night. The discussion here has given me a lot of food for thought though. What will I be using 7 years from now? How can I retrieve it? I'm sure there are enterprise storage systems out there, but we're a large small, privately owned company, so I tend to have a small company budget with medium sized company technology needs.

by db8abl

To test and extend tape life in the days of tape reels and with large show that use 1/2" cartridge tapes, the common practice was to test and clean each new tape and before it went back in service after becoming a scratch tape. These were typically tapes that were used for daily, wkly, monthly, quarterly backups. The cleaning and testing was done on a Kybe machine (or similar) that also wrote and read back for errors then printed out a error report by location on the tape. When the errors reached a certain threshold the tape was discarded. Most often the backups were not full reels and the first 100 feet or so which got the most use would log errors. In those cases the tape was stripped of the error section and put back in service.
When you use the same media for archive the shelf life depends on how it is stored and time. In an environmentally controlled environment the tape should have a life of about 3 years at which time they should be and re-written. There are a number of service bureaus that do that kind of thing. If you are working with data that cannot be replaced (i.e. test results for the FDA) you may have regulatory issue the require these type of measures - most situations do not.
These services exist for newer media - DLT and IBM cartridge 1/2" - I don't know about DAT or 8MM.
In practice, however, most smaller shops do not go the expense of cleaning and testing. They typically log the number of times a given media has been used and track when they begin to have errors. Then just set the "life" of the media at 10% below the point where errors start showing up. Because much depends on the specific tape drives you are using, the environment, the frequecy of head cleaning, and the quality of the tape you can only guess without real information from your operators. If you talk to your operations staff they will be usually able to tell you when the tape media begins to fail. A good operator (if they are paying attention) will be able to tell when the tape drive begins to go into write recovery in an attemp to correct errors. That's the time to replace the media - assuming you have cleaned the machine before each backup cycle (as I know you all do).
One last note: sometimes you can calculate the time a backup should take based on the amount of data it is backing up. If the time is considerably longer than normal, it is likely that the drive is doing a lot of error corretion and you should suspect a bad tape.

by msg2612

At our offices we still use DDS4 20/40. I have a 9 tape set in place -- one each for Monday thru Thursday, then five separate tapes for Fridays, so we can go back four weeks if necessary. I run full backups each night. That way, all I have to do is take home the previous night's tape, and if we have a fire or something at work, all we've lost is a day's work.

I use these sets for one year before pulling them and creating a new set. I found in the past that the DDS3 & DDS4 tapes I was using for a little over a year would become error prone. Not all of them, but I'd start to have intermittent errors.

A DVD loader would be great to have as a backup resource, though each backup solution has its drawbacks. I've been reading a lot lately about how easily CD/DVD is damaged, so...

by dilbert-tom

This is from experience with large mainframe data (often multiple volumes, so the tapes could be considered to be 100% used).

A Payroll system that I was on call for had problems with 'tape stretching' (not just 'dropout', but where the media was actually beginning to be stretched enough to be unreadable - we had some white 'stuff' we could 'paint' onto the tape to see the magnetic flux patterns, so we knew what it was). I found out that that shop simply used tapes till errors like this occurred, then retired them. Occasionally I needed to recreate the data using older backups to re-run the cycle that generated them (oh, what a pain - but payroll needs to be correct). I found out that this was the practice by asking the tape librarian what we did... then I asked how many 'passes' that the bad tapes had been used for (it varied from 350-1200 partly depending on brand, apparently the premium tapes are more expensive for a reason).
We did two things...
1) Stopped buying 'bargain' tape. Specified brand and type (eg; Scotch, Black Watch) without regard to price except that if another vendor could provide the same media for less, we'd buy from them.
2) When I explained to the tape librarian that bad tapes could make for late paychecks...we decided to place new tapes into service as 'Payroll Scratch' and used them for 50 passes before letting them into the 'General Scratch' pool. The tapes were moved to a 'Test Scratch' pool after 250 passes, where they would serve till they failed on some hapless programmer's test run.
I think that in correct temp/humidity without magnetic 'disruption' (including electric wires) that the tapes would last for at least 12 years (our maximum retention), although in reality the data seemed to be reformatted sooner than that (and history tapes converted - re-written)...
Optical media isn't 'Immortal' either, although it can be very handy, since it is direct access. Optical media may be vunerable to 'hazing' as well as scratches - some are made with this in mind (use compuonds that are more resistant to oxydation, perhaps gold...).

by mstoumba

How much data are you going to backup is the key here. Even small shops keep alot of data. Fifteen years ago. You might have gotten away with backing up under 4 gig. Not today ! I still use tape and it work's fine. If you are to cheep to replace your DAT or DLT tapes each year then I guess you will loose data. I backup to Hard drive first and the to tape its faster that way. Our backup window get's smaller and smaller that's why I don't thing DVD is a viable at this time. Also look at the "LEGAL STUFF".If you have the data then you will have to give it to the other side in a legal dispute.

by chasster

Much of this answer is within the quality of tape purchased.

If VIDEO quality tape (cheaper) is being used for DATA storage/backup then you will experience additional errors and shorter TOL of the media.

If proper quality DATA tape is purchased (more expensive) then the "ule of thumb" varies from 10 to 100 usages before replacing the physical tape.

Often tape is used for only a short amount of its total length and recovery/reuse problems will occur within the first 10 to 50 feet of tape. In a cartridge tape, well cutting it off is not viable so that you could restablish a new BOT as you used to do with Reel Tape.

Bottom line - Buy Cheap - Replace Often.

cheers,
cs

by john.a

We were running Veritas NetBackup Pro, which is actually a fantastic product, however our tape drive we were using at a cost of $5000, sucked. We kept buying new tapes, thinking they were getting dirty. Plus we had issues getting the tapes offsite each night.(Sound familiar? Finally, after loosing some data we went with a remote online backup provider - actually we tried out 3 providers over a 14 month period. I encourage everyone to try out remote backup service (e-vaulting). Word of caution though - we found out most remote online backup providers use incremntial for daily backups. Our 342MB initial backup turned into 4.2GB in 7 months with incremntial! Finally we stumbled across one that does "byte level" backups. Without boring everyone with details what byte level backup is, the bottom line is our 342MB backup was 384MB after 6 months, not 4.2GB. We settled with Subterranean Data Services - www.remote-online-backup.com My data is stored 13feet underground. It is sweet to have a backup every night that is offsite. Oh, we pay $26.95 month. Not $$$ for 4.2GB and growing.

by harold.j.ballinger

We've been disappointed with the cost/benefit ratio of tape storage solutions and lifespan when compared to the low price of high speed usb external hard drives. The tape drives themselves are so extremely expensive, and the tapes are only good for a very finite number of backups, so cost justification for multiple external hard drives is easy to make.

Additionally, these drives are much faster, have greater capacity, and are very affordable. We use hard drives or WAN/Internet backup solutions almost exclusively for the purpose of rotating backups. In some cases we use them for archiving purposes as well if the amount of data exceeds what would be easily archived onto DVD media.

These external units are also very portable, and if you purchase one of the models that is USB powered, there is only a single cable to unplug and replug when swapping your drives.

As a side note, since silicon is much more accepting when it comes to heat, some of our clients with fire-proof safes consider placing these external hard drive units in their fire proof safes instead of plastic tapes that might melt under higher temperatures.

As far as the lifespan of tapes goes, I think that everyone has to have a living disaster recovery plan in place in which you not only test your backups and restores, but in which you evaluate the quality and accessibility of your data. I believe that the longevity of tape life will vary depending on your environmental factors of tape storage, tape drive equipment choice, the actual choice of tape technology, and quality of the manufacturing process for a given tape/manufacturer. For an answer that fits your environment, I would test your tapes over time to see how long they last. I'm sure that someone that keeps their server and tapes in their boiler room where temperature and environmental factors are not controlled, and does not regularly run a cleaning tape, and that buys different manufacturer's tapes based on what was the cheapest this month will have wildly differing results than someone with an organized approach to managing their data.

As a side note, I have found many a client that is required by law to maintain archives for X number of years, but they never thought about the fact that they no longer have a drive that will read their archived data (and that the manufacturer is no longer existent).

The question that was raised brings to light that getting the data "backed up" is only the tip of the iceberg.

by sue's comment

First wish is to store up-to-date data off site in case of a disaster. Worse case scenario - if the tape fails you have lost 24 hours data.

Second is to be able to replace a deleted file/folder or compare a recent copy to an prior one. The file is only available if tape hasn't been overwritten. Failed tape may possibly mean the loss of one file/folder or it may mean no loss at all as the data could well be available on another tape.

Third is to be able to read the data after x years. Not only do you need hardware but you also need to consider whether the software is still available! Some say that archived files should ideally be stored in csv format!

None of these issues answers how long a tape will last. I have been using DLT tapes for 20 months and I have known one tape to fail so far. I have also had a hard drive fail in that time and a CDs/DVDs that failed to read. There is a law that says this will always happen at the wrong time so if the data is vitally important back it up twice - preferabaly to two different media at different sites.

We do daily backups and then overwrite Mondays', Wednesday's, Thursday's and Friday's tapes weekly. The first Tuesday of the month gets a new tape and the tape is then stored offsite. This keeps end of month data (like payroll) relatively secure. For the remaining Tuesdays there is a cycle of 4 tapes though the fourth one doesn't get used a lot. I guess that Monday's, Wednesday's, Thursday's and Friday's tapes are likely to fail first!

Being paranoid (experienced)I also backup our important databases to CD on the 1st of the month (copied to hard drive first) and I copy our Standards (Word templates and spreadsheets) to CD whenever there is an important change. These CDs are stored on-site so if a user deletes a standard it is easily restored. I can also check what data was on the database at the beginning of the month in case of query. All DLT tapes go home with me (nicely 23 miles from site) so it is nice to have an on-site archive.

BTW I still have some punched cards - sentimental value - I really miss the sound they made passing through the reader! A real zzzziiiiiiipppppp!

by dduncan4

HP has tape life down covered. I only use HP tapes and drives. My backup software can monitor the usage and tells me when it's time to get rid of this tape. It may be more expensive, but just wait till you need to restore that backup.

by warren sever, mcp

If your site is a large campus, you could send it to another building to your backup server(s) there. This has enough value to even build a small backup "vault" onsite at a large campus. Then you have some disaster proofing built right in.

Smaller backups can be sent to an offsite server via the Internet if you have a big enough pipe.

Removable drives would be an option only for a small system. Much care would need to be taken to ensure the drives don't get damaged while transporting on/offsite.

"Think outside the box of tapes."

by vip_bsd

I don't think anyone has mentioned this so yeah.
While media life in terms of time is good for storage, its less useful when your using sets of media in time cycles ( 1 week, 2 weeks etc). When I evaluate media I run it thru multiple read write cycles. For example:
Write to media
wait 30 minutes
Read from media
wait 30 minutes

Doing this 24/7 on a tape until it dies will give you a decent idea about its real world life span. For more accurate results do this on multiple tapes to get an average.

by reonline

I think most of the replies have hit the point. Check the documentation for your tapes, it should indicate the number of useable hours for that type of media. Also, your backup software may keep track of the number of hours your media has been in use and stop using the media once it reaches that number of hours.

As for archiving, if you are looking to restore from tape media that is several years old, you are taking the risk that it may no longer be readable. Also, if it is something that is several years old, you may have moved onto a different backup technology and may no longer have the hardware available to read the media. I have had this happen due to harware failures.

This discussion is closed to new replies.