Questions

What type of storage/backup system

+
0 Votes
Locked

What type of storage/backup system

dwillms
Hey everyone,
I just started with a medium sized business doing IT work, and they have asked me to research types of backup solutions. At this point we are running a server that holds about 350 GB of data, and we have 2 remote offsite locations which they would also like to back up as well. The total approximate size would be around 500GB, but they'd like to start off with 1TB and have the option for expandability as well. Right now they are using tape backups, and I believe would like to get rid of that somehow, but the data would also have to be stored offsite as well somewhere, in case of some sort of disaster at the main office. In the small bit of research I've done, I think for this size of storage would a NAS device not be the best option, running Windows Storage Server 2003 or similar, and then maybe the possibility of a second NAS running offsite or at one of the other locations. If you ran both of these NAS's with RAID 1 or RAID 5, that would give you multiple data redundancy, and also offsite storage, and from the prices I've been able to find, it seems like that would be much cheaper than a similar SAN solution. If the company did go with something like a SAN device, they would most likely go with iSCSI, not Fiber Channel, just for the reduced cost. Of course a third option would be something like an automated tape loader, but if they want to get away from tape storage, that may not be the best option. Also, what kind of companies provide offsite storage services such as this? What would services like that cost?

Thanks alot for your help!
  • +
    0 Votes
    HAL 9000 Moderator

    Is about to migrate to in the near future.

    If they are seriously looking at Vista or whatever the Longhorn Server product becomes known as you could have problems restoring your backup in the event of hardware failure or something worse happening.

    Vista currently doesn't like to recover data from HDD that the particular application was not installed to so you can save from one computer and not read on another. The Longhorn Server Betas has similar problems so that may be worth a look at.

    NAS is fine provided that you can store off site and have a large enough bandwidth to transfer the data but you'll have to make sure that it's actually recoverable on different hardware.

    You don't need anything really serious to happen to make your existing hardware unusable a simple over voltage event can cause this to happen in a fraction of a second. Or a burst water main can flood an area quite quickly and destroy all the computer equipment.

    Personally if you don't like the idea of Auto Loaders and Tape I would be seriously looking at Optical Storage in the form of Dual Layer DVD but even these have a problem when you add Vista to the mix so what works perfectly well today may not work after an incident where hardware has to be replaced.

    Despite the slowness of the backup Auto Loaders still have the advantage of being both reliable and safe and with the cost of tape as low as it currently is you could have several generations backed up and rotated instead of just relying on the one backup without any Generational Backup being available in case something goes wrong.

    No matter what you deploy try recovering the data on different hardware just to make sure that it really works as there is nothing worse than having a backup that you can not access.

    Col

    +
    0 Votes
    dwillms

    Well we will not be moving to Vista anywhere in the near future, probably not for at least 2-3 years at this point I would think, so that's not a problem. I don't think I was quite correct in my assumption that he wanted to move totally away from tape. I think now I'm thinking something like a RAID 1 or RAID 10 NAS with all onsite and offsite servers with a weekly tape backup would be more along the lines of what we need. Would a SAN not really be useful in this type of application if we would be going with an iSCSI SAN anyways?

    +
    0 Votes
    HAL 9000 Moderator

    In 1974 we had a fairly decent flood here which put a lot of the server rooms well below the water line so the off Site Storage that was organised didn't do much good as it was as deeply underwater as the main site.

    So since then if we are looking at off site storage I try to have it in a completely different area well away from the main site but even that doesn't guarantee that it will not be adversely affected if you get severe weather conditions. I've also had to replace several iSCSI HDD's from a SAN because they don't get treated as well as they should when being moved around. I've only had to send one drive away for data recovery but even that one was very expensive because the person responsible for it didn't realise just how fragile they are. Where as a tape mistreated and left in direct sunlight in a car the cartridge will warp but you can always pull out the tape and place it into another cartridge and have it running quite quickly which unfortunately isn't possible with any HDD Solution. But as I say it all depends on how solid a DR Plan you are instigating is.

    In small business I've used something as simple as USB HDD's and they work OK but for full protection I still don't think that you can beat an Auto Loader with Incremental Backup that are regularly checked on both the original hardware and some replacement hardware just to be on the safe side.

    But as they say any DR Plan is only as strong as its weakest link.

    Col

    +
    0 Votes
    dwillms

    I was sort of rethinking my options. I was thinking on somehow doing live mirroring of all of the offsite and onsite servers onto a NAS or SAN (the offsite servers aren't that big, only about 2 or 3 computers on each of the 3, so bandwidth shouldn't be an issue). Then you could do incremental backups of that NAS/SAN as well as fully weekly backups. That way you have quick onsite storage if something goes wrong, your databases (they are relatively small) won't lose any new data in case of failure, and you still have safe offsite tape storage.

    +
    0 Votes
    HAL 9000 Moderator

    If it suits the needs of the business and what eventualities that they are planing for.

    Col

    +
    0 Votes
    dwillms

    Okay so I just had a small meeting with the main IT person. Basically we are looking at something that can be expandable easily (500gb of data now, approx 25-35% growth of data per year, possible addition of servers in the near future), be able to be stored off site in some way safely.

    He may want to move to virtualization in the near future. Right now there are 3 on site servers, 2 mainly are used, 1 is not very used. There is also 2 off site offices each with their own storage (they don't have independent servers right now, but NAS devices instead).

    We're looking at a way to easily backup all the data to a central location (and then from there off site by tape storage, or another NAS/SAN off site, etc.) We also need something that in case of total server failure we can get up and running again fast. Would a NAS device be suitable for this, or would a SAN be more of what we are looking for in this case? Would there be any reason to move to Fiber Channel instead of iSCSI assuming we go with SAN?(he said our network is not really being heavily taxed right now, and the main server is definitely not being taxed) He sounded like he didn't want to move into a NAS device, I'm not sure why, but in my eyes there is not much difference other than block level storage correct (assuming iSCSI), and yet SAN devices seem to be much more expensive.

    Thanks alot for all of your help HAL9000!

    +
    0 Votes
    HAL 9000 Moderator

    With their existing NAS Devices and that is the main reason why this person isn't overly interested in going that route.

    SAN will work quite well and for a 500 GIG Backup it would be much faster to get up and running again over tape but depending on how well the Drives are stored may not be quite as reliable. Then again depending on the tape drive used you may find that a new Tape drive can not read the existing tapes so that is something to look at as well. Whenever I deploy Tape Drives I always buy a second Drive and place it with the Off Site Storage of Tapes so that in the event of a total failure of hardware I always know that I have a working Tape drive that can read the tapes. It goes without saying that I always test the drive before packing it up for storage.

    As far as fast fairly reliable backups are concerned either NAS or SAN will work out OK but it depends on how much money is in the budget and what previous problems that they have encountered with existing hardware that may swing the build to one option over another or even personal preference and this is where things get very hard. When there is a Personal Bias involved you are better off supplying what the person wants instead of something that may be technically superior as they will never be happy with what they consider as Second Best not at all technical but something very important to keep in mind none the less as you don't want to be constantly called out to fix a problem that only exist in the mind of the person responsible for the hardware.

    Moving to more servers latter may cause problems as whatever Longhorn Server gets called will be available fairly soon and there will most likely be integration problems with the existing hardware so that is something to keep in mind. I've only been playing with the Beta Version a reasonably short time so I'm not fully sure of everything involved and what changes will be made prior to the RTM stage but it currently doesn't integrate at all well with 2003 Servers not sure what will happen if 2003 SBS is involved as that isn't something that I've tested with as yet.

    As for Off Site Data Storage there are several things that need to be kept in mind the first is if there is any Legal Obligations that they are required to meet with in Data Protection this may restrict the available off Site Storage Facilities. Then you need to make sure that they are far enough away not to be affected by some local adverse Climatic Conditions but still close enough to be able to drive there and retrieve a copy just in case of Power Failures preventing a backup copy being transfered across the Net.

    While a couple of hundred miles sounds like a long way off that can shrink dramatically depending on location if floods are involved and the roads can be totally blocked off by Floods and people trying to escape flood waters so what at first seems like a great solution doesn't work out all that well and can even prevent a business from getting up and running in the event of a total loss of hardware and the most recent Backup Data. If tape is used and a complete loss if NAS, SAN is used and not collected prior to the evacuation of the main business site.

    As for costs as I'm in AU I'm probably no the right one to give any advice on this as the storage prices differer dramatically between the different countries and what I've always tried to do in the past with any business is to get a full backup held in a remote office just so that the data is never out of the companies control but as I've worked both Medical and Banks I'm more than a bit paranoid about Data Security and the Legislation that accompanies it. Mainly because the person instigating the system can be held responsible for any Data Theft and here with the Legislation that means Jail Time but again it all depends on the sensitivity of the data being stored and the Privacy Legislation where you are.

    Col

    +
    0 Votes
    dwillms

    Okay I was just wondering if you could answer a few questions that I have. Let's say you're running either a NAS or SAN device, could you do something like still keep DAS hard drives on your server (our server isn't being really taxed at the moment, so no problems there), but then mirror those DAS onto the NAS/SAN device (this is just an idea, I don't know if you can even do this...)

    What would happen in the case of complete server failure, would there be any interruption to the users, could they still access their files right away like nothing ever happened? If not, would there be some sort of backup plan for what the users could do to access their files in this case?

    Would this even make sense to do this, it seems like it would reduce or even completely eliminate this problem in the case of a server failure. What is the typical setup when businesses merge to a SAN/NAS device?

    Edit: Also I've found a few companies who offer software based SAN solutions, such as SANMelody. Would configuring your own server and then installing this type of software be a better option than something like a pre-built SAN?

    Thanks!

    +
    0 Votes
    HAL 9000 Moderator

    A server to a NAS/SAN and from previous experience if the workstations are pointed to the server as a File Server if it failed completely the data access would also fail. If something like that happened the only option would be to point all the workstations to the NAS/SAN for files and bypass the Server which is doable provided that the server isn't the DCHP authorising point as well.

    Most of the NAS/SAN installs that I have done refer all workstations to the NAS/SAN for Files and by pass the server and allow it to do other things unrelated to File Sharing but this is always in a heavy use environment.

    As for a Software NAS or whatever that would effectively be on the server so if the server breaks you would loose everything at the one time. That's just an opinion though as I've never actually used a Software NAS or RAID as they appear to me to give you the worst of everything possible and only have some advantage when you have a tight budget to work with and should only ever be used as a stop gap measure where possible.

    But again it depends on the business and what they actually need and how much redundancy that you need to build in.

    Col

    +
    0 Votes
    dwillms

    A software SAN. It wouldn't be run off of the same server as the main, but rather a separate server, and would be running software like "SANMelody" I was just reading a bit about it the other day. To tell you the truth I don't think it is that much cheaper than getting a true SAN, so I don't know if there would be much of a point. Also, the main server right now is not really under heavy usage. CPU usage is usually low, and 4GB of ram is about 1/2 to 2/3's used, so it is fine doing some file sharing. I just though mirroring all the data from all of the servers to a SAN/NAS would be a little bit useful in case something goes wrong, you can just tell the computer to look for the data there instead, I just don't know if you can actually do that...

    PS - Thanks for all the help again!

    +
    0 Votes
    TonytheTiger

    stored at two off-site locations since October, 2005 and have been very pleased with the results.

    This may give you an idea.

    http://techrepublic.com.com/5208-11187-0.html?forumID=86&threadID=193085&messageID=1992859

    +
    0 Votes
    dwillms

    That sounds quite similar to what I was thinking of. With your mirrored server, if the main server would crash would you be fine still to users accessing files? That's my goal, although it isn't a necessity, as long as we won't have more than an hour or two of downtime in the case of server failure I think that would be acceptable, I'm just trying to have it be instant if that is possible. The only problem with that in our case is that our remote offices are across the country so it won't be able to be connected fast enough to do backups over it I don't think, so I think tape would be a better solution, as long as it was kind of a "worst case scenario" because they take so long to restore files.

    +
    0 Votes
    TonytheTiger

    If main server fails, I change one line in the logon script, users log off and back on, and they have yesterday's data. There are ways to set up redundant servers with instant failover, but that's not what we wanted. We intentionally wanted yesterday's data.

    Our two remote sites are 10 and 40 miles away, and it's not fast enough for backup (T-1), but plenty fast enough for restores.

    Have you considered perhaps doing full backups locally and incrementals remotely?

    +
    0 Votes
    dwillms

    Sounds about what I'm looking at doing if possible. I was thinking if possible I'd like a live backup on all servers, and this could very well be possible, as the offsite locations are extremely small, especially compared to the main office (offsite offices are 2 or 3 users max, the onsite is about 50 or 60) so they would have very little data, and wouldn't be creating much more data either. If that couldn't be done, incremental backups of the offsite servers would be fine as well.

    May I ask how you have it all setup so you only have to change one line in the login scripts?

    +
    0 Votes
    TonytheTiger

    in the logon batch.

    Set fileserver=FS0001

    then , for example, the user's private network drive is a share with their username followed by a '$' so the share is hidden, so...

    net use W: \\%fileserver%\%username%$

    Since all our shares are on the same file server, we just have to change it in the batch file once (FS0001 to FS0002).

    Also look at kixstart for scripting.

  • +
    0 Votes
    HAL 9000 Moderator

    Is about to migrate to in the near future.

    If they are seriously looking at Vista or whatever the Longhorn Server product becomes known as you could have problems restoring your backup in the event of hardware failure or something worse happening.

    Vista currently doesn't like to recover data from HDD that the particular application was not installed to so you can save from one computer and not read on another. The Longhorn Server Betas has similar problems so that may be worth a look at.

    NAS is fine provided that you can store off site and have a large enough bandwidth to transfer the data but you'll have to make sure that it's actually recoverable on different hardware.

    You don't need anything really serious to happen to make your existing hardware unusable a simple over voltage event can cause this to happen in a fraction of a second. Or a burst water main can flood an area quite quickly and destroy all the computer equipment.

    Personally if you don't like the idea of Auto Loaders and Tape I would be seriously looking at Optical Storage in the form of Dual Layer DVD but even these have a problem when you add Vista to the mix so what works perfectly well today may not work after an incident where hardware has to be replaced.

    Despite the slowness of the backup Auto Loaders still have the advantage of being both reliable and safe and with the cost of tape as low as it currently is you could have several generations backed up and rotated instead of just relying on the one backup without any Generational Backup being available in case something goes wrong.

    No matter what you deploy try recovering the data on different hardware just to make sure that it really works as there is nothing worse than having a backup that you can not access.

    Col

    +
    0 Votes
    dwillms

    Well we will not be moving to Vista anywhere in the near future, probably not for at least 2-3 years at this point I would think, so that's not a problem. I don't think I was quite correct in my assumption that he wanted to move totally away from tape. I think now I'm thinking something like a RAID 1 or RAID 10 NAS with all onsite and offsite servers with a weekly tape backup would be more along the lines of what we need. Would a SAN not really be useful in this type of application if we would be going with an iSCSI SAN anyways?

    +
    0 Votes
    HAL 9000 Moderator

    In 1974 we had a fairly decent flood here which put a lot of the server rooms well below the water line so the off Site Storage that was organised didn't do much good as it was as deeply underwater as the main site.

    So since then if we are looking at off site storage I try to have it in a completely different area well away from the main site but even that doesn't guarantee that it will not be adversely affected if you get severe weather conditions. I've also had to replace several iSCSI HDD's from a SAN because they don't get treated as well as they should when being moved around. I've only had to send one drive away for data recovery but even that one was very expensive because the person responsible for it didn't realise just how fragile they are. Where as a tape mistreated and left in direct sunlight in a car the cartridge will warp but you can always pull out the tape and place it into another cartridge and have it running quite quickly which unfortunately isn't possible with any HDD Solution. But as I say it all depends on how solid a DR Plan you are instigating is.

    In small business I've used something as simple as USB HDD's and they work OK but for full protection I still don't think that you can beat an Auto Loader with Incremental Backup that are regularly checked on both the original hardware and some replacement hardware just to be on the safe side.

    But as they say any DR Plan is only as strong as its weakest link.

    Col

    +
    0 Votes
    dwillms

    I was sort of rethinking my options. I was thinking on somehow doing live mirroring of all of the offsite and onsite servers onto a NAS or SAN (the offsite servers aren't that big, only about 2 or 3 computers on each of the 3, so bandwidth shouldn't be an issue). Then you could do incremental backups of that NAS/SAN as well as fully weekly backups. That way you have quick onsite storage if something goes wrong, your databases (they are relatively small) won't lose any new data in case of failure, and you still have safe offsite tape storage.

    +
    0 Votes
    HAL 9000 Moderator

    If it suits the needs of the business and what eventualities that they are planing for.

    Col

    +
    0 Votes
    dwillms

    Okay so I just had a small meeting with the main IT person. Basically we are looking at something that can be expandable easily (500gb of data now, approx 25-35% growth of data per year, possible addition of servers in the near future), be able to be stored off site in some way safely.

    He may want to move to virtualization in the near future. Right now there are 3 on site servers, 2 mainly are used, 1 is not very used. There is also 2 off site offices each with their own storage (they don't have independent servers right now, but NAS devices instead).

    We're looking at a way to easily backup all the data to a central location (and then from there off site by tape storage, or another NAS/SAN off site, etc.) We also need something that in case of total server failure we can get up and running again fast. Would a NAS device be suitable for this, or would a SAN be more of what we are looking for in this case? Would there be any reason to move to Fiber Channel instead of iSCSI assuming we go with SAN?(he said our network is not really being heavily taxed right now, and the main server is definitely not being taxed) He sounded like he didn't want to move into a NAS device, I'm not sure why, but in my eyes there is not much difference other than block level storage correct (assuming iSCSI), and yet SAN devices seem to be much more expensive.

    Thanks alot for all of your help HAL9000!

    +
    0 Votes
    HAL 9000 Moderator

    With their existing NAS Devices and that is the main reason why this person isn't overly interested in going that route.

    SAN will work quite well and for a 500 GIG Backup it would be much faster to get up and running again over tape but depending on how well the Drives are stored may not be quite as reliable. Then again depending on the tape drive used you may find that a new Tape drive can not read the existing tapes so that is something to look at as well. Whenever I deploy Tape Drives I always buy a second Drive and place it with the Off Site Storage of Tapes so that in the event of a total failure of hardware I always know that I have a working Tape drive that can read the tapes. It goes without saying that I always test the drive before packing it up for storage.

    As far as fast fairly reliable backups are concerned either NAS or SAN will work out OK but it depends on how much money is in the budget and what previous problems that they have encountered with existing hardware that may swing the build to one option over another or even personal preference and this is where things get very hard. When there is a Personal Bias involved you are better off supplying what the person wants instead of something that may be technically superior as they will never be happy with what they consider as Second Best not at all technical but something very important to keep in mind none the less as you don't want to be constantly called out to fix a problem that only exist in the mind of the person responsible for the hardware.

    Moving to more servers latter may cause problems as whatever Longhorn Server gets called will be available fairly soon and there will most likely be integration problems with the existing hardware so that is something to keep in mind. I've only been playing with the Beta Version a reasonably short time so I'm not fully sure of everything involved and what changes will be made prior to the RTM stage but it currently doesn't integrate at all well with 2003 Servers not sure what will happen if 2003 SBS is involved as that isn't something that I've tested with as yet.

    As for Off Site Data Storage there are several things that need to be kept in mind the first is if there is any Legal Obligations that they are required to meet with in Data Protection this may restrict the available off Site Storage Facilities. Then you need to make sure that they are far enough away not to be affected by some local adverse Climatic Conditions but still close enough to be able to drive there and retrieve a copy just in case of Power Failures preventing a backup copy being transfered across the Net.

    While a couple of hundred miles sounds like a long way off that can shrink dramatically depending on location if floods are involved and the roads can be totally blocked off by Floods and people trying to escape flood waters so what at first seems like a great solution doesn't work out all that well and can even prevent a business from getting up and running in the event of a total loss of hardware and the most recent Backup Data. If tape is used and a complete loss if NAS, SAN is used and not collected prior to the evacuation of the main business site.

    As for costs as I'm in AU I'm probably no the right one to give any advice on this as the storage prices differer dramatically between the different countries and what I've always tried to do in the past with any business is to get a full backup held in a remote office just so that the data is never out of the companies control but as I've worked both Medical and Banks I'm more than a bit paranoid about Data Security and the Legislation that accompanies it. Mainly because the person instigating the system can be held responsible for any Data Theft and here with the Legislation that means Jail Time but again it all depends on the sensitivity of the data being stored and the Privacy Legislation where you are.

    Col

    +
    0 Votes
    dwillms

    Okay I was just wondering if you could answer a few questions that I have. Let's say you're running either a NAS or SAN device, could you do something like still keep DAS hard drives on your server (our server isn't being really taxed at the moment, so no problems there), but then mirror those DAS onto the NAS/SAN device (this is just an idea, I don't know if you can even do this...)

    What would happen in the case of complete server failure, would there be any interruption to the users, could they still access their files right away like nothing ever happened? If not, would there be some sort of backup plan for what the users could do to access their files in this case?

    Would this even make sense to do this, it seems like it would reduce or even completely eliminate this problem in the case of a server failure. What is the typical setup when businesses merge to a SAN/NAS device?

    Edit: Also I've found a few companies who offer software based SAN solutions, such as SANMelody. Would configuring your own server and then installing this type of software be a better option than something like a pre-built SAN?

    Thanks!

    +
    0 Votes
    HAL 9000 Moderator

    A server to a NAS/SAN and from previous experience if the workstations are pointed to the server as a File Server if it failed completely the data access would also fail. If something like that happened the only option would be to point all the workstations to the NAS/SAN for files and bypass the Server which is doable provided that the server isn't the DCHP authorising point as well.

    Most of the NAS/SAN installs that I have done refer all workstations to the NAS/SAN for Files and by pass the server and allow it to do other things unrelated to File Sharing but this is always in a heavy use environment.

    As for a Software NAS or whatever that would effectively be on the server so if the server breaks you would loose everything at the one time. That's just an opinion though as I've never actually used a Software NAS or RAID as they appear to me to give you the worst of everything possible and only have some advantage when you have a tight budget to work with and should only ever be used as a stop gap measure where possible.

    But again it depends on the business and what they actually need and how much redundancy that you need to build in.

    Col

    +
    0 Votes
    dwillms

    A software SAN. It wouldn't be run off of the same server as the main, but rather a separate server, and would be running software like "SANMelody" I was just reading a bit about it the other day. To tell you the truth I don't think it is that much cheaper than getting a true SAN, so I don't know if there would be much of a point. Also, the main server right now is not really under heavy usage. CPU usage is usually low, and 4GB of ram is about 1/2 to 2/3's used, so it is fine doing some file sharing. I just though mirroring all the data from all of the servers to a SAN/NAS would be a little bit useful in case something goes wrong, you can just tell the computer to look for the data there instead, I just don't know if you can actually do that...

    PS - Thanks for all the help again!

    +
    0 Votes
    TonytheTiger

    stored at two off-site locations since October, 2005 and have been very pleased with the results.

    This may give you an idea.

    http://techrepublic.com.com/5208-11187-0.html?forumID=86&threadID=193085&messageID=1992859

    +
    0 Votes
    dwillms

    That sounds quite similar to what I was thinking of. With your mirrored server, if the main server would crash would you be fine still to users accessing files? That's my goal, although it isn't a necessity, as long as we won't have more than an hour or two of downtime in the case of server failure I think that would be acceptable, I'm just trying to have it be instant if that is possible. The only problem with that in our case is that our remote offices are across the country so it won't be able to be connected fast enough to do backups over it I don't think, so I think tape would be a better solution, as long as it was kind of a "worst case scenario" because they take so long to restore files.

    +
    0 Votes
    TonytheTiger

    If main server fails, I change one line in the logon script, users log off and back on, and they have yesterday's data. There are ways to set up redundant servers with instant failover, but that's not what we wanted. We intentionally wanted yesterday's data.

    Our two remote sites are 10 and 40 miles away, and it's not fast enough for backup (T-1), but plenty fast enough for restores.

    Have you considered perhaps doing full backups locally and incrementals remotely?

    +
    0 Votes
    dwillms

    Sounds about what I'm looking at doing if possible. I was thinking if possible I'd like a live backup on all servers, and this could very well be possible, as the offsite locations are extremely small, especially compared to the main office (offsite offices are 2 or 3 users max, the onsite is about 50 or 60) so they would have very little data, and wouldn't be creating much more data either. If that couldn't be done, incremental backups of the offsite servers would be fine as well.

    May I ask how you have it all setup so you only have to change one line in the login scripts?

    +
    0 Votes
    TonytheTiger

    in the logon batch.

    Set fileserver=FS0001

    then , for example, the user's private network drive is a share with their username followed by a '$' so the share is hidden, so...

    net use W: \\%fileserver%\%username%$

    Since all our shares are on the same file server, we just have to change it in the batch file once (FS0001 to FS0002).

    Also look at kixstart for scripting.