Enterprise Software

Cloud storage for primary storage as an alternative to file servers

For a network administrator, few tasks share the frustration levels of administering a file server. IT guru Rick Vanover explains how some cloud solutions can solve file server issues.

There is nothing that drives me crazier than unstructured data on a file server. If there is an application that we can blame, that helps. If the data is a smattering of personal and workgroup data, self-censorship starts to kick in. Of course we can do all of the standard remediation attempts to consolidate file servers and put concise group membership with appropriate permissions.

One option that exists to administrators now is to leverage cloud storage for the primary instance of file servers. One solution is the Nasuni Filer to replace NAS servers that function as file servers in your organization. While I’m quite familiar with a number of cloud storage solutions, I have long thought what will make a “cloud” solution very relevant to the mainstream organization is a turnkey solution.

The Nasuni Filer is very simple in that it is distributed as an open virtual format (OVF) virtual machine. The virtual machine is then assigned a local cache on storage resources on-premise. The local cache is a nominal storage allocation, 500 GB for example, that is the most commonly accessed data in the file server’s namespace. The rest of the data is in a storage cloud with the ingress and egress traffic managed by the Nasuni Filer. The Nasuni Filer is also smart in that you can have the data reside in the Amazon Simple Storage Service (S3) cloud, Nirvanix Storage Delivery Network (SDN), Iron Mountain Archive Services Platform (ASP), or Rackspace Cloud Files.

The Nasuni Filer does a few things that make its architecture attractive. First of all, the OVF deployment is attractive as any administrator with a virtualized infrastructure can do so quickly. The second thing I really like is that the filer shows up on your local network to be managed in Microsoft Active Directory for full permission and share management through familiar interfaces. Figure A below shows the Filer’s architecture: Figure A

Figure A

Click image to enlarge
The Nasuni Filer also starts to get smart with the data before it uploads it to the cloud. Realizing that the transfer bandwidth is the most sensitive link in a cloud-based storage solution, Nasuni performs four critical processes on data before it is uploaded to the cloud. These are: chunking into blocks, de-duplication, compression, and then encryption. The data is protected with OpenPGP AES-256 bit encryption. Figure B shows this pre-transfer process: Figure B

Figure B

Click image to enlarge

Nasuni was introduced to me by one of my colleagues, Greg Knieriemen, who produces the popular Infosmack podcast. Episode 52 features Andres Rodriguez, CEO and founder of Nasuni. I highly recommend that you check out this episode not just for this solution, but how cloud storage has evolved out of necessity as well as what it can and cannot do.

I am always trying to find inroads to a cloud solution when the conditions can be right. With cloud storage as easy as being a CIFS endpoint within your Active Directory domain on your network, does it appeal to you? Share your comments below.

About

Rick Vanover is a software strategy specialist for Veeam Software, based in Columbus, Ohio. Rick has years of IT experience and focuses on virtualization, Windows-based server administration, and system hardware.

47 comments
crowleye
crowleye

We dont want to go to cloud storage, but we have way too many servers for a company our size, and they are always out of date in some way. Security patches seem to come out every 3 hours lately; we can't keep up. No matter how much space we throw at the users, they suck it up and beg for more. (We aren't allowed to throw out old data...that's another blog altogether.) It seems like the cloud is our only hope of keeping at least the servers in compliance, by getting someone else to do it.

david.hunt
david.hunt

It puts your data outside your company. The way people talk about contracts, you'd think nobody ever breaks them. I don't suppose the contract includes liquidated damages if the data is lost or unavailable. First rule of outsourcing: Never outsource the critical aspects of your core business. "Cloud" is just the new politically correct term for "Outsourcing". I don't think the term actually fools too many people.

Chuckchuckj
Chuckchuckj

It looks like the data will leave the Copr. network and being store in a site provide by Vendor? is there any issues with compliance? SOX HIPPA etc...

Systems Guy
Systems Guy

Why would anyone put their company data, let alone their personal data out in the cloud? I'm sure most cloud storage vendors have good intentions of keeping things safe and secure and available but do a reality check. Don't any of you managers, etc. read the papers? This cloud storage is an accident waiting to happen. Someone somewhere will get hacked, will loose data, etc. Business data/information is too priceless to let some 3rd party manage. This comment is directed to ALL cloud storage vendors. The gulf oil fiasco wan't supposed to happen. Space shuttles are supposed to blow up. Lunatics aren't supposed to fly jets into buildings. In my opinion, any manager making the decision to move their companies data to cloud storage should be sued along with all the other parties when the storage breech eventually happens. What's more important company profits or company data (rhetorical question, it ALWAYS boils down to money).

reisen55
reisen55

A server is an in-house, hands-on device and storage is cheap these days. If your data is sensitive, HIPAA or financial, cloud computing is shot through with security holes. I like to keep my lawyer's address in my rolodex and not as a friend in courtroom. It may be neat and nifty, but CLOUD storage does not exist. The data has to reside on a server SOMEWHERE in the world and to have under my care and control allows me to sleep better at night.

elrico-fantastica
elrico-fantastica

but... i still am weary of the thought of storing critical information in the cloud.. also as with all cloud services it brings another company into the failure chain as well as the connection to the cloud.. questions pop up like.. what if they go bankrupt, how do i get my data back what if my internet link is down and we need to access something from the archive how trustworthy and competent are the staff can i guarentee its security.. i dunno i think theres still too many questions surrounding cloud based tech for me to make any sort of leap.

Walthy
Walthy

I just went to a presentation by Dell and their subsidiary EqualLogic yesterday. They basically have a storage subsystem that would be able to consolidate all of that mess you describe and provide unprecedented fault tolerance and connectivity options. One fellow I spoke with at the seminar is using a much more expensive solution from a competitor, but says if he had had the choice he would definitely have gine with this solution. Anyway, check it out and no I'm not a salesman for any of this stuff.

Tony Hopkinson
Tony Hopkinson

some people to think about what they are doing. When the costs get explicit, as in cheque to vendor, bean counters start paying serious attention. Make sure you do the required arse covering, it could get messy.

anilkool
anilkool

Storage on Cloud is like giving your girlfriend your credit-card and expecting all will be smooth and safe :)-

Walthy
Walthy

I am not a believer in outsourcing core business. I am not a believer in cloud file services, it just doesn't make sense to me to keep your file system off site. It does make sense to me to keep disaster recovery storage off site and physically separated from my location. My ideal, and I'm in the process of building that right now for very small businesses, is a local file system, on premises near real-time backup, nearby offsite backup, and finally physically distant cloud backups. My goal is to provide quick recovery from any disaster from a lost file, to a burned out or flooded data system, to a natural catastrophe such as an earth quake, flood, or volcanic activity (I live near Mt St. Helens). No data leaves the local site without deduplication, block level breakout separation (even if you can see the data it probably wouldn't make any sense and you would never see any two consecutive blocks together), and encryption before it leaves the premises. There will be nearby local backup and dispersed cloud storage. This is all possible now with solutions similar to what was described at the top of the story, but cheaper (check out Symform version 2 due out 6/26/2010, also using Amazon's cloud). The most expensive piece of the puzzle for a small business is the backup/deduplication software and/or hardware. Some proposed solutions I've seen are just way too expensive for small businesses. Hard disk space and even the cloud services are not expensive, the backup/deduplication software is another story. My goal is to do this for an unlimited amount of data (bandwidth is the limitation here) and at a cost under $2000 initially for software and hardware (this may be difficult???) and less than $100 a month ongoing for cloud "services" (not necessarily cloud storage, again see www.symform.com). "Cloud" does not necessarily mean "Outsourcing," but it certainly can protect data from disasters. My goal is to be able to recover from any disaster in three steps. On site, almost immediately; off site or with replacement hardware within 1-2 days; and major disaster, to be able to reconstruct a small business's systems totally within 3 days in another physical location. In the worst instance, you may be out of business for awhile, but you'll have the data you need to satisfy the insurance, finance, and tax people as well as being able to re-establish contact with your vendors and customers. I think this is a pretty comprehensive disaster recovery that has not been available to small businesses until the advent "OF THE CLOUD."

Tony Hopkinson
Tony Hopkinson

As a for instance I refused to use a service to store personal data about me, because it required me to waive my rights under the european data protection act. Now if I was a business and that was my customer's data I would have been waiving their rights, which would have put me in court...

david.hunt
david.hunt

A company is required to disclose if it will hold an individual's private data on servers off-shore, and if so whether the country / provider is bound to provide similar privacy protections to the Australian legislation. Mind you I've not seen many multi-national organisations actually disclose this fact :-(

sysop-dr
sysop-dr

Has the breach of cloud data not already happened? Remember google telling us that Chinese hackers got into it's data and it's users data? Isn't gmail and google docs the poster child for Cloud data? And there was the data loss of some phone companies user data and how many other companies been hit that we have not heard about? Do these not already show that putting your data in the could is inherantly insecure? If the companies i have money in or the companies I work for were to put their data in the cloud i would automatically sue just for exposing their data to the risk, even if they were not breached. Don't do it, it's just plain stupid. We tell Facebook users don't put anything on Facebook you wouldn't want your mother or your boss to see. Google, Microsoft and the others can not be many times lower risk then Facebook. They all have the same idea, put the data youwant to share with people here and we will let them see it. OOPS, we let the wrong guys see it, or we were hacked, it's not our fault. And they are right, it's not their fault, it's your fault for trusting the inherantly insecure web with your data. This is just facebook for companies, do you really want to put your companies trade secrets on facebook? Buy a hard drive put it on a server, put a firewall on your network with a web access vpn, use Linux on all of your computers and be safe out there. LLAP

disasterboy.info
disasterboy.info

I hear and agree. It has value in the context data duplication for availability, backup and/or disaster recovery. As you say it comes down to risk, costs and benefits For some organisations there may well be good case for cloud technologies. If you * have a lot of mobile workers, * are a geographically distributed organisation and/or * need dynamic offsite storage for rapid disaster recovery and are happy with the data integrity risks then it is probably an excellent solution. As many say. onsite storage has its perilous risks too. I could JUST get it but it seems to add a number of extra potential points of failure.

Gabriel G
Gabriel G

I think the name CLOUD has everyone in awe.. This is off site storage, it is not anything new. No matter how integrated, the files reside somewhere else. I agree with your point. Well said.

amahan
amahan

Well said. I am much more comfortable running my own servers than dealing with a vendor.

marks
marks

There is no performance contract with the "Cloud" or any of its components that are required to protect and provide access to the data. Even if there were, many of the critical aspects of access to the data are beyond the control of the provider. Only a fool would put anything of value in such an environment.

andrewgauger
andrewgauger

With these questions, I'd have to respond with more questions: Why would you chose a provider to store your information that has shaky business stance? Isn't it more likely that you will go out of business than Amazon? Why don't you have redundant internet links if your storage is cloud based? Isn't that worth budgeting? How trustworthy is your own staff? Sometimes it is better to have contractual agreements with a 3rd party than to insight distrust with your own staff. How secure is your own network? You already admit to having an internet connection. Don't you think the same vectors that the attackers take on your 3rd party they will try on you as well? People who think that security is diminished using a cloud provider have their information backwards. It is more secure to use a cloud solution than to trust your own network with the data. They have the budgets to implement security processes and protocols that an individual network may not have. Just be aware that you are on a shared service, and that you need to encrypt your data to protect it from other members.

rob_nasuni
rob_nasuni

These are great questions and valid concerns we hear pretty often at Nasuni. We've thought a lot about these problems and have good answers to each of them. I'd recommend you download the free trial of the Nasuni Filer and spend some time with the product and talk to our support and sales people to understand the quality of the product and the team that is behind it. Meanwhile some things to look for in addressing your questions: 1) Does the vendor have separate accounts at the cloud provider for each of its customers? 2) Does the vendor state in its EULA that you have the right to take your account with you if you want to leave the vendor at some point in the future? 3) Does the vendor state in its EULA that their data is self-described or is described such that you can interpret the format in the cloud once you leave the vendor? 4) Does the vendor have caching to offset cloud outages so you data is accessible even when the clouds are momentarily unavailable? 5) Does the vendor let you vary the size of the cache to fit your working set to adjust for performance and outage concerns? 6) Does the vendor have security experts on staff? 7) Does the vendor speak often and clearly about cloud security, its concerns and how it mitigates them? Many of these we've spoken about on the Nasuni blog: http://www.nasuni.com/news/nasuni-blog And in our current and upcoming whitepapers. We encourage you to follow along and find out that primary storage in the cloud is a reality today for our customers. Regards, Rob

Kevin@Quealy.net
Kevin@Quealy.net

I use IDrive to backup about 280 GB each night off site. Having the actually data off site (instead of a backup) is different. What I do like is that I have another TB of large datafiles I'm unable to backup with IDrive because of the sheer size of the files. This solution will allow me to only allocate a singe 500 GB locally for all my data. The best pricing option is $200 a month for two years. That's still probably too high for a company our size ... but if the service is as fast (or nearly as fast) as having the files local then I could be interested.

b4real
b4real

How useful is that? About as useful as some random packet on the Internet if you ask me!

b4real
b4real

I seriously think that everyone that dings cloud, for storage, assumes that your data is at rest unencrypted.

b4real
b4real

This is likely more secure than unencrypted at rest 'on your own gear'. No?

Silvosky
Silvosky

We are in an innovative and dynamic age and for one to be afraid of change can't be helpful. Instead, we neede to look critically on the usefulness as against the odds. You would agree that most coporate compliance now factor a life time data achaiving which involves enomous cost and insufficient resources to handle. With many companies daily demand for streaming video as part of intellectual property, it becomes necessary to look for a cheaper, yet more secure way of hanling the data under serious scruitiny of modern day encryption. In my opinion,it isn't necessary for all companies to seek cloud storage until the need arises. Gradually seperate your most accessed data from the general and keep the others in the cloud. If you have heard talks about the new designs of multimedia centered networks, then it becomes clear that for a global access to vital information for staffs located around the globe in real time, it becomes necessary to depend on a more reliable storage location with no down-time than your companies network with intermittent drops.

Tony Hopkinson
Tony Hopkinson

You haven't got a business, get people you can trust, move to the sidelines and put someone in charge who can engender some trust. Cloud is a cop out in that scenario. Tell me are you a salesman or an IT person? You do understand the enormous financial potential of the cloud for providers don't you? Any promise they make now in terms of getting us on there so they can realise it, is either bollocks now, or will be as soon as they get lock in. Natural acretion into a small number of players will mean they will have all their customers by the nuts. We would be utterly and completely dependant on a third party to stay in business. No different to what happened in the finance industry, to what is happening in power. You end up paying more and more for less and less. Try again.

gechurch
gechurch

All of your points address company-wide issues. Those issues are all going to exist whether or not cloud storage is added into the mix. Adding cloud storage cannot possibly reduce any of these existing issues, but it sure can add new ones. To reply to each of your points. You wouldn't deliberately choose a dodgy provider. What's to say the management of Amazon won't decide that books are their cashcow, and they are going to sell off their other departments. Who knows who will buy the cloud storage side. What's to say they won't do something stupid that gives them terrible PR that eventually takes them down. These things aren't likely, but if you're relying on them for your critical data, you need to consider them. What's to say redundant links are enough? What if a guy with a backhoe down the road slices all the phone cables. Down go both your links. As others have said... this is another expense. You'd better take it into account. Your own staff might not be trustworthy. If you are concerned about that you can do background checks when you hire people, you can implement auditing, you can install video cameras etc etc. What can you do about the storage providers staff? Your own network flaws will of course remain if you move to cloud storage. But now the attack surface is greater - someone can try to get into your network, or to the cloud providers to get your data. The best case scenario is the storage provider has perfect security, and you will be just as secure as before. If they aren't perfect, your data is now less secure. I agree that storage providers are set up for this, and are likely to have better security than the average company, and better backup and continuance plans etc. But moving to the cloud while keeping a local cache only adds to the risks, and there's a lot less you can do about issues if you're not in control.

elrico-fantastica
elrico-fantastica

i wouldnt pick a shaky provider intentionally but stuff happens we have no control of... is it likely amazon will go bust? of course not but you still have to facter the additional point of failure. redudant links may not be an option for a smaller company.. and if im paying for extra bandwidth, redudant links and failover systems wouldnt it just be cheaper to buy myself some onsite storage... the reduction in cost is supposed to be one of the clouds selling points... i would like to think my own staff is trustworthy... at least i can vet my own staff and maintain a degree of control.. yes i would hope there staff is trustworthy but its another leap of faith im being expected to make with my companies security. well i work for a security company that specialises in network security, firewalling and penetration testing so i seriously doubt they are more secure than us.. this arguement could well be valid for companies in other areas of industry. like i said it could have some potential.. i certainly wouldnt risk early adoption when i can see so many flaws/questions.. but then thats from my standpoint, maybe people in different positions have less concerns about their data.

The 'G-Man.'
The 'G-Man.'

What happens when the vendor of the connection software / cache (not the cloud storage solution) goes bust? What happens when the vendor of the connection software / cache gets bought by a bigger player? My current solution has all the files on site on their servers and off site 'cloud based' backups for files, exchange databases, SQL databases, VM Images and others items. These files are compressed / de-duped / encrypyed as well. More expensive yes but it handles all the data needs not just files.

ddannenbrink
ddannenbrink

What are the differences between this and a products like Riverbed and Citrix WAN Scaler?

b4real
b4real

AES 256-bit encryption is enough or that no one addresses security?

daboochmeister
daboochmeister

Other than the locally cached data (which is on the same system as the key), the data and the key to decrypt are on two different systems. That actually provides a measure of increased security, no?

Tony Hopkinson
Tony Hopkinson

of your specific data being that valuable to a bad guy they are prepared and capable of attempting to crack the encryption, but not able to to do something far easier i.e. get the key makes your argument for a stronger encryption irrelevant. If it was important you'd already have it anyway....

b4real
b4real

Referring to this encrypted cloud data.

Tony Hopkinson
Tony Hopkinson

And how does 256 bit encryption change that? You are talking about buying a better lock and then still leaving the key under the same mat you kept the 128 bit one. So your entire point in terms of being more secure in the cloud is at best specious.

b4real
b4real

If you lose it, provider cannot help you.

Tony Hopkinson
Tony Hopkinson

It's as secure as the key, and where are they? Exploiting access to get a key is far more effective than code cracking.

Tony Hopkinson
Tony Hopkinson

in terms of is doesn't give a provider a lot of scope for offers or a potential client especially a decent sized one a lot of scope to use the cloud. I can see uses for the cloud, I can see advantages. I don't like the way it's being sold as some sort of IT panacea and I especially don't like the continual side stepping of real issues with a little jink and it will save you money. I feel that providers gain far more from cloud than the bulk of their potential customers. The obvious imbalance makes me very very suspicious of it's proponents.....

b4real
b4real

"Off site storage of encrypted data with encryption only being done locally has a very limited utility." If you are referring as it being limited for "compute" (like an Amazon AMI), I agree. But for bulk object stores, like a file server -> virtualized to the cloud -> I don't see how it is limited? SPOILER: I'm going to debunk cloud storage, from one perspective, next week!

Tony Hopkinson
Tony Hopkinson

Off site storage of encrypted data with encryption only being done locally has a very limited utility. Data protection act says the same thing near enough, sort of misses the point. You need a data centre per legislative area. Only applies to the really big boys, but they are the ones most concerned with the issues the cloud introduces.

b4real
b4real

You can with your cloud storage account, at least in the case of S3 and SDN, specify it is in one of the countries they provide. (USA, JP, Germany, Ireland, etc.)

Walthy
Walthy

I doubt it. If it does, how come so much medical transcriptionis done overseas? If HIPAA says it stays here, it lies. What makes you think the data will stay in this country? With outsourcing of data centers, who knows where the cloud resides? The only protection any user has is to make sure it is encrypted, maybe doubly, before it leaves the premises.

b4real
b4real

We've beat that to death. The only note is if the compliance requirement mandates that the data reside in a country. For example, HIPAA data cannot leave the USA as I understand it. So, with the cloud provider you would specify which locales the dataset would exist, fully capable. Dorian and I still have our $100 giveaway if you can prove cloud solutions NOT compliant in any established framework: http://bit.ly/fWl2b

Tony Hopkinson
Tony Hopkinson

Can I interest you in a bridge? Nice location, one careful owner.

b4real
b4real

Riverbend and others are expecting a wide spectrum of traffic. Local cache of Nasuni and others works on the principle that 10, 15 or some percentage of your file server is ACTUALLY accessed. If you administer a file server, you should know that most of your data is not accessed.

rob_nasuni
rob_nasuni

There are definitely products out there that work to make interfaces like CIFS and NFS better over long distances. The protocols weren't designed for long distances, poor response times and perhaps intermittent internet connections. That plus some of the target cloud vendors, like Amazon, don't have a CIFS or NFS interface natively. You'd be better off using these NAS interfaces locally for their intended use and then using more efficient protocols and methods to communicate with the cloud storage providers. Caching, compression, and other techniques help take the load off the network and get your data to the cloud efficiently and securely.

Editor's Picks