Software

Define disk quotas to keep users from hogging drive space


With massive hard drives, one might be tempted to think disk quotas a thing of the past. However, with larger capacity comes larger amounts of data, and in multiuser systems, one user can easily consume more drive space than they should, preventing other users from utilizing storage space.

Using disk quotas is easy and is especially useful to keep the digital pack-rats from claiming more space than they should. As well, you can set soft and hard quotas -- soft quotas to remind individuals that they are nearing their capacity and hard quotas to enforce that maximum capacity.

Quotas are defined per-filesystem. Most distros support quotas, although not all do it out-of-the-box, and you may have to install the quota package. To enable quota support, edit /etc/fstab as root and add the usrquota and grpquota options to the filesystems you wish to enable quota support for. For instance:

/dev/md5 /home xfs defaults,nosuid,nodev,usrquota,grpquota 1 2

Once you have made the changes, remount the filesystem(s) you have changed:

# mount -o remount /home

To check that quota support is indeed enabled, execute:

# quotacheck -augmv

This will instruct quotacheck to check all filesystems for user and group quotas without remounting them as read-only. The first time quotacheck is run, it will display a number of errors because it has never checked the systems before. Now you can enable quotas with the quotaon command:

# quotaon -augv

Once quotas have been turned on, use edquota to edit the quotas for a particular user:

# edquota -u joe

This will open the default system editor (usually vim) where you can edit the hard and soft limits for both blocks and inodes for each filesystem that supports quotas.

You can then view current quota usage by using the repquota tool:

# repquota -a

Once a soft quota has been exceeded, the user is notified once that they have exceeded their quota, but will be able to continue writing to the system unless they reach the hard quota; at which point, any new files created will be 0 bytes in size. You can determine whether or not to allow this default behavior by changing the grace time; the default is seven days. For the time of the grace period, if the soft quota is exceeded, the user can continue to write files, unless they hit the hard limit. After the grace period, they will no longer be able to write files.

With a combination of soft and hard limits and a decent grace time for users, administrators can ensure that individual pack rats will not negatively affect other users' fair share of storage space.

Delivered each Tuesday, TechRepublic's free Linux NetNote provides tips, articles, and other resources to help you hone your Linux skills. Automatically sign up today!

About

Vincent Danen works on the Red Hat Security Response Team and lives in Canada. He has been writing about and developing on Linux for over 10 years and is a veteran Mac user.

16 comments
feral
feral

Put yourselves in my position, I work for a majore airline who has outsourced its IT to a major Telco here in Australia. Last month I receive an email informing me that all my users and groups are going to be subject to a 200 megabyte, yes you read correctly a 200 megabyte quota limit. To add insult to injury I was also informed that we would be forced to pay an annual fee of $100.00 per 100 megabytes over this limit for all users and groups. Some quick maths based on the old spurious a meg is a meg measurement, a 148GB disk for my server cost a $1000.00. My users currently only use 20G of disk space however if we do not clean up and archive we are up for about $11,000.00 PA in excess quota charges. We clean up I can bring that down to about half. The outsourcer states that the new policy is fair and equitable! Wish me luck as I try to manage our data and keep that annual cost down. These quotas apply to notes mailboxes and they have generously given us 150Mb for mail storage, it will be interesting when the management team some of whom have 1G+ of mail stored for regulatory reasons stop receiving mail. In this day and age 200Mb of storage is a joke and someone needs a reality check however, it is not a case of throwing more hardware at the issue either as someone has stated in a previous post. It is a combination of common sense, good data management policies and above all else the tools to do the job, which surprise surprise we have not been given apart from a copy of winzip.

delajt
delajt

Seriously, disk quota management really should be a thing of the past. Cost of hardware is lower that cost of man hours managing it all. A staffer, lets call him "Junior Disk Quota Manager", costs $2000 a month. How much capacity does that buy you these days? Another staffer, lets call him "Junior Disk Quota Manager - contractor", costs you $1000 a day. You figure out the costs associated.

DanLM
DanLM

You can always upgrade to provide a higher quota, but by setting the quota... You stop users from possibly using all current space from other users. Quota's on log partitions. I can't think of a better example of where quota's should be set. You have some ahole pounding your machine which is generating logs out the wazoo. This may cause a windows machine to have serious performance issues if it fills the drive. In a Unix/Linux environment. I know it would. Yes, you can have automatic rotation of logs and other automated process's to stop this. But a runaway log can screw you every time. You have a moron for a user that is storing all his mp3s. Nothing will stop the twit like a quota which was set for expected business use. Why I use quota's. I have a server which I rent out shells. You think I'm going to trust someone that is sending me 30 dollars a month to not put to much on their shell space. Yea right, not happening. Quota, thats what they pay for. Thats all they get. I work in a graphic company(phone books). The hard drive space you think is so cheap is reserved for the graphics they are constantly being created for ad's.. Drive space is required for that, not some user who saves every email. Every word document. Every peace of crap. Dan

w2ktechman
w2ktechman

the server HDD's cost more than the SATA drives you pick up at Fry's on sale. The server HDD's are often using RAID, so each GB is actually multiples of GB's There needs to be staff on hand to do server maintenance there needs to be backups made on the servers, in case of problems, and those backups can go back for some period of time. Not only that, often a data service is used for data protection by transferring the backups to different locations across the country

vdanen
vdanen

Is this symptomatic of an american need to give everyone a title because I've *never* heard of this role, and I can't possibly conceive of anyone paying someone $2k/mo or, even stranger, $1k/day for a contractor to manage a disk quota. Where do you come up with these figures? You implement the quota once and leave it. There's nothing to manage. Users have to mange it themself -- if their quota is exceeded, it's not up to some fulltime "disk quota manager" to, what, increase their quota? Kinda defeats the purpose of the whole thing. The user has to clean up some of their junk... they manage their own quota. The man hours to "manage" this could be measured in *minutes*. With no recurring costs associated to "manage" anything. Sure, drives are cheap. Get a big drive, give everyone a 50GB quota and leave it alone. You'll never have to touch it again. I'd like to see a large harddrive without quotas stop user Idiot A from running a script that writes nothing but 0's to a file until the disk is full. Your big harddrive would not stop this and it would bring a lot more expense in man hours to fix it than it would to have the same file fill Idiot A's quota and not impact anyone else on the system. Disk quotas are relevant, even if we move to 100TB drives as standard. Especially on multi-user production systems. And your cost examples are completely bogus too.

S,David
S,David

Let's see. With a disk quota I save the cost of the disk drive itself and any associated hardware needed to install it, the cost of the hardware dude (non-SAN disk) to install it, the cost to the company of bringing down a 24/7 system for the install, and, as I don't use a "Junior Disk Quota Manager", I save that cost as well. There is also the a savings in wear and tear on my rump by not having to explain to management why I can't control hardware purchases. An almost free software installation to keep a packrat with no concept of system resources under control is a great thing. If he wants to fill up his development system, fine, but I am going to restrict how much he can put on the production systems so they don't get used for overflow.

feral
feral

All of these costs you refer to are service costs. Backup, transfer and storage offsite should be costed in to an agreement at commencement as should the quotas. The agreement would allow for IT growth and certain increases in service delivery cost over the life of the agreement. In addition to this companies have to take into account the cost of data growth, which is heavily dependant on the business and its processes. You refer to the cost of hard disks being more expensive. 148 gigabytes of disk will set you back about $1,0000 dollars which translates to roughly $6.00 a gigabyte. When an IT service provider wants to charge $100.00 per 100 megabytes for exceeding a 200Mb quota and the math used to calculate space is 1MB = 1MB then that business needs to think about it's quota strategy very carefully. Bear in mind just how much maintenance does one person have to physically do to that disk on a daily basis. Cheers

Adam Eisenberg
Adam Eisenberg

While I agree with your statements, I think that most of problems with quota management correspond to problems with desktop and enterprise management. The complexity of management of huge and even relatively small but frequently changing network environments remains quite challenging for a typical administrator. Sure, the File Server Resource Manager snap-in [url=http://technet2.microsoft.com/windowsserver2008/en/library/d6d480ca-18ec-4dee-aafc-a9e7971038cf1033.mspx?mfr=true]goes far away[/url] from what we've [url="http://www.windowsitpro.com/Articles/ArticleID/3679/3679.html"]seen in Windows 2000 Server[/url]. Alas, it's still complex for administrator to keep pace with the changes that you need to handle when managing such complex environments as we have now. Although at the same very moments we see major improvements and extensive development in network and desktop management software and [url=http://redmondmag.com/features/article.asp?editorialsid=2370]advances achieved[/url] in automating desktop routine and maintenance. The last thing makes us hope that we don't need to expand stuff accordingly to manage with the dramatic growth of management complexity having tools as such under the thumb.

delajt
delajt

... I am not an American ;-)

delajt
delajt

As pointed out by the posters, the title is of course bogus, but the situation sure is not. At my previous job there was a contractor whose job was to ensure that the disk quotas where maintained and people abusing the quotas were chased down (they didn't to have a hard quota, as some people needed temporary storage for a small period of time). In effect this was his day job. And he was a contractor on an 80 euro/hour rate. So no, the situation is real. And this was no mom&dad shop, it was within a company on the Forbes Top 100 list of 2007.

feral
feral

Yes if they think they can get away with it they will try. If you are doing your job and auditing regularly and dealing fairly with recalcitrants they will behave. The other thing is if the users are not educated in what the imapact is to the company, they will also try and get away with it because they have no understanding of the flow on effects. Even today I found half a gig of mp3's on a share and instead of blasting the owener with policy I sent him a simple email requesting the material be removed by COB tomorrow. If it is not I will escalate things in the appropriate manner. This is a fair thing Im giving the user the opportunity to clean up, we have some change comms meetings going on this week and I will reiterate the need to remove personal data. My users are pretty good and fortunately are happy with the way I do things.

DanLM
DanLM

Users are much smarter then everyone gives them credit for with things that they can get away with. Thats why I think quota's will never go away. Dan

feral
feral

Yes you are correct users do need to learn how to manage data, however most companies do not include that as part of the training. This has fallen back to me to provide the training which Iam happy to do. As for users and personal files on the server, the clean up we initiated in the last week has resulted in nearly 5 gig of recovered disk space. Once I kick off the duplicate clean up this week I expect to recover another 3 gig at least. Week after that is archive week and Iam going after any data that has not been accessed for more than 12 months which should return a minimum of 5 gig of space. Based on the costs in my last post Im looking at saving close to $7,000.00 pa however I still have to look at growth rates. I have only been here 10 months and have not had a chance to do the capacity planning as I have been cleaning up the mess that was left by my predecessor. Im glad to see that you do agree that 200 MB of disk space is out of touch with reality for a workgroup in the aviation industry. My users do not have unlimited disk space and wont ever have that luxury, the issue I will be fighting is users will use up their 200 personal and start dumping in the shared volumes. Hopefully my storage policy and change management will prevent this happening. We shall see. Cheers

w2ktechman
w2ktechman

that you are listing, it seems a bit out of touch with reality. But, costs of services do add up, along with staff to do it/them. If you take away quotas, then people will abuse the unlimited space. Then what, buy more servers? Pop in a few more drives?? Out of curiosity, have you seen the kind of data that people store on server drives??? I have! People store pictures from trips/family/etc.. People store downloads of things that they dont want others to know about. Including movies/mp3's, etc.. People just drop stuff there because it is 'unlimited' to them, often never to even look at it again. I saw someone who every few months cloned his HDD, and stored the image on a server. never got rid of any because he wanted to be able to go back to any of them. How long should these sit there??? Users DO NEED to learn how to manage their data! just because it is not convenient for them to go through and delete things once in a while is no argument for keeping it all.

DanLM
DanLM

That emails the sysadmin of people that are abusing their quota's. It could send an email to the user also.. Chuckle, I'd write the script for a lot less then this guy is making for managing quota's. Dan

vdanen
vdanen

That still seems out to lunch to me. I guess being in Forbes Top 100 means they can throw money away. Me, I'd be setting a soft quota, with a hard quota over top of that (perhaps generous) in order to prevent that kind of thing. People needing temporary storage is fine, but like you had said about large drives, toss a few large drives in there and set the soft quota to twice what they could normally be expected to use, with a reasonable hard quota over that. Then if people hit (would be a generous hard quota), I'd respond with the standard reply: sucks to be you. Clean up your garbage. Of course, if someone wanted to pay *me* 80euros/hr to do quota management, I would respectfully say they were genius and laugh my way to the bank. =)

Editor's Picks