Linux File System
how may files can be stored in a linux dir?
is there a limitation in linux file sytem?
Linux
how may files can be stored in a linux dir?
is there a limitation in linux file sytem?
by spidererrol
By linux file system I would assume you mean the most common: ext2
AFAIK there is no actual limit, but if you start putting lots of files in one directory you will find your machine slowing to a crawl when accessing files in that dir. Reiserfs ("www.reiserfs.org", in 2.4.x kernels) goes a long way to reducing this slowdown.
The only limitation people generally come across is the 2GB file size limit. Although I believe this can be got around too (I never had to, so I'm not sure how too. Also this limit can help reduce the impact of a silly program filling up the harddrive).
Hope this helps you.
by fta7kch
by pdb1
This article might make it clearer ( Credit should go to the the www.suse.de site)
---
Large FIle System 9LFS) in the Linux Kernel
Since Linux 2.4.0-test7 most of the kernel interface is included into the kernel. The open problems and restrictions are described below.
File Systems
We can separate two levels of LFS compliance in the file systems:
Full support for files > 2 GiB and O_LARGEFILE
Limited LFS support: it gives proper EINVAL/EFBIG/EOVERFLOW error messages when you try to use O_LARGEFILE or positions > 2 GiB.
At least the second level should be generally reachable, but is some work to audit all the weird file systems.
Some bugs in NFSv2 regarding (2) have been fixed already, but some are missing (like the O_LARGEFILE check). Other file systems probably miss it too. A complete audit of all file systems is needed (see also the 2.4 kernel TODO page at http://linux24.sourceforge.net/).
The situation about the different filesystems used in Linux 2.4.0 can besummarized as follows:
ext2
Full support for LFS
NFSv2
Cannot handle LFS due to protocol restrictions (limited to 2 GiB - 1); limited LFS support but expect some bugs
NFSv3
The protocol is ok, but I'm not sure about the Linux implementation status
ReiserFS 3.5.x (not part of the kernel, separate patch)
Does not support LFS
ReiserFS 3.6.x (part of kernel 2.4.1 and newer)
Full support for LFS if the new on disk format is used. This format is incompatible to the format used by 3.5.x (see below for some more details).
coda
Does not work with LFS (local cache issues, protocol is ok)
UFS
Full support for LFS (although not complete vs. O_LARGEFILE flag use)
minix
limited to 2 GiB - 1 (file size is limited to 65804MiB but note that filesystem size is limited to 64 MiB - but holes are allowed)
SysV (aka SCO)
limited to 2 GiB -1
by fta7kch
Actually there are a few limits to keep in mind.
1) if you're using an older machine, the /boot directory needs to be before the first 1023 cylinders, otherwise the bios will not be able to read it, and thus you will not be able to boot to linux via the harddisk, BUT if you have a newer HDD with LBA mode, you can boot directory to linux regardless of the location. It's best to check for it in bios to ensure LBA is available and being used.
2) There is no 2 gig limit to linux (with the exception of above statement). (with my system, i have some storage partitions that span 100 gigs!)
3) if you're going to dual boot linux & some other operating system, be careful of the placement, most other OS (Win 9x, ME, Xp, and 2000) ususually REQUIRE a primary parition to work correctly.
here are some articles about the linux file system that may help you get started
http://www.freeos.com/articles/3102/
http://www.linuxgazette.com/issue55/florido.html
by fta7kch
Thx for your answer.
So if i want to store many many files to a linux directory,
and the total of files if growing up,
e.g, 10K , 100K number of files
will the file system corrupt or cause a disaster?
by paulo.sedrez
On ext[23] filesystems, there are not a limit imposed for number of files on a directory; althought ext[23] is not very efficient when dealing with very large directories. The directory can grow as much as it needs, up to the maximum size allowed toa file (usually, 2GB).
The number of files on a filesystem, however, is limited by the number of inodes established when the file system was created (or grown, but thats another issue). Each file or directory uses one inode; you can check how many inodes are left on a filesystem with "df -i".
If you are planning one filesystem with a very large number of small files, I sugest you create it with 1 inode per block (more them that is useless).
--Sedrez
by fta7kch
by fta7kch
This discussion is closed to new replies.
Sort by Trending