Windows Forum

Ask something
Discussion Locked

Accessing a folder with 200,000+ files

0 helpful
12 replies

by tiffany2

Closed to new repliesThis discussion is closed. It stays readable, but it takes no new replies.
In a lapse of good judgement a program was written to query a single folder for JPG files that number about 270,000 files that are 100K each. I have been asked by my boss to see if we can do anything to optimize the server to allow access to this folder via Windows Explorer. Currently it can't be opened from a desktop and it pegs the processor and takes 25 minutes to open on the server.

12 replies

by curious_george

Set all user windows explorer settings to default to list and to therefore not show thumbnails. At least for this particular folder.

by maxwell edison

This might help:

On NTFS volumes, whenever a directory is accesses, the file system keeps track of the last access date/time. Although it's a small overhead, it can make a difference when accessing a large number of directories (for example, using the dir /s command on a volume with thousands of directories). To prevent this timestamping on NTFS volumes, make add the following to the Registry:

Hive: HKEY_LOCAL_MACHINE
Key: System\CurrentControlSet\Control\FileSystem
Name: NtfsDisableLastAccessUpdate
Data Type: REG_DWORD
Value: 1

A value of 1 disables the access time stamping, a value of 0 enables it. Remember to use caution and frequent backups when editing the Registry.

----------

Another option:

IrfanView ishandy for digital camera users who manage large folders of images. For information and a free downlaod, see this link.

http://www.cnet.com/software/0-3923247-8-5788833-3.html

(REMOVE SPACES from the pasted URL.)

Maxwell

by dklippert

List view sounds like a possible work around, but in the end you should reorganize the folder. Perhaps you could write a simple DOS bat file to xcopy files using some wildcard combination that would work with your naming convention.
Good luck!

by hazzard_005

I would first share the drive. This is assuming that your server is windows based. If it isn't Im sure that someone would be able to supply you the commands to do this.

Name this batch file jpegcopy.bat
Just have this line in the batch filexcopy [source] [destination][%1] /d:%1

[source] is the directory address which is in question
[destination] is the directory in which you are going to dump these files and directories. eg: d:\stuff Best to create this directory before starting.

PS don't forget to take the []'s out of the equasion.

%1 is a variable which you type after the command and can only be a Date in this case. Use this format for the date 4-8-02.

Then at a dos prompt type: jpegcopy 3-8-02
It will create a new directory with the date as a name and have only the files marked with that date in it.

Now all you have to do is change the date and you get a new directory with the files of the same date in it.

Then all you have to do is when it finishes copying the files is hit the up arrow or F3 i think and it will bring up the command again. Then you just have to backspace and edit the date.

This may be a lot of work but when you are done it will be very organized and not spend lots of time trying to find a way to organize them. And I thought this was a cool way to do it. :)

by hazzard_005

You may need to test this to see if it works properly. I could not test it on a scale in which you need it to work. But the command workes well in windows 2000. Just remember not to set the destination file as your root directory. You may run into another type of problem.

Hope this was helpfull.

by gigelul

Maybe a database with links to these files can be easily accessed. Or a minimal Web server with a search page and indexes for these files.

This discussion is closed to new replies.