Microsoft

Remove clutter with Windows XP SP2's Duplicate Finder tool

Hidden clutter exists on your Windows XP machine in the form of duplicate files. Here's how to free up valuable hard disk space by doing some early spring cleaning with the Duplicate Finder tool.

Even if you're a conscientious computer user (i.e., you regularly delete unnecessary files, empty the Recycle Bin, and run Disk Defragmenter), you may be unaware of a potentially big waster of hard disk space: duplicate files. Applications can litter your hard disk with duplicate files, or you can actually create duplicate files by copying files from one folder to another.

Windows XP's default installation doesn't provide you with a decent utility for tracking down duplicate files. However, Microsoft does have a tool called Duplicate Finder, which is part of the Windows XP Service Pack 2 Support Tools. Here's how to install and use the Duplicate Finder tool:

  1. Download the Windows XP Service Pack 2 Support Tools and follow the instructions for installing the Complete installation version.
  2. Open the Run dialog box by pressing [Windows]R.
  3. Type Dupfinder in the Open text box and click OK.
  4. Once DupFinder loads, simply select the drive or folder to search and then click the Start Search button.
  5. When DupFinder completes its search, you can scan through the list and examine the duplicate files.

Here are tips for working with the list of duplicate files:

  • Use either the Print Report or Export Data commands on the File menu to create a permanent record of the duplicate files.
  • Use the Sort command on the View menu to reorganize the list for better analysis.
  • To get more detailed information about any file, select the file, pull down the File menu, and select the Info command.
  • Leave duplicate files in the Windows folder and its subfolders alone.
  • If you don't recognize the duplicate file, it's better to use the Rename or Move commands on the File menu rather than the Delete command.

Note: This tip applies to both Windows XP Home and Windows XP Professional.

Miss a tip?

Check out the Windows XP archive, and catch up on our most recent Windows XP tips.

Stay on top of the latest XP tips and tricks with our free Windows XP newsletter, delivered each Thursday. Automatically sign up today!

About

Greg Shultz is a freelance Technical Writer. Previously, he has worked as Documentation Specialist in the software industry, a Technical Support Specialist in educational industry, and a Technical Journalist in the computer publishing industry.

70 comments
markbun
markbun

I use "DuplicateFilesDeleter" Its best for finding and deleting duplicate files.

n.ravi
n.ravi

excellent information, thanks

rlevander
rlevander

How about removing the files that are marked as the lauguage of TURKISH, or FRENCH, ETC. that i'll never access here is the U.S.?

kelly.welch3
kelly.welch3

I had no problems downloading or installing. Everything works fine! Seems to be a useful tool. My question, where can I get info on all the other tools that come in the Support Tools pack?

stevecripe57
stevecripe57

Followed the instructions and downloaded and installed the Support Tools program from Microsoft. Typed "Dupfinder in Run dialog box and it said file not found. Went into Support Tools file and looked to see what files were there. Lots of files, but NO Dupfinder file was found in the System Tools folder!

jomasenator
jomasenator

Using winxp pro command "dupfinder" my system gave over 32,000 duplicates. My question is: without clicking the files one by one, how can I get rid of the duplicates?.

biglittle
biglittle

Actually more of a question than an comment. I have the dilema of having two hard drives. Somehow I have managed to get most of my Windows on my "D" drive and in a lot of cases I have a Windows (2). Is it safe to delete all of those Windows files? They were initially under Windows XP Home. I now have WIndows XP Pro. Oh BTW thanks for the reminder of the Support Tools.

damorag
damorag

I ran dupfinder and it fould almost 60,000 duplicate files. I am only seeing the option to select and delete one file at a time by right-clicking. The shift or control keys do not let you pick. Thats a lot of files to delete one at a time. Any suggestions?

D.H. Cesare
D.H. Cesare

Using WinXP Pro, after downloading and installing the "support tools", going to RUN and typing in Dupfinder, I get an error message saying "Can't find Dupfinder". What's next?

hellolegalplus
hellolegalplus

Maybe I haven't had enough coffee this morning yet, but there doesn't seem to be a way to delete duplicate files in groups. Am I supposed to choose and delete individual files one at a time? That would take about a week on this system.

philthee
philthee

Although the article says this tool is available in both XP Pro and XP Home, there's no sign of it on my XP Home, SP-2 machines.

BALTHOR
BALTHOR

A duplicate file would be a file that exactly matches the existing original file.What causes duplicate files?I also have caches that fill up with non essential files.If I were a bank or corporation would I need to rid my computer of these files?Is a computer something that we are just stuck with.A device that we,as owners,have to make work because we can not get any response from manufacturers?Or are we reluctant to contact the manufacturer.Do I have to be an engineer to talk to someone about my computer problem?Can I say:"It sounds like that thingy in there isn't right".

Morris1368
Morris1368

Use Duplicate Finder to Remove Duplicate Files.

ultimitloozer
ultimitloozer

If using a CRC or MD5 hash alone, false positives will result more frequently. But if it is also comparing file size simultaneously, the chance of false positives becomes extremely remote.

cerec
cerec

Download the free EasyCleaner. It lets you select multiple files using standard CRTL SHIFT modes.

kmsmax
kmsmax

You could export the list (perhaps sorted) to a text file (output.txt), then enter the following into a cmd file for /f "tokens=1,2 delims= " %%i in (output.txt) do del "%%i%%j" (note that is a TAB after the delims=) The command will read lines from the output.txt file, parsing into tokens delimited by tabs, then delete the files. So, one could order the sort, set the file filter options, exported the result, edited the file to contain the files to delete. An admin might be more creative for a permanent tool by incorporating other script commands into the cmd. Although I could not find info, possibly one could execute the dupfinder as a command line.

isemmel
isemmel

These tools are not meant for mass distribution to the general user, hence the description of them : "The Windows Support Tools for Microsoft Windows XP are intended for use by Microsoft support personnel and experienced users to assist in diagnosing and resolving computer problems". I would not delete any system files, or any .dlls. Rather, you should use the program only to analyse your data files and files that you know about.

cerec
cerec

I got all the support programs except dupfinder. XP Pro too.

Greg Shultz
Greg Shultz

Deleting duplicate files, especially operating system and application files, can be a dangerous proposition, which is why Microsoft made this tool without a mass deletion control. The goal behind requiring you to individually delete files, is to make you think about each file you're deleting so that you don't delete something that will make your operating system unstable. As I mentioned in the article: * Leave duplicate files in the Windows folder and its subfolders alone. (The reason is that some applications and operating system components my use different copies of a duplicate file and be unaware of the other copies. For example, App A may look for DupeFile in location X while App B looks for DupeFile in location Z. If you delete DupeFile from location Z, it's possible that App B will malfunction.) * If you don't recognize the duplicate file, it's better to use the Rename or Move commands on the File menu rather than the Delete command. The reason here is that if rename DupeFile in location Z, and discover that App B malfunctions, then you can simply rename it back. And, renaming the file will be easier if you use either the Print Report or Export Data commands on the File menu to create a permanent record of the duplicate files.

xtra
xtra

I tried it on my work desktop which has multiple user profiles and I'm also getting a boatload (22,000+) and most are dlls', bin's and the like and are in system, setting, windows, and program folders. I'm only getting a few in duplicates in data folders, dowloads, and the like. So with a handful of exceptions I have neither the time nor the inclination figure out which and where I can safely be delete dll's... So even though it was an interesting exercise, in may not be particularly useful.

wendyleriche
wendyleriche

It is afternoon now and plenty of coffee but like markgrogers i have not got the time to sift through this lot, so were is the secret short-cut button.. if no such radio button excist's well there's a suggestion for the makers,

rwwone
rwwone

In defense of Mr. Shultz, his article did say to perform a complete installation.

deICERAY
deICERAY

"Use either the Print Report or Export Data commands on the File menu to create a permanent record of the duplicate files." PRINT REPORT? What universe does that command exist in? EXPORT DATA? Maybe in Excel... but not on my SP2 system! The only way you can accomplish these tasks is to use 3rd party softare; I've seen some good freeware that will print directories and some that find duplicate files, but nothing within my Home Win XP SP2 system.

innes
innes

Did you do a Typical Install? As I di and noticed the file was missing so I reran the install and added the optional extras. Now it appears :)

johncymru
johncymru

Download it from the link (1.) in the article and install it. When installing it usually puts a folder named Windows Support Tools in Start\All Programs which has a command shell link, a help file detailing the tools and a readme file.

vjtay
vjtay

According to the Read Me File you have to have Windows Professional only before you can use these tool. Also, When you open the run file you have to use either a forward slash or a back slash (can't remember which). Am I right or wrong? Jon

gardnerh
gardnerh

There must be 10,000 files that are listed as duplicates! I really don't wish to spend the rest of my life sorting through them but definitely something needs to be done since the occupy a total of a half gigabyte on my drive. Most notably are duplicate .jpg files which are in the "My Pictures" folder and in the "Documents and Settings" folder A suggestion or two would really be helpful. As suggested I will not touch the Windows Directory.

summerby
summerby

file not found after i installed it.

rwwone
rwwone

I think the main problem here is that the Windows SP2 Support Tools has an important heading BEFORE you download it: "The Windows Support Tools for Microsoft Windows XP are intended for use by Microsoft support personnel and experienced users to assist in diagnosing and resolving computer problems." Enough said.

computerd}}
computerd}}

How about we just get Microsoft and all the other OS programmers to just take a little more time to write programs that don't make duplicate files when a service pack or a major OS install is created.

Cussy
Cussy

Many components, including Dupfinder, are only installed when you select "Complete" for the installation options. Make sure you change to this from the "Typical" default button.

jocaan
jocaan

OK, so Windows, system files and .dlls would stay. I have a lot of dupes in Documents and Settings-hopefully it would be safe enough to delete, move or rename these. You are right-I am not an expert and maybe should not mess around. There were other possibly safer, more conservative dupe removers mentioned above that could be more for the average home user. Frank C.

nickbush
nickbush

chose "Complete" not "Typical" as it states in the article. I missed that the first time myself... HTH

jocaan
jocaan

With over 100 pages to Print I cancelled printing on the 18th page. The printed page shifted the margin to the right so that the page number could not be read. When you do select Print from the File menu you do not get options to print a range or individual pages, which does not allow printing on both sides (unless you take the printed copy(ies) and put them back into the feeder as the printing continues-which is what I did for a while). Yes your instructions were clear about not deleting and renaming or moving, still, it would take a month and a day to do the job on the non-Windows files. Being able to group-select the files would cut down the time to, maybe just a day. Frank C.

Antagonist
Antagonist

PRINT REPORT and Export Data are both options under the file menu of DupFinder. DupFinder is part of the support tools, an app that you download from a link in the first step of the article. Try reading the article.

ThumbsUp2
ThumbsUp2

It clearly says to run the TOOL and use the FILE drop down menu to "Print Report" or "Export Data". You can't do this from Windows alone. It's in the TOOL.

jensham
jensham

Having done the scan and obtained a long long list is it a matter of reviewing each file and removing as necessary?

Greg Shultz
Greg Shultz

...and what types of files they are, you can use Search or Windows Explorer to track them down and kill them.

bluesrenegade
bluesrenegade

After downloading and installing the latest Windows XP Support Tools, I immediately pulled up the Help for the XP Support Tools and looked through its alphabetic listing of (all?) the support tools; no dupfinder was in the Help. Even a search of Help turned up nothing. Now I'm left wondering what other support tools were installed that are not documented? Comments, anyone, please? Thanks!

asyousay
asyousay

You don't have to intall the lot. Many work independently. I don't have a definitive list but I always use my same 'ol faves. Just open the cab take out the ones you want and dice the rest. I use windiff a lot but find it's very clunky - hasn't changed for yonks.(think it was in W95 or at least W98?) A drag and drop would be nice. Hmmm.

peyton.lewis
peyton.lewis

Since the gist of the article was the use of DupFinder, you would think that it would be mentioned in the article that you need to perform the complete install to get it.

malachiwillcox
malachiwillcox

Thanks Cussy... Good call, saved me a headache of working it out. I wouldn't mind your opinion on how Dupfinder compares to other DUPFILE FINDERS (freeware). Do you think it would be better to go with an alternative for data security purposes etc?... I appreciate your advice and ANYONE else who might have some. Thanks...

ultimitloozer
ultimitloozer

Unless a user has been engineering these duplicates on his/her own system (or possibly downloading from what should be considered untrusted sources), it doesn't really apply in real-world situations. Or maybe there are other more serious security implications on the system. Have you seen any non-engineered files that have identical sizes and MD5 hashes?

deICERAY
deICERAY

What are you talking about? Where did you get the idea that you have anything to contribute to a discussion to which you have no understanding? I can't believe how cavalier posters can be in berating others. This entire article depends on a detail not in the article, and please read this very slow-ly, so you'll get it and go post on some other question, ok? The ONLY way you get this utility is IF you download the tools for sp2 AND you do NOT do the typical setup. Period. Whether it works or doesn't or anything else is moot. The writer didn't include a critical detail, and confused most readers of merely human ability, including me. "ranting" and "snide" are words you might want to avoid using until you can apply them properly. Not one post here is a rant and none are snide; stupid, maybe, irrelevant, more likely, silly, for sure. But really - move on.

Elfman42
Elfman42

Go back to the article and re-read it. Twice. Dupfinder is part of a package of tools you need to download and install separately from Windows...it is NOT part of the default Windows installation. Before you go off questioning things and berating people here, be sure you know of what you speak. You clearly did not understand the article or the instructions therein; you have little justification for your ranting and snide replies.

s31064
s31064

Is it a congenital defect or did someone drop you on your head from a third floor window? How long did it take before you figured out you had to click "Start" to shutdown?

deICERAY
deICERAY

duh, nope I don git it. naught won beet. eyes r stuupit two much fer ewe. eyes dun postin naw, y'all have a gud day, heah?

sleepin'dawg
sleepin'dawg

not in SP2 itself. They assume you can not only walk and chew gum at the same time but can actually comprhend english. I do sincerely hope this is not too condescending for your limited linguistic abilities. [b]Dawg[/b] ]:)

deICERAY
deICERAY

I love sarcasm; since there is no dupfinder in my install of sp2, I assumed that the results would be displayed in the standard windows explorer, in which there are indeed no such categories as "info", etc. So try using your brain for more than a baseball cap holder and attempt to be helpful instead of condescending - or is that too big a word for you?

circlestrafe999
circlestrafe999

It's true that the article says to do a complete install, and since it's all of a whopping 5megs I dunno why anyone wouldn't just complete install anyway. However, I think the guy you replied to meant if the instruction to do a complete install was upfront, i.e. the first sentence of the article, people wouldn't have had such horrible dupfinder experiences. Reading comprehension isn't all that anymore I guess.

imccabe
imccabe

It is very clearly stated in Step #1 that you need to perform a complete install- 'Download the Windows XP Service Pack 2 Support Tools and follow the instructions for installing the Complete installation version.'

dariced
dariced

I've had it for awhile and had no problems with it. I could also use it to clean the registry with no problems. but then I discovered CCleaner and I prefer the latter. I do check over the deplicate files that Easy Cleaner wants to delete before it does so, just in case I see something important. I'm kind of new at this. Dawn

jomasenator
jomasenator

After having used Easy Cleaner (it made it easy to remove over 30,000 files); I have a petition for the creator of the program Tony Arts: Go and work for Bill Gates. We need you in there!!.

breadtrk
breadtrk

Been using it for years, never has trashed a system. Well worth a small donation although not required. When you realize that it is about the best cleaning tool in your toolbox you will do the right thing.

htan
htan

I downloaded the program, clear Temporary files, as directed and still was not able to install it. There is an error 6002 message, saying that I have to close all programs and clean the temprary folder. Still the error persisted. htan

cerec
cerec

EasyCleaner is much better than dupfinder and has bettter and many other useful options.

wendyleriche
wendyleriche

hi, i went to download easycleaner but it is not free any more:(

Matureman
Matureman

You might try EasyCleaner 2.0 by ToniArts. It is safe and trustworthy...also a great registry cleaner included. Have used it every week for a couple of years. Good luck.

Editor's Picks