Apps

Five great apps for removing duplicate files

DoubleKiller

DoubleKiller offers powerful comparison features, including hash comparison. It also lets you compare a small number of "fresh" files against a collection of "known" files -- a big help in a team or collaborative environment.

About

Justin James is the Lead Architect for Conigent.

17 comments
TobyTobin
TobyTobin

I'm surprised that DiskState was not one of the duplicate file finders. It uses MD5 signatures to compare exact duplicates either on a single folder on one hard drive or other folders across multiple drives. It has many other features that make it a valuable tool that I have been using for many years. Because it uses a MD5 signature, it cannot detect pictures that are essentially the same, but have minor differences.

approbat
approbat

Thanks to TechRepublic for the the nices "Tips & Tricks". I suggest put apps in a comparison table with ranks, including Free/Paid. Sincerely, Mario Approbato, MD, PhD.

Harry44Callahan
Harry44Callahan

Although I have no personal experience with this type of program I must agree. The reason I have no experience is because I have read to many bad things about them, like stated by bhusebye, and refused to try.

bhusebye
bhusebye

Do not be duped into running these programs without going through and visually verifying you need each duplicate. Many programs point to a file that is called the same thing, but performs a different function per each program. I disagree with passing these out without telling people the havoc that could ensue from running them blindly, especially if they are not experienced in recovery.

Otto Roth
Otto Roth

All these apps are great at finding dupes if you are just running a single store of files. It would be great if, as if in a spreadsheet, the files can be listed down the left while the drives (partitions), external drives, portable drives. san/nas's be listed along the top showing where all the "duplicates" are (Qty, Quality, Age, etc) as I would like to have duplicates of certain files in certain places. The way most of the ones above work is that it will give you the duplicated file, say 6 times, one for each dulicate it found an you have to manually distinguish which are the actual file vs the downloads, dsktop, Mywork, backup, etc copies it finds.

jmandres
jmandres

You should have a look at anti-twin. Good and reliable.

becksdark
becksdark

Try Audio Comparer. I've saved countless hours whittling down my music collection. This program actually 'listens' to the songs, creates an 'aural' database and matches them up on the basis of similarity. It shows you a percentage match figure based on sonic qualities. It will match songs of differing bit-rates and codec...all it cares about is how the song sounds. So it will show you ALL the matching songs regardless of MP3, AIF, WAV, M4a... They also make a photo comparing program, even though I haven't had a chance to use it yet. If it works anything like Audio Compare, it'll be another winner.

GSCOTTJ
GSCOTTJ

Thank you for posting the link to the complete article all on one page, rather than forcing everyone to step through an entire gallery of images one at a time. I don't mind the gallery format for somethings, but most times I much prefer getting all the information at one time in a one-page article (even if it is a long page). I hope this trend continues.

Joe_Wulf
Joe_Wulf

Consider this superior filemanager (at: www.ztree.com). Has its history in the stellar XtreeGold from the past. One of its many functions is finding duplicate files, based on the criteria you select, across ALL disk drives.

wingnut1024
wingnut1024

Duplicate images finder seems to have a number of complaints about not searching subdirectories and not finding files and sometimes looking like it skipped checking as it shows completion too quickly. Only 2 very old posts, 2008 and 2009 under discussions. Doesn't look like this should be on the list IMHO.

fmosh
fmosh

My laptop has been plagued with duplicate files from over a year. Just what I need.

sh10453
sh10453

Only a novice would delete blindly. It's obvious that the user should examine what is being presented as duplicates. On the other hand, not all programs are created equal. A good program has various options for search. That includes the option to compare the content byte by byte, or bit by bit (and other properties). Good programs do not compare by file name, duh! Besides, these programs should not be used to find duplicates in critical folders, such as System (Windows) or "Program Files" folders. They should be used to find duplicate files in folders such as documents, pictures, music , downloads, etc., with no automatic deletion. The human must ultimately select what to be deleted.

sh10453
sh10453

Agree ... Good program, and free too. Very fast! There is a portable version as well (just the executable).

jpnagle59
jpnagle59

Good old XtreeGold...it's still around?

sh10453
sh10453

I have used it with good results (for small size folders). To search sub-folders, the user must check the box for that option before the search starts. The GUI is over-simplified, and can use some improvement, but it's OK. It's free!!! It's very slow, and a processor hog. The "Cancel" button doesn't work well if the folder size is relatively large & the program is scanning, and attempting to cancel I'd get the hourglass (program hangs), then I'd have to terminate the program with the "X", or from Task Manager. Bottom line, IMHO, is that it is not recommended. By the way, Version 3 is in its very early stages. It's not going to be out anytime soon.

jegreen56
jegreen56

If I say this Scooter Software app is the second most useful utility besides Windows Explorer or one of its many clones, I am not exaggerating one bit. Especially for someone who programs devices and needs to track changes on a daily basis. It's also great for doing incremental backups with the ability to synchronize in a custom semi-automated fashion. I know that there are some great configuration management apps that will allow you to back out to a more stable version when things start crashing. But if you're operating on a shoestring budget and you don't mind creative and inventive solutions that can be a useful substitute, then Beyond Compare is a useful and often versatile all-around performer.