A program to remove duplicate files. How to Easily Find Duplicate Files in Windows and Remove Them

Surely, any of us have duplicate files accumulated on our disk over time. Files in “Downloads” that you have downloaded several times, identical photographs and musical compositions lying in such depths that your hands simply cannot reach them. You can get rid of all this manually, but special utilities that search for identical files will work much faster for you.

A very popular “cleaner” that probably everyone has installed. Yes, it not only searches for system garbage and clears browser history and cookies, but also removes duplicate files.

Platforms: Windows, Mac.

Price: free, $24.95 for premium version.

The program searches for files with the same or similar names and identical content. Works well with music and can find identical music files even if they have different tags. In addition, dupeGuru can compare images to find not only identical, but simply similar photos.

Developed for Mac and Linux. The Windows version is no longer supported by the developer, but it can be downloaded from the official website - it is fully functional.

Platforms: Windows, Mac, Linux.

An advanced file search application that, among other things, can remove duplicates. SearchMyFiles has flexible filters so you can customize your search results the way you want.

Platforms: Windows.

A popular Mac application that searches for the same or similar files and shows you the difference between them. Copies in “Photos”, in the iTunes music library - nothing will pass by Gemini 2. The developers have announced a smart duplicate search mechanism that remembers which files you leave and what you decide to delete.

Platforms: Mac.

Although AllDup is free, it does a lot. Searches for, deletes, copies and moves duplicate files, including identical audio recordings with different tags. There is a flexible search setting. Using the built-in viewer, you can examine files and choose what to delete.

Platforms: Windows.

Duplicate File Finder quickly and efficiently finds duplicate files. Provides an interesting opportunity to search for duplicates not only on the hard drive, but also on the local network. Can work with images and music, comparing both tags and content. The preview function will help you figure out what to really delete and what to leave. Unfortunately, quite a few options are not available in the free version.

Platforms: Windows.

Price: free, $29.95 for premium version.

A universal file manager that can do anything with your files. Including finding duplicate files. You can enable the search for copies option on the search parameters tab, in the same place where you specify other attributes of the searched files.

Platforms: Windows.

DupeGuru looks like the most attractive option. It's free, but it offers all the necessary features to rid your drive of accumulated junk. The only sad thing is that development of the Windows version has been stopped. For Windows users who don't want to pay for commercial alternatives, AllDup is a better choice. CCleaner and Total Commander are more universal and widespread solutions that are probably already installed on everyone.

A very convenient feature in the CCleaner application is the search for duplicate files. Very often, there are files on your computer that are identical in date, size and name. Of course, some of them are needed, and some may have been created accidentally or downloaded several times from the Internet. All these files eventually accumulate, the free space becomes less and less, and, as a result, the computer begins to slow down. Therefore, from time to time, you need to get rid of such files. If you are an advanced PC user, then it will not be difficult for you to find CCleaner duplicate files which to delete, but if you are new to this matter, then we will help you figure it out.

What files should not be deleted

Before we start searching for duplicates and deleting them, let's look at whether it is possible to delete duplicate files using Cyclener? To begin with, I would like to note that the program will not allow you to delete absolutely all copies of a file. One of them must remain untouched. Further, we do not recommend deleting system files. It is quite normal for them to have duplicates. Typically, system files are located on drive C in the Windows folder.

Files that can be deleted

Typically, a computer consists of several partitions (disks). The amount of information that is stored on each of them is most likely impressive. There are pictures, music, videos, photographs, and much more. Some of the duplicate files could have been duplicated by the user by mistake, for example, due to forgetfulness, the file was saved in different sections. Some files may have been downloaded from the Internet several times, etc. And when the program finds such files, you can safely delete them from your computer.

Find duplicates

In the " Service"there is a section" Search for duplicates».

In this section, at the user's discretion, you can set search criteria. You can search for duplicates by any one search parameter: by size, by date, by name and by content, or by several parameters at the same time, marking them with checkboxes.

You can also define the files that need to be skipped. There are several options here:

Zero size files;
Files that are read-only;
Hidden files;
System files;
Files whose size does not exceed the megabyte size you specify;
Files larger than the specified megabyte size.

In the " Inclusions» you can specify the places where the search will be carried out. To scan a specific folder, you must select " Add" The following window will appear

Click on the button " Review" and select the desired folder, indicating the path to it.

In the " Exceptions» you can specify those folders that should not be affected during the search.

You can add them similarly to the “ Inclusions" Click " Review" and select this folder.

After setting all search parameters, click on the button “ Find».

Duplicate search results

After searching, the results will be shown in table form.

It will indicate the file names, locations where they are located, their sizes and creation dates.

To remove duplicates, check the boxes next to them. If you right-click on any file, a context menu will appear.

Select all	Possibility to mark all found duplicates. All files will have only one copy - the bottom one.
Remove selection	The ability to uncheck all found duplicates if they are selected.
Select type/Remove type	Ability to check (uncheck) all files of the same type.
Exclude/Limit/Select Duplicates	The ability to perform the selected action in relation to one of the folders in which the file is located.
Save report...	Ability to save the report in a text document.
Open folder	Allows you to open the folder in which this file is located.

After you have selected all the duplicates that you want to delete, click on the “ Delete».

A very popular “cleaner” that probably everyone has installed. Yes, it not only searches for system garbage and clears browser history and cookies, but also removes duplicate files.

Platforms: Windows, Mac.

Price: free, $24.95 for premium version.

Developed for Mac and Linux. The Windows version is no longer supported by the developer, but it can be downloaded from the official website - it is fully functional.

Platforms: Windows, Mac, Linux.

An advanced file search application that, among other things, can remove duplicates. SearchMyFiles has flexible filters so you can customize your search results the way you want.

Platforms: Windows.

Platforms: Mac.

Platforms: Windows.

Price: free, $29.95 for premium version.

Platforms: Windows.

Sometimes in everyday computer activities the task arises of finding duplicate files. There can be many reasons for this: lack of space on your hard drive, attempts to reduce entropy in your files, dealing with photos from your camera that were dropped at different times, and many other necessary cases.

You can find a large number of programs on the Internet that allow you to search for duplicate files. But why look for any programs if a smart tool for such work is usually always at hand. And this tool is called Total Commander(TC).

In this article I will show all the methods based on Total Commander versions 8.5 , in this version, the search for duplicate files has become very rich in functionality.

!!!A small important digression. What do you mean by duplicate file? Two files are IDENTICAL only if they are exactly the same bit by bit. Those. Any information in a computer is represented by a sequence of zeros and ones. So, files match only when they completely match the sequence of zeros and ones that make up these files. All talk about how you can compare two files on any other basis is deeply erroneous.

TC has two, essentially different, methods for finding duplicate files:

Synchronize directories;
Search for duplicates;

Their features and application are best illustrated with examples.

1.Directory synchronization.

This method is used when the two folders you are comparing have an identical structure. This usually happens in many cases, here are a few of them:

Have you regularly archived your work folder? After some time, you need to find out which files have been added or changed since the archive was created. You unpack the entire archive into a separate folder. The folder structure in it practically coincides with the working one. You compare two folders “original” and “restored from archive” and easily get a list of all changed, added or deleted files. A couple of simple manipulations - and you remove from the recovered folder all the duplicate files that are in the working one.
You work in a folder on a network drive and regularly make a copy to your local drive. Over time, your working folder has become quite large and the time spent on a complete copy has become very large. In order not to copy the entire folder each time, you can first compare it with the backup one and copy only those files that have been changed or added, and also delete files in the backup folder that were deleted from the main one.

Once you get the hang of it and feel the full power of this method, you yourself will be able to come up with thousands of situations where the directory synchronization method will be of great help to you in your work.

So, how does everything happen in practice? Let's get started.

Let's assume we have a main folder "Working", which contains the files with which you are working. And there is a folder "Archive", which contains an old copy of the folder "Working". Our task is to find duplicate files in both folders and remove them from the folder "Archive".

Open TC. In the right and left panels, open the folders being compared:

Press menu “Commands” - “Synchronize directories...”

The directory comparison window opens

Next we need to set the comparison parameters. Put a tick in the parameters “with subdirectories”, “by content”, “ignore date”

"with subdirectories"— files in all subdirectories of the specified folders will be compared;
"by content"- this is the key option that forces TC to compare files BIT by BIT!!! Otherwise, files will be compared by name, size, date;
"ignore date"— this option forces TC to show differing files, without attempting to automatically determine the direction of future copying;

!!! Only files with the same names will be compared!!! If the files are identical, but they have a different name, then they will not be compared!

Press the button "Compare". Depending on the size of the files, the comparison can take a very long time, do not be alarmed. Eventually the comparison will end and the result will be displayed in the bottom status line (section 1 in the figure):

If the buttons in the “Show” section (section 2 in the figure) are pressed, then you will see the comparison result for each file.

— this button enables the display of files that are in the left panel, but not in the right;

— this button enables the display of identical files;

— this button enables the display of differing files;

— this button enables the display of files that are in the right panel, but not in the left;

If you initially have all display buttons turned off, then the result of the comparison can only be assessed by the status bar (section 1 in the figure above), in this case we see that 11 files were compared, of which 8 files are the same, 2 files are different, and There is also a file in the left panel that is not in the right panel.

To complete our task, it is necessary to leave the display of only identical (identical) files, so we turn off all other display buttons

Now we only have identical files left, and we can safely delete them in the folder "Archive". To do this, select all files. The easiest way to do this is by pressing the universal combination CTRL+A. Or first select the first line with the mouse, then press the key on the keyboard SHIFT and without releasing it, select the last line with the mouse. As a result, you should get something like this:

The final step is to right-click on any line and select the item in the menu that opens "Delete on the left"

TC kindly asks us about our desire,

and if we press "YES" then it deletes all marked files in the folder "Archive".

After this, the two folders are automatically compared again. If you do not need a repeated comparison, the process can be interrupted by clicking on the button "Abort" or press a key ESC on keyboard. If the repeated comparison was not interrupted, and we turned on all the display buttons, then we will see a window like this

All. The task has been completed. All identical files were found and deleted in the folder "Archive".

Educational video on the topic

2.Search for duplicates.

The fundamental difference between this method and the directory synchronization method is that TC ignores the names of the files being compared. In fact, it compares each file with each, and shows us identical files no matter what they are called ! This search is very convenient when you do not know either the folder structure or the names of the files being compared. In any case, after searching for duplicates, you will receive an exact list of identical files.

I will demonstrate finding duplicates using one practical task, finding duplicates of personal photos. Quite often you download photos from your digital gadgets to your computer. Often the situation gets confused, something is reset many times, something is skipped. How to quickly delete files that have been dropped multiple times? Very simple!

Let's get started.

Let's say you always dump all your photos into a folder "PHOTO" on drive D. After all the resets, the folder looks something like this:

As you can see, some files are located in folders named by the date of shooting, some are dropped to the root of the folder "_New" And "_New1"

To start searching for duplicates, open the folder in which we will search in any TC panel. In our case this is the folder "PHOTO"

Next, press the key combination on the keyboard ALT+F7 or select from the menu “Commands” - “Search for files”

The standard TC search window opens. String "Search files:" leave it empty, then all files will be compared.

Then go to the bookmark "Additionally" and check the boxes “Search for duplicates:”, “by size”, “by content” and press "To start searching".

The search can take a VERY long time, do not be afraid of this, since there are a huge number of comparisons of a large volume of files. At the same time, the progress percentage is shown in the status bar

When the search ends, a search results window will open, in which we press the button "Files to panel"

In the search window and in the panel window, identical files are collected in sections separated by dotted lines

Each section displays the file name and full path to the file. The names of IDENTICAL files can be completely different!
In this case, it is clear that the same photograph was recorded THREE times, twice under the same name( IMG_4187.JPG) and the third time this photograph was recorded under a completely different name ( IMG_4187_13.JPG).

Next, it remains to select unnecessary identical files and delete them. This can be done manually by selecting each file by pressing a key Ins. But it takes a long time and is not effective. There are better and faster ways.

So our task is to remove duplicate files in folders "_New" And "_New1".
To do this, press on the additional keyboard, the large key on the right [+] . Typically, using this key in TC, files are selected by mask. The same operation can be done through the menu “Selection” - “Select group”

File garbage accumulated during computer operation usually consists of more than just temporary or unnecessary files. A separate category are duplicate files, which are identical in content, but are stored in different folders or on different computer drives.

This could be, for example, documents that were saved repeatedly, duplicate or similar photographs, the same music compositions that you downloaded from the Internet and saved in different folders or under different names. It is important to understand that duplicates are essentially two versions of the same photo that have different resolutions. Or the same song by the same artist, but recorded at different concerts. Or two versions of the same musical composition, but recorded in different audio formats.

An example of duplicate files can also be two versions of a movie, saved with different quality or, again, in different video formats. Of course, it is quite difficult to manually detect such duplicates, so we will call on special programs that search for duplicate files to help. As you probably already understood from the previous paragraph, duplicates of each type of file (drawings, music, videos, documents) have their own characteristics.

Algorithms for searching for duplicates of all types of files differ significantly from each other, therefore, as a rule, different programs are engaged in searching for duplicates of files of different types. Let's look at some of them. In addition, it is important not only to detect duplicates, but also to select the latest version of the document or the highest quality video or audio file. We will look at each type of file separately, and also provide a list of popular programs for working with duplicates.

Search for duplicate images

The most striking example of duplicate pictures can be identical or very similar photographs. Many modern cameras allow you to shoot in burst mode, when pictures are taken at very short intervals - from a few fractions of a second to several seconds. However, the resulting images differ from each other very slightly. As practice shows, there is no real need to store similar photographs; it is enough to select one or two successful photographs from a series and delete the rest.

Another case where duplicate image files may appear is when creating backup copies of photos on your hard drive, or selecting images for a presentation, in which they are copied to another folder. In general, the user often creates copies of images himself and then forgets to delete them. However, the reasons for the appearance of duplicate images can be very different. Our task is to get rid of such duplicates and clear up space on your hard drive. Among the most common programs designed to search for duplicate images on a computer are the following.

SWMole Clone Remover. Website address: http://www.clone-remover.com/. The program is paid. It can search not only for duplicate images (including similar pictures, and not just exact copies), but also for copies of music files. In the unregistered version, the user has the ability to delete no more than 20 files. Finds copies of images in pairs, even if there are more than two similar images on the disk. Has a Russian-language interface.
Image Comparer. Website: http://www.bolidesoft.com/rus/imagecomparer.html. The program is shareware and has a trial period of 30 days. A multilingual user interface is supported, including Russian. It is possible to search for duplicate pictures both within one folder and between several folders. The program interface is somewhat complicated for a novice user, but it is possible to use a step-by-step wizard mode that will help you find and remove duplicate image files.
Picasa. Official program page: https://picasa.google.com. The program itself includes many different tools for working with photographs. The program also has a function to search for duplicate images. To use this function, you need to select Tools>>Experimental functions>>Show duplicate files in the main menu of the program. Let us immediately pay attention to the fact that the program searches for duplicate image files, that is, absolutely identical pictures. But it finds them regardless of location and with virtually no user intervention. The program is absolutely free and has a Russian-language interface. In addition, using Picasa, you can process images, apply various filters to them, find people's faces, mark shooting locations, and perform many other useful actions.
Similar Images Finder. Program website: http://www.crown-s-soft.com/ru/sifinder.htm. The program is shareware. You can use it for 30 days without purchasing, but there is a limit on the number of duplicate files found. The program interface is in English. It is possible to adjust the depth of “similarity” of photos, as well as set preferences for found duplicates: for example, you can tell the program to automatically delete the older duplicate file, which has less color or smaller size. The program has a duplicate search wizard that allows you to specify criteria for comparing image files in a step-by-step mode.

Find duplicate music files

Just like images, music files can sometimes take up a lot of disk space, and in some cases it can be difficult to manually find the same or similar music tracks. Specialized applications that automate this process will come to the rescue. Let's talk about some of the most popular ones.

Audio comparer. You can download the program on the website: http://audiocomparer.com/rus/. The application is shareware. It can be used for 30 days without registration. The analysis of musical compositions is based on a comparison of the names of performers and song titles. This data is taken from the file name and internal service information, the so-called ID3 tags. It is possible to specify the threshold of similarity of names at which the compositions will be considered the same. The application allows you to compare files both within one directory on your computer disk and between several directories. The program interface is somewhat complicated for a novice user, but this is compensated by the presence of a step-by-step wizard that facilitates the process of finding duplicates.
Music Duplicate Remover. Internet site: http://www.maniactools.com/. The key difference between this application and its analogues is the ability to search for duplicates by sound. This function allows you to find duplicates even among files that are incorrectly named. The program is shareware and allows an unregistered user to work for 30 days. It has a simple user interface in Russian.
SWMole Clone Remover. As already mentioned, this application can search not only for duplicate images, but also for repeated musical compositions. Program website address: http://www.clone-remover.com/. The program interface is a bit complicated, but thanks to Russian language support, it’s easy to figure it out on your own. The application can search for duplicate music files only by file name and ID3 tags, without taking into account the sound. By using one of the programs listed or all of them at once, you can free up a lot of useful space on your computer's disks.

Finding duplicate video files

Dealing with duplicate video files has its own characteristics compared to finding duplicate pictures or music. Due to the large size of video files, there are usually not as many of them on computer drives as music or images. In addition, the appearance of duplicate films or videos is not as common as the “cloning” of photographs or songs. But deleting even one clone of a video file will help free up a significant amount of disk space. In addition, the algorithm for comparing video streams is much more complex than drawings or even the sound of music. For this reason, there are not many programs designed to find duplicate video files. Let's look at some of them.

Duplicate Video Search. Web address: http://duplicatevideosearch.com/rus/. The program is shareware - with a trial period of 30 days. It has an English-language interface, which, however, is not overloaded with unnecessary elements, and therefore is quite understandable. The user must specify in which folders to search for video duplicates. The application can compare the contents of video files of ten different formats. It is possible to specify the comparison depth.
Teemoon Video Matching. Program home page: http://teemoon.name/videoid/Default.html. Unlike its “brothers”, this application is free. Working with the program involves two stages. The first is indexing, which is done on the Indexing tab. This process creates index files for each video file in the specified folder. The process is lengthy, but it only needs to be done once, then the existing index files will be used when comparing files. The second stage is comparison, which is carried out on the Matching tab. The disadvantages of the program include the lack of support for the Russian language and a non-obvious operating algorithm.
Video Comparer. Program website: http://www.video-comparer.com/. The application has paid and free versions. The free one is quite suitable for home use. The program interface is quite simple. The user only needs to specify the folders in which to search for duplicate files and click the button to start the comparison process. The user interface is in English.

If you are a film fan and store a large collection of different films on your computer, or are involved in video editing and have a lot of video clips on your hard drive, then you should take an “inventory” of your video archive. Perhaps you will find hidden reserves of your computer.

Finding other duplicates

As you probably already understood, there is no universal program that allows you to get rid of all duplicate files in one fell swoop - each type of data requires its own tool. However, if you leave aside the complex algorithms for comparing images and sounds, and limit yourself to simply comparing the name, size and date of the file, you can find many duplicates among those file types that we have not yet mentioned here. These can be documents, files of other formats that waste disk space. To find such duplicates, you can often do without specialized programs. For example, if you use a popular file manager, you can use its tools to find and delete duplicate files. Among the file managers that have the function of searching for duplicate files are:

FAR Manager;
FreeCommander;
Total Commander.

If you are not used to using file managers, then you should not install them on your computer just to find duplicates. There are specialized applications for this task:

CloneSpy - http://www.clonespy.com/;
DupKiller - http://dupkiller.com/index_ru.html;
Duplicate Cleaner - http://www.digitalvolcano.co.uk/duplicatecleaner. html;
Free Duplicate File Finder - http://www.ashisoft.com/;
NoClone 2014 Desktop - http://noclone.net/

These and many other paid and free applications for every taste will help you get rid of duplicate files on your hard drives.