Filters - Using regular expressions - user guide/examples?

Find true duplicate files, byte for byte, anywhere, regardless of file name and date/time. Clean up your drive of duplicate files.
jgm7683
Posts: 4
Joined: Sun Jan 08, 2012 5:42 pm

Filters - Using regular expressions - user guide/examples?

Post by jgm7683 » Sun Jan 08, 2012 5:47 pm

I can't seem to be able to use the regular expressions very easily, and was wondering if there was a guide detailing some examples. I've been unable to find anything useful, and because the forums were reset in October, there's nothing in here. (Do you have those archived somewhere so we might search them, even if we can't post there any longer?)

Thank you - I've got thousands of files to go through, and it's hard to do this easily.

-- Jeff

User avatar
Support
Site Admin
Posts: 1727
Joined: Sun Oct 02, 2011 10:49 am

Re: Filters - Using regular expressions - user guide/examples?

Post by Support » Sun Jan 08, 2012 7:07 pm

The regular expression used in CloneMaster is the same as Perl. Take a look at this http://www.troubleshooters.com/codecorn ... omparisons, under "Using simple wilcards".

What you will use the most are the * ? and * characters. For example...

To scan only for jpeg, gif and png files, you would have an include filter of...

Code: Select all

\.(jpg|gif|png)$
The above filter can also be used in the Exclude filter to skip those kind of files from the scan.

You can include folders too...
/windows/;/my\s+documents/;/temp/

Just remember to remove any spaces before and after the ;
/temp/ ; /backup/ will not work but /temp/;/backup/ will work.

If you a have specific thing that you can't make to work, let me know and I'll provide you with all the details.
Your support team.
http://SoftByteLabs.com

jgm7683
Posts: 4
Joined: Sun Jan 08, 2012 5:42 pm

Re: Filters - Using regular expressions - user guide/examples?

Post by jgm7683 » Sun Jan 08, 2012 7:18 pm

Is there any filter already set up that will verify that the filenames are the same, not just the data?
I had a HD crash, and had to back the files up to several smaller hard drives to restore... and I had many items backed up, but not all

So I'm stuck with, for example,
00000013.MPG
It's all about the Pentiums - Weird Al.MPG

Now in that case, I want to make the decision myself which to delete... but if both were named 00000013.MPG, it wouldn't matter.

I don't know Perl well enough (though I've used it before about 18 years ago in college, and rarely with my own Linux box) to know for certain that this would be possible or give the right results...
But even if you don't have a means for this, I appreciate the guide and knowledge that the expressions are like Perl - That will give me a means to start going through this a bit more quickly.

-- Jeff

User avatar
Support
Site Admin
Posts: 1727
Joined: Sun Oct 02, 2011 10:49 am

Re: Filters - Using regular expressions - user guide/examples?

Post by Support » Sun Jan 08, 2012 7:40 pm

No, CloneMaster does not assume duplicates based on names only. Our Comparator maybe what you need, but, your backups folder structures needs to be the same as the originals. What it does is, compare a source and destination folder for differences. It will then list which files are duplicates (same file name, same size, same date, same data), files that exist in source that are not in destination (and vice-versa), newer files (duplicates but diffrent dates). This may help you rebuild the source to its original state from the backups.
Your support team.
http://SoftByteLabs.com

jgm7683
Posts: 4
Joined: Sun Jan 08, 2012 5:42 pm

Re: Filters - Using regular expressions - user guide/examples?

Post by jgm7683 » Mon Jan 09, 2012 1:51 am

I realize this... but because of the problem with the file names, I don't want to autodelete without verifying them.

If the name is the same AND it's a duplicate, it can be removed

If the names differ, I need to verify that I'm saving the correct LFN, and not the short 8+3 name of the file.

I was hoping the expressions would allow me to do this more easily, but from your response it seems not. I'll have to go through the 6000+ files manually, which could take weeks instead of what I was hoping to be a day to a week tops.

Thanks anyway;

-- Jeff

jgm7683
Posts: 4
Joined: Sun Jan 08, 2012 5:42 pm

Re: Filters - Using regular expressions - user guide/examples?

Post by jgm7683 » Mon Jan 09, 2012 1:54 am

FYI: It was hardware RAID failure... I was able to find the directories, but the order was all mixed up, and for some reason the LFNs were lost (due to MFT corruption, I believe.)
So I technically have two copies of each file, 3-5 if I include the backups that weren't technically backups... more like "Ok; here's enough to fill a CD - now I can reorganize these files."

Word of advice to anyone using RAID 1: Use software, not hardware...

User avatar
Support
Site Admin
Posts: 1727
Joined: Sun Oct 02, 2011 10:49 am

Re: Filters - Using regular expressions - user guide/examples?

Post by Support » Mon Jan 09, 2012 11:25 am

What you can do is sort the list of duplicates by clicking on the Folder column header so that your good drive is listed first, and the backup drive listed last. Then click on the Select All icon, and all the files in the backup that already exist on the main drive will be selected. You can review the selection before deleting them.
Your support team.
http://SoftByteLabs.com

CLS
Posts: 9
Joined: Tue Oct 23, 2012 9:11 am

Re: Filters - Using regular expressions - user guide/examples?

Post by CLS » Tue Oct 23, 2012 11:43 am

I'd like to be able to tell CloneMaster to ignore files under a certain size. For example - only compare files over 10 megs in size.

I know I can compare everything then sort by size, but I have millions of smaller files that would get processed first and that would take a lot longer then just ignoring them and focusing on the couple of hundred larger files.

Can anyone help me with a regular expression that does this? (I've tried and failed so far...)

User avatar
Support
Site Admin
Posts: 1727
Joined: Sun Oct 02, 2011 10:49 am

Re: Filters - Using regular expressions - user guide/examples?

Post by Support » Tue Oct 23, 2012 11:47 am

It can not be done with the current version. You are the 2nd person to ask for this, and we are working on implementing a size range setting to do just that. It will be available in the next version due by the end of the year.
Your support team.
http://SoftByteLabs.com

CLS
Posts: 9
Joined: Tue Oct 23, 2012 9:11 am

Re: Filters - Using regular expressions - user guide/examples?

Post by CLS » Sun Jan 13, 2013 3:41 pm

How is the new version of CloneMaster coming? Is there a new expected release date?

User avatar
Support
Site Admin
Posts: 1727
Joined: Sun Oct 02, 2011 10:49 am

Re: Filters - Using regular expressions - user guide/examples?

Post by Support » Sun Jan 13, 2013 3:47 pm

We are working on it right now. I expect the new release to be in a couple of weeks.
Your support team.
http://SoftByteLabs.com

CLS
Posts: 9
Joined: Tue Oct 23, 2012 9:11 am

Re: Filters - Using regular expressions - user guide/examples?

Post by CLS » Sun Apr 14, 2013 3:29 am

How's that upgrade coming? Three months ago you said a couple of weeks...

User avatar
Support
Site Admin
Posts: 1727
Joined: Sun Oct 02, 2011 10:49 am

Re: Filters - Using regular expressions - user guide/examples?

Post by Support » Tue Jun 25, 2013 3:25 pm

It's still isn't done yet, sorry. Our priority was to finish Raylectron v3 first.
Your support team.
http://SoftByteLabs.com

CLS
Posts: 9
Joined: Tue Oct 23, 2012 9:11 am

Re: Filters - Using regular expressions - user guide/examples?

Post by CLS » Fri Jun 28, 2013 4:39 pm

Do you have a new estimated release date? (I have a need for the feature I asked for above and my money is burning a hole in my pocket!)

User avatar
Support
Site Admin
Posts: 1727
Joined: Sun Oct 02, 2011 10:49 am

Re: Filters - Using regular expressions - user guide/examples?

Post by Support » Fri Jun 28, 2013 5:54 pm

Now that we are releasing Raylectron next week, CloneMaster is next. So it should be sometimes in July.
Your support team.
http://SoftByteLabs.com

CLS
Posts: 9
Joined: Tue Oct 23, 2012 9:11 am

Re: Filters - Using regular expressions - user guide/examples?

Post by CLS » Tue Aug 13, 2013 6:26 pm

Are we looking at July 2014 or another year?

User avatar
Support
Site Admin
Posts: 1727
Joined: Sun Oct 02, 2011 10:49 am

Re: Filters - Using regular expressions - user guide/examples?

Post by Support » Mon Oct 07, 2013 7:40 pm

ok, I have a beta 5.06 available which you can set the minimum and maximum file size. You can download it here...

http://SoftByteLabs.com/DirectDownload/ ... ter506.zip

Unzip to your Desktop and then move the file CloneMaster.exe to your CloneMaster installed folder. If all works well, we'll release the final version.
Your support team.
http://SoftByteLabs.com

CLS
Posts: 9
Joined: Tue Oct 23, 2012 9:11 am

Re: Filters - Using regular expressions - user guide/examples?

Post by CLS » Sat Nov 02, 2013 9:40 pm

After a half second or so it goes 'ding'... Nothing is ever displayed on the screen.

System Internals shows it is looking for:
C:\Program Files (x86)\CloneMaster\imageres.dll

After not finding it there it resolves to:
C:\Windows\SysWOW64\imageres.dll

But errors with:
FILE LOCKED WITH ONLY READERS

Thoughts?

User avatar
Support
Site Admin
Posts: 1727
Joined: Sun Oct 02, 2011 10:49 am

Re: Filters - Using regular expressions - user guide/examples?

Post by Support » Sat Nov 02, 2013 10:02 pm

Perhaps this solution will fix it...

http://answers.microsoft.com/en-us/wind ... fe317cd3f1
Your support team.
http://SoftByteLabs.com

CLS
Posts: 9
Joined: Tue Oct 23, 2012 9:11 am

Re: Filters - Using regular expressions - user guide/examples?

Post by CLS » Sun Nov 03, 2013 5:30 am

The link did not fix the problem.

I tried it on another Windows 7 x64 computer as well as 32 bit XP mode - same issue. (I watched it with task manager this time.) Launch it, half a second later it plays ding.wav and closes - nothing is ever displayed on the screen.

User avatar
Support
Site Admin
Posts: 1727
Joined: Sun Oct 02, 2011 10:49 am

Re: Filters - Using regular expressions - user guide/examples?

Post by Support » Sun Nov 03, 2013 9:39 am

I see what it is, we forgot to include the language files in the zip. Here is the installer instead, it contain all the files...
Attachments
CloneMaster_Setup.exe
(1.68 MiB) Downloaded 874 times
Your support team.
http://SoftByteLabs.com

CLS
Posts: 9
Joined: Tue Oct 23, 2012 9:11 am

Re: Filters - Using regular expressions - user guide/examples?

Post by CLS » Sun Nov 03, 2013 11:53 am

That seems to run and the screens look very promising!

I will make a copy some folders and test... I'll report back on my findings. (I'd never run a beta against data I'd want to keep - sorry! :D) Once finalized I'll be purchasing!

One thing I noticed - A checkbox for ignoring MP3 headers was added, any thoughts on adding an ignore checkbox for EXIF headers on images and metadata for movies?
http://en.wikipedia.org/wiki/Exchangeab ... ile_format

(I know everyone's first thought was porn collection! Shame on you all! I work with a couple of hundred engineers who feel the need to edit comments on images of inspections... multiple copies of the same pics and vids with slightly different metadata/properties on the 'details' tab. - It would be useful to be able to identify those duplicates and tell them to clean up their work when they complain about being out of storage space.)

User avatar
Support
Site Admin
Posts: 1727
Joined: Sun Oct 02, 2011 10:49 am

Re: Filters - Using regular expressions - user guide/examples?

Post by Support » Sun Nov 03, 2013 12:08 pm

ok, lets see what we can do for the EXIF header, shouldn't be that hard.
Your support team.
http://SoftByteLabs.com

User avatar
Support
Site Admin
Posts: 1727
Joined: Sun Oct 02, 2011 10:49 am

Re: Filters - Using regular expressions - user guide/examples?

Post by Support » Sun Nov 03, 2013 12:31 pm

Any chance you can provide me with a few sample EXIF files (images and videos) so we can test the "Test EXIF data only"? (you can PM them to me if you don't want them to be public).
Your support team.
http://SoftByteLabs.com

User avatar
Support
Site Admin
Posts: 1727
Joined: Sun Oct 02, 2011 10:49 am

Re: Filters - Using regular expressions - user guide/examples?

Post by Support » Tue Nov 05, 2013 12:39 pm

We have released v5.07 now and is ready for download. The "Check only audio data for MP3 files" now works correctly. Minimum and Maximum file size was also implemented and is available in v5.07+
Your support team.
http://SoftByteLabs.com

Post Reply