Erase CRW/CR2/DNG files without corresponding JPG

Barney Fife

Deleted

wontolla

It is working now regardless which folder you stored your RAW or JPG files. It will find them in any folder inside DCIM.

I struggled with it for days until I discovered you ALWAYS have to close the folders with closedir(). The compiler doesn't warn you and you think your code is simply wrong. I will never forget to include this bloody command.

It works as follows:

First, you delete the outtakes in playback mode (using Canon's interface).
Then, to delete the corresponding RAW files, you go to CHDK File Browser, select DCIM folder, push left button to display the popup menu and select "Purge RAW". A warning will ask you to confirm and voila! You have regained some SD space.

This feature will search through all the folders in DCIM looking for a RAW file (CRW/CR2 prefix or file extension) and if it finds one then will look for its JPG partner anywhere inside DCIM (comparing the 4 digit number assigned by the camera). If it is not found, the RAW file is erased.

I forgot to take some shots of the screen, I will post some tomorrow. Don't tell Nikon I'm using their camera with the only puropose of showing CHDK in action.

See reply #31
You can try it here. Remember this is NOT an official build. The purpose should be testing the "Purge RAW" feature only. Do not use it with important shots. I think it should be tested in all cameras first. because deleting files is always a sensitive operation.

allbest-w-a460-100d-78.zip - 0.21MB
allbest-w-a5xx-all.zip - 0.62MB
allbest-w-a6xx-all.zip - 1.32MB
allbest-w-a7xx-all.zip - 0.49MB
allbest-w-g7-all.zip - 0.84MB
allbest-w-ixus-all.zip - 1.24MB
allbest-w-s2is-all.zip - 0.63MB
allbest-w-s3is-100a-78.zip - 0.21MB

Any comments/suggestions will be appreciated.

This is the function:

static void fselect_purge_cb(unsigned int btn) {

    DIR             *d,  *d2,  *d3,  *d4;
    struct dirent   *de, *de2, *de3, *de4;
    char            sub_dir[20], sub_dir_search[20];
    char            selected_item[256];
    int             i, found=0;

    if (btn==MBOX_BTN_YES) {
        sprintf(current_dir+strlen(current_dir), "/%s", selected->name);
        d=opendir(current_dir);
        while ((de=readdir(d)) != NULL) {
            if (de->name[0] != '.' && de->name[1] != '.') {//If item is a file
                sprintf(sub_dir, "%s/%s", current_dir, de->name);
                d2=opendir(sub_dir);
                while ((de2=readdir(d2)) != NULL) {
                    if (de2->name[0] == 'C' || de2->name[9] == 'C') {//If file is RAW (Either CRW/CR2 prefix or file extension)
                        d3=opendir(current_dir);
                        while ((de3=readdir(d3)) != NULL) {
                            if (de3->name[0] != '.' && de3->name[1] != '.') {//If item is a file
                                sprintf(sub_dir_search, "%s/%s", current_dir, de3->name);
                                d4=opendir(sub_dir_search);
                                while ((de4=readdir(d4)) != NULL) {
                                    if (de2->name[4] == de4->name[4] && de2->name[5] == de4->name[5] &&//If the four digits of the Canon number are the same
                                        de2->name[6] == de4->name[6] && de2->name[7] == de4->name[7] &&
                                        de4->name[9] == 'J' && !(de4->name[0] == 'C' || de4->name[9] == 'C' || de4->name[0] == 0xE5)) {//I file is JPG and is not CRW/CR2
                                        started();
                                        found=1;//A JPG file with the same Canon number was found
                                    }                                 
                                }
                                closedir(d4);                 
                            }  
                        }
                        closedir(d3);
                        if (found == 0) {
                            sprintf(selected_item, "%s/%s", sub_dir, de2->name);
                            remove(selected_item);
                            finished();
                        }
                        else {
                            found=0;
                            finished();
                        }                             
                    }
                }
                closedir(d2);
            }
        }
        closedir(d);
        i=strlen(current_dir);
        while (current_dir[--i] != '/');
        current_dir[i]=0;
        gui_fselect_read_dir(current_dir);
    }
    gui_fselect_redraw = 2;
}

Yes, I know, 4 nested while loops, ugly.

Cheers!

PurgeRAW.diff

6.72KB

Barney Fife

Deleted

PhyrePhoX

hm, i tested your build for a620, to no effect. dont know what i'm doing wrong. at least nothing at all gets deleted - not the other way around. will make some more tests during next days.

wontolla

I'm askeered. Smiley (colloquialism for "scared", "afraid", for the non-english crowd) There's no safety "lock file" or "Purge RAW this folder only!" option?

I've done a lot of thinking and my conclusion is that the best "lock file" solution is having the JPG. If the JPG exists somewhere, the RAW will never be deleted.
On the other hand, as you mentioned, we have the Canon lock. It sets the JPG file as "read only" the same way Windows does. The problem is I have no clue how to know if a file is "read only" in C language, let me keep researching or see if somebody has a hint.
But as I said, there is little point to know if the JPG is "read only" because the moment THERE IS a JPG, the function won't erase the RAW file, no matter if it is "read only" or not.
I also tried protecting the RAW directly using the CHDK File Browser marking feature: If the RAW is marked, it won't be erased even if there is no correspondent JPG. But when you go back to DCIM folder the files are no longer marked.
Conclusion: I am also askeered from the user point of view, but I still have no idea how to program the "lock file" option.

I'm going to have to make a "Wontolla's Builds" folder now. I'm losing track of whose stuff I'm testing lately.

Jeje me too!, I have a mess. I tried Jucifer's configurable menu yesterday. There is something knew in this forum almost every day! Remember to put the Wontolla folder under Tests folder since it is not a usable build.

hm, i tested your build for a620, to no effect.

Thanks PhyrePhox, I knew there would be problems. Can you elaborate in what you are doing?

Barney Fife

Deleted

wontolla

Now I am pissed off! >🙁 <- note this is the first time I use one of this yellow faces so I'm really pissed off!

I must have done something wrong when generating builds for every camera, it's the first time I do it.

This is much worse than GrAndsenberg uncertainty principle, this is Murphy's Law. But well, when they opened Disneyland nothing worked too right?

Let me try again tonight, I will need another bag of coffee. Keep those folders of yours for latter Barney...I'll be back 😎

Barney Fife

Deleted

Jucifer

wontolla
Jeje me too!, I have a mess. I tried Jucifer's configurable menu yesterday. There is something knew in this forum almost every day!

The custom user menu is fingalo's, not mine... speaking of which; new day, new stuff. trunk-r314-mod: get_file_counter + faster_menu_navigation

wontolla

@PhyrePhox and Barney
I tried on my camera they very same files I uploaded and they work!
My guess is you didn't execute the Purge RAW in DCIM folder, I mean, while the cursor is highlighting the folder.
I noticed the Purge RAW option always displays in the popup menu regardless it's a file or folder highlighted. So last night I worked in the code and now it considers all possibilities and situations when the user executes the Purge RAW feature (partly to solve Barney's concerns). Now if you execute the command in a non allowed item it will cancel the purging and display a window saying so. I will post detailed explanations of this new version of Purge RAW.

Thanks for helping me debug this thing.

Errata:

The custom user menu is fingalo's, not mine...

That is right, I meant/shoud've said: "Jucifer's modifications to Fingalo's custom menu."

Barney Fife

Deleted

wontolla

It works as follows:

The first step is to delete the JPG outtakes in playback mode (using Canon's interface).

Now you have 3 options to use Purge RAW function and erase the corresponding RAW files:

1.- Delete all RAW files in DCIM folder including subfolders.
Useful when:
You keep a separate folder for RAW files.
You set up the camera to create a new folder everyday (several days trek, holydays etc).
You want to erase the occasional mischievous RAW file stored in a different folder.

Go to CHDK File Browser, select DCIM folder, push left button to display the popup menu and select "Purge RAW" (See photo 1). A warning will ask you to confirm (See photo 2) and voila! You have regained some SD space.

This feature will search through all the folders in DCIM looking for a RAW file (CRW/CR2 prefix or file extension) and if it finds one then will look for its JPG partner anywhere inside DCIM (comparing the 4 digit number assigned by the camera). If it is not found, the RAW file is erased.

2.- Delete all RAW files in a selected folder.
Useful when:
You keep RAW files in the same folder as JPG files
You only want get rid of some RAW files. (Very important shots in some other folder).
You know which folder has a lot of outtakes.

Go to CHDK File Browser and inside DCIM folder select the folder you want (E.g. "102CANON") and push the left button to display the popup menu. Select "Purge RAW". A warning will ask you to confirm and all the RAW files in ONLY that folder will be erased.

This option works like the previous one but only on the selected Canon folder, the rest are untouched.

3.- Delete some or all RAW files in a list.
Useful when:
You want to protect specific RAW files (to use them later for HDR)

Go to CHDK File Browser, enter DCIM folder, enter a sub folder (E.g. "102CANON") push the left button to display the popup menu. Select "Purge RAW" (See photo 3). A warning will ask you to confirm and all the RAW files (See photo 4) will be gone.

Now the useful thing of this option is that you can mark the RAW files you DON'T want to erase (protect files). The Purge RAW function will erase the rest of the RAW files ignoring the marked ones.

You can try it here but this is NOT an official build. The purpose should be testing the "Purge RAW" feature only. Do not use it with important shots. Remember, the only way for the community to get stable releases is to try/test these aditions BEFORE commiting. Erasing files in camera is an irreversible operation.

Links updated 7th March. Please report if it works in your camera or not.

allbest-w-a460-100d-43.zip - 0.21MB
allbest-w-a5xx-all.zip - 0.63MB
allbest-w-a6xx-all.zip - 1.12MB
allbest-w-a7xx-all.zip - 0.49MB
allbest-w-g7-all.zip - 0.84MB
allbest-w-ixus-all.zip - 2.07MB
allbest-w-s2is-all.zip - 0.63MB
allbest-w-s3is-100a-43.zip - 0.21MB

ixus950_sd850-100c.zip - 0.21MB

Any comments/suggestions will be appreciated.

I'm not writing the C code in this post, it is too long. If you want to see it here is the diff file. It is getting complicated so I started adding comments.

purgeraw.diff - 0.01MB

Cheers!

wontolla

Thanks Barney. Yes, I always say: "Some of these days I will tune my CHDK configuration, let's wait until developers stop making all this new changes" AHAAA!!

"This is like painting the Forth Bridge."
This expression reffers to a big red bridge we have here similar to the Golden Gate. They are always painting it to protect it from salted water. The legend claims that whenever they finish painting, it is time to start again from the beginning. It's and endless job.

I have several SD cards, some for photography and some for testing.

Barney Fife

Deleted

PhyrePhoX

hm, sorry to say that, but my a620 dies when i boot your firmware. blue led lights up briefly, then cam shuts down. i guess this is having to do with ram (same happened to s3is builds last time). you can find out more about it here: Remaining RAW+JPG shots + filespace icon/text- done 😁 (and following post).
my guess, at least 😀

edit: btw, the diff is so small in size, you can attach it directly in your post next time, please 😀

wontolla

Thanks PhyrePhox. I made the changes suggested by GrAnd and uploaded a new build to the link above.

I hope more users can test this in different models to see if this is a problem with the A620 or is a problem with all non-S3_IS cameras.

I uploaded the diff file to zshare because you can only show four files in this forum. In fact, originally I had 6 photos.

For the record:
The changes where made in \platform\a620\sub\100f\Makefile.inc
MEMISOSTART=0x9F990 > MEMISOSTART=0xAF990
MEMISOSIZE=0x30000 > MEMISOSIZE=0x40000

PhyrePhoX

hm, same behaviour.
i will append your diff to my local version to see if memory is the issue here.

PhyrePhoX

argh, just lost an hour figuring out why i cant compile your changes in a freshly exported svn trunk. turns out official trunk is broken since #316.
see error compiling chdk since #316

PhyrePhoX

i applied your diff to a fresh export of #314 (by the way #319 fixed the issue i reported) and it compiled fine, and also boots fine on my a620. i didnt touch the memory stuff. so there must be something wrong in your environment, wontolla. as i see it, MEMISOSIZE doesnt have to be resized, at least not for a few more builds.

on the other hand: your feature works,nice! maybe enable it so you can use it in the root folder as well, not only when you mark the dcim folder (that was my mistake at first).
another suggestion: when finished, generate a popup stating the number of raw files purged, will make debugging and using easier.

wontolla

Thanks PhyrePhox.
Now we know my code is not the problem but my builds. I'm clueless about the process of building for all cameras, I will open a new thread to ask for advice.

Come on people! All the links have been downloaded at least one time. At least comment if you could load it in your camera or not. If the function is working or not. You can do it in just one sentence!

on the other hand: your feature works,nice! maybe enable it so you can use it in the root folder as well, not only when you mark the dcim folder

Originally it was like that, but I wanted to limit the execution of "Purge RAW" to avoid possible problems. It is not supposed to be called in the CHDK folder is it? It wouldn't cause any harm (there are no RAW files in there duh!) but I am trying to make sure people KNOW what they are doing before executing it.

another suggestion: when finished, generate a popup stating the number of raw files purged, will make debugging and using easier.

That is a very good idea, a simple counter will do the trick. May be also displaying the amount of memory claimed just for fun!
I was thinking on display the RAW files to be purged BEFORE the actual deleting. But that requires a two-pass process: One to count the RAW files found and the other to do the erasing.

« Previous Page Next Page »