supplierdeeply

Erase CRW/CR2/DNG files without corresponding JPG

  • 79 Replies
  • 18534 Views
*

Offline wontolla

  • ****
  • 413
  • S3 & G9 & A720
  • Publish
    Erase CRW/CR2/DNG files without corresponding JPG
    « on: 15 / February / 2008, 13:27:32 »
    Advertisements
    Last edit: 11 / May / 2009

    I rewrote the code because the Purge RAW function was not working with some cameras.
    This new version will:

    Erase DNGs too.
    Show before hand, the amount of memory to be claimed.
    Warn you when there are no RAW/DNGs to be erased.

    Due to my poor debugging skills, I deleted the function's capability of searching through several folders, which was the faulty bit. It was complex and un-useful in my opinion (I hope no-one misses this).

    From now on, you will need to run the function from inside a Canon folder. Remember that you still can keep important RAW files from being wiped out by selecting them before purge.

    Any comments/suggestions are welcomed.

    The diff file is here.

    I hope I can build and upload for people to test. The number of supported cameras has increased and it seems compilation time will be huge!  :o

    Have fun!







    Last edit: 24 / April / 2008

    It works as follows:

    The first step is to delete the JPG outtakes in playback mode (using Canon's interface).

    Now you have 3 options to use Purge RAW function and erase the corresponding RAW files:

    1.- Delete all RAW files in DCIM folder including subfolders.
    Useful when:
       You keep a separate folder for RAW files.
       You set up the camera to create a new folder everyday (several days trek, holydays etc).
       You want to erase the occasional mischievous RAW file stored in a different folder.


    Go to CHDK File Browser, select DCIM folder, push left button to display the popup menu and select "Purge RAW" (See photo 1). A warning will ask you to confirm (See photo 2) and voila! You have regained some SD space.

    This feature will search through all the folders in DCIM looking for a RAW file (CRW/CR2 prefix or file extension) and if it finds one then will look for its JPG partner anywhere inside DCIM (comparing the 4 digit number assigned by the camera). If it is not found, the RAW file is erased.


    2.- Delete all RAW files in a selected folder.
    Useful when:
       You keep RAW files in the same folder as JPG files
       You only want get rid of some RAW files. (Very important shots in some other folder).
       You know which folder has a lot of outtakes.


    Go to CHDK File Browser and inside DCIM folder select the folder you want (E.g. "102CANON") and push the left button to display the popup menu. Select "Purge RAW". A warning will ask you to confirm and all the RAW files in ONLY that folder will be erased.

    This option works like the previous one but only on the selected Canon folder, the rest are untouched.


    3.- Delete some or all RAW files in a list.
    Useful when:
       You want to protect specific RAW files (to use them later for HDR)

    Go to CHDK File Browser, enter DCIM folder, enter a sub folder (E.g. "102CANON") push the left button to display the popup menu. Select "Purge RAW" (See photo 3). A warning will ask you to confirm and all the RAW files (See photo 4) will be gone.

    Now the useful thing of this option is that you can mark the RAW files you DON'T want to erase (protect files). The Purge RAW function will erase the rest of the RAW files ignoring the marked ones.

    Some screenshots in replay #31

    You can try it here but this is NOT an official build. The purpose should be testing the "Purge RAW" feature only. Do not use it with important shots. Remember, the only way for the community to get stable releases is to try/test these aditions BEFORE commiting. Erasing files in camera is an irreversible operation.

    Links updated 24th April. Please report if it works in your camera or not. Any other link in this thread is outdated.

    http://wontolla.freeweb7.com/CHDK/PurgeRAW/allbest-a450-100d-50.zip
    http://wontolla.freeweb7.com/CHDK/PurgeRAW/allbest-a460-100d-50.zip
    http://wontolla.freeweb7.com/CHDK/PurgeRAW/allbest-a530-100a-50.zip
    http://wontolla.freeweb7.com/CHDK/PurgeRAW/allbest-a540-100b-50.zip
    http://wontolla.freeweb7.com/CHDK/PurgeRAW/allbest-a550-100c-50.zip
    http://wontolla.freeweb7.com/CHDK/PurgeRAW/allbest-a560-100a-50.zip
    http://wontolla.freeweb7.com/CHDK/PurgeRAW/allbest-a570-100e-50.zip
    http://wontolla.freeweb7.com/CHDK/PurgeRAW/allbest-a570-101a-50.zip
    http://wontolla.freeweb7.com/CHDK/PurgeRAW/allbest-a610-100e-50.zip
    http://wontolla.freeweb7.com/CHDK/PurgeRAW/allbest-a610-100f-50.zip
    http://wontolla.freeweb7.com/CHDK/PurgeRAW/allbest-a620-100f-50.zip
    http://wontolla.freeweb7.com/CHDK/PurgeRAW/allbest-a630-100c-50.zip
    http://wontolla.freeweb7.com/CHDK/PurgeRAW/allbest-a640-100b-50.zip
    http://wontolla.freeweb7.com/CHDK/PurgeRAW/allbest-a650-100d-50.zip
    http://wontolla.freeweb7.com/CHDK/PurgeRAW/allbest-a700-100b-50.zip
    http://wontolla.freeweb7.com/CHDK/PurgeRAW/allbest-a710-100a-50.zip
    http://wontolla.freeweb7.com/CHDK/PurgeRAW/allbest-a720-100c-50.zip
    http://wontolla.freeweb7.com/CHDK/PurgeRAW/allbest-g7-100e-50.zip
    http://wontolla.freeweb7.com/CHDK/PurgeRAW/allbest-g7-100g-50.zip
    http://wontolla.freeweb7.com/CHDK/PurgeRAW/allbest-g7-100i-50.zip
    http://wontolla.freeweb7.com/CHDK/PurgeRAW/allbest-g7-100i-50.zip
    http://wontolla.freeweb7.com/CHDK/PurgeRAW/allbest-ixus55_sd450-100b-50.zip
    http://wontolla.freeweb7.com/CHDK/PurgeRAW/allbest-ixus55_sd450-100c-50.zip
    http://wontolla.freeweb7.com/CHDK/PurgeRAW/allbest-ixus55_sd450-100d-50.zip
    http://wontolla.freeweb7.com/CHDK/PurgeRAW/allbest-ixus65_sd630-100a-50.zip
    http://wontolla.freeweb7.com/CHDK/PurgeRAW/allbest-ixus700_sd500-101a-50.zip
    http://wontolla.freeweb7.com/CHDK/PurgeRAW/allbest-ixus700_sd500-101b-50.zip
    http://wontolla.freeweb7.com/CHDK/PurgeRAW/allbest-ixus70_sd1000-100c-50.zip
    http://wontolla.freeweb7.com/CHDK/PurgeRAW/allbest-ixus70_sd1000-101b-50.zip
    http://wontolla.freeweb7.com/CHDK/PurgeRAW/allbest-ixus70_sd1000-102a-50.zip
    http://wontolla.freeweb7.com/CHDK/PurgeRAW/allbest-ixus800_sd700-100b-50.zip
    http://wontolla.freeweb7.com/CHDK/PurgeRAW/allbest-ixus800_sd700-101b-50.zip
    http://wontolla.freeweb7.com/CHDK/PurgeRAW/allbest-ixus850_sd800-100e-50.zip
    http://wontolla.freeweb7.com/CHDK/PurgeRAW/allbest-ixus860_sd870-100c-50.zip
    http://wontolla.freeweb7.com/CHDK/PurgeRAW/allbest-ixus950_sd850-100c-50.zip
    http://wontolla.freeweb7.com/CHDK/PurgeRAW/allbest-s2is-100e-50.zip
    http://wontolla.freeweb7.com/CHDK/PurgeRAW/allbest-s2is-100f-50.zip
    http://wontolla.freeweb7.com/CHDK/PurgeRAW/allbest-s2is-100g-50.zip
    http://wontolla.freeweb7.com/CHDK/PurgeRAW/allbest-s3is-100a-50.zip
    http://wontolla.freeweb7.com/CHDK/PurgeRAW/allbest-s5is-101a-50.zip
    http://wontolla.freeweb7.com/CHDK/PurgeRAW/allbest-s5is-101b-50.zip
    http://wontolla.freeweb7.com/CHDK/PurgeRAW/allbest-tx1-100g-50.zip
    http://wontolla.freeweb7.com/CHDK/PurgeRAW/allbest-tx1-101b-50.zip

    Any comments/suggestions will be appreciated.

    I'm not writing the C code in this post, it is too long. If you want to see it here is the diff file. It is getting complicated so I started adding comments.

    purgeraw.diff - 0.01MB

    Cheers!











    Since I started using CHDK for RAW pictures, my 2Gb SD cards feel like 512Mb ones. I use to erase outtake pictures in the field to reclaim some space and also to save time when choosing keepers back home. But for it to be worth, the CRW must be erased too.

    I've been studying the CHDK code and I manage to erase the CRW automatically when erasing the JPG in the file browser (small addition to the fselect_delete_file_cb() function). But it would be much easier for the user to be looking at the picture instead the file name.

    I think it would be nice when in playback mode, when pressing the "erase" button the CRW goes bye bye too, but I don't know how to sense keys when in playback mode. Any suggestions?

    Other way is to first erase all the JPGs you want and then in the file browser, hit the left-key menu and select some kind of "erase crw of missing jpgs", but for that I need a SEARCH function or two nested "while" loops to search for every CRW file if it exist the corresponding JPG (with the file number).


    I hope the experienced guys with the CHDK C code can give some hints or suggestions on this.

    In few words:
    How to react to the "erase" button when in playback mode?
    If this is not possible, how to search for a file name in the directory?

    For the first approach, it is only to add this lines in the appropriate place:
    Code: [Select]
           
    selected_file[16]='C';
    selected_file[17]='R';
    selected_file[18]='W';
    selected_file[25]='C';
    selected_file[26]='R';
    selected_file[27]='W';
    remove(selected_file);

    For the second approach, I want to create a new function similar to fselect_delete_folder_cb(), for example
    Code: [Select]
    while (de) {
       if (de->name[0] != 0xE5 /* deleted entry */ && (de->name[0]!='.' || (de->name[1]!='.' && de->name[1]!=0) || (de->name[1]=='.' && de->name[2]!=0))) {
          started();
          sprintf(selected_file, "%s/%s", current_dir, de->name);
          remove(selected_file);
          finished();
       }
       de = readdir(d);
    }
    closedir(d);

    I want to modify this part to detect if the file is a CRW
    Code: [Select]
    if (de->name[0]=='C'...
    Then to start a search in the directory looking for the JPG file with the same number assigned by the camera
    Code: [Select]
    if (canon_number[0]==de->name[4] && canon_number[1]==de->name[5] && canon_number[2]==de->name[6] && canon_number[3]==de->name[7])
    If not found, erase the CRW file and do the same with the next CRW file until all of them are searched.

    Thank you in advance guys and have fun!
    « Last Edit: 11 / May / 2009, 11:57:26 by wontolla »

    *

    Offline whim

    • ******
    • 2013
    • A495/590/620/630 ixus70/115/220/230/300/870 S95
  • Publish
    Re: Erase JPG along with corresponding CRW in playback mode
    « Reply #1 on: 20 / February / 2008, 09:18:47 »
    Hi !
    Must have somehow missed this thread - very interesting !
    Haven't looked into the code yet, but it made me think
    that we should at least also have an indicator pop up when
    you display a JPG, like +RAW or sth, that shows you a RAW
    is associated with it.
    Not very helpful to you atm, but at least gets the thread a bump  :D

    wim

    *

    Offline wontolla

    • ****
    • 413
    • S3 & G9 & A720
  • Publish
    Re: Erase JPG along with corresponding CRW in playback mode
    « Reply #2 on: 20 / February / 2008, 10:02:21 »
    Thanks whim for the response, I was about to ask again to bump it.

    That's a good suggestion; I didn't think about it before but of course whould be useful to know if there is a CRW associated. The drawback is you have to perform a search each time, I don't know if it's really a drawback in terms of performance. If I can get this to work I will implement it and see.

    At the time I managed to detect the ERASE key and to cancel the erase if you hit CANCEL in the display or if you push another key.

    Next step is to understand how the readdir() function works. I am studying the gui_fselect.c file but it seems that the only way to search a directory is deleting a file each time because it always reads the first file, am I wrong?

    Someone with experience with the readdir() function or knows who wrote the File Select feature in the file browser?

    Thanks for your help

    *

    Offline jetzt

    • ****
    • 316
    • [A710IS,(SD200)]
  • Publish
    Re: Erase JPG along with corresponding CRW in playback mode
    « Reply #3 on: 20 / February / 2008, 10:21:11 »
    I don't think it is necessary to serach, you know the filename, so you can check wether there is some file or not.

    Topic idea: Yes, I think it would be useful.


    *

    Offline PhyrePhoX

    • *****
    • 2253
    • make RAW not WAR
      • PhyreWorX
  • Publish
    Re: Erase JPG along with corresponding CRW in playback mode
    « Reply #4 on: 20 / February / 2008, 11:49:36 »
    I don't think it is necessary to serach, you know the filename, so you can check wether there is some file or not.

    Topic idea: Yes, I think it would be useful.
    well you'd have to look for that file in all directories though. not always are RAW files written in the same dir.
    you also can do it the other way around: i'd like a function that deletes all RAWs to which there is no corresponding jpeg. but with your new function, i dont need that anymore (because there will be no "single" raw files anymore"

    *

    Offline GrAnd

    • ****
    • 916
    • [A610, S3IS]
      • CHDK
  • Publish
    Re: Erase JPG along with corresponding CRW in playback mode
    « Reply #5 on: 20 / February / 2008, 14:07:57 »
    Someone with experience with the readdir() function or knows who wrote the File Select feature in the file browser?
    readdir() :)
    CHDK Developer.

    *

    Offline GrAnd

    • ****
    • 916
    • [A610, S3IS]
      • CHDK
  • Publish
    Re: Erase JPG along with corresponding CRW in playback mode
    « Reply #6 on: 20 / February / 2008, 14:13:09 »
    BTW. There is an icon in review mode which indicates the existence of some paired with Jpeg file. But that icon appears only if RAW file is saved in the same directory as a Jpeg, and RAW file was taken in a previous shooting session (camera was switched off/on). If that icon is shown the RAW file will be deleted with a Jpeg file at once by standard erase procedure.
    CHDK Developer.

    *

    Offline wontolla

    • ****
    • 413
    • S3 & G9 & A720
  • Publish
    Re: Erase JPG along with corresponding CRW in playback mode
    « Reply #7 on: 21 / February / 2008, 10:03:31 »
    Thanks guys for the support!

    Quote
    not always are RAW files written in the same dir.
    Yes, that's another problem. For now I am ignoring it. Once it works with files in the same directory I will address this.

    Quote
    a function that deletes all RAWs to which there is no corresponding jpeg
    Good idea, in fact that would eliminate the previous problem. I was thinking to mark in red the JPG files in playback, marking the CRWs automatically and then you just have to erase them all in the file browser. But the current marking function is implemented exclusively in the file browser with pointers and stuff and I would have to copy all that structure to my function witch sounds redundant. But is a very good plan B.

    Quote
    readdir()
    Sorry for the googleable question. The thing is I don't have internet at home where y play with the camera, so I thought "I'll ask this in the forum tomorrow" instead of "I'll google this tomorrow". Thanks for the link, it even contains an example and I already compiled it here at work.

    I'll continue with this function back home and let you know the progress.

    Have fun!

    Note: adjetive "googleable" and verb "google" do NOT exist!"


    *

    Offline wontolla

    • ****
    • 413
    • S3 & G9 & A720
  • Publish
    Re: Erase JPG along with corresponding CRW in playback mode
    « Reply #8 on: 25 / February / 2008, 09:59:45 »
    After 10 cups of coffee it is working.

    If you are in the middle of the Amazon jungle running out of memory, you can go to playback mode and erase some outtake pictures. Then, in the file browser enter a folder, left-click and select "Purge RAW" (see Figure 1). All the RAW files without a JPG couple will be erased giving you 3 times more free space than erasing JPGs only!

    Done with the marketing! Now the stuff behind the scenes:

    I thought the easiest thing was to just create a new function that takes a RAW file (either CRW/CR2 prefix or extension), then looks for a correspondent non RAW file by comparing the number assigned by the camera. If not found, erase that file and go with the next. Here is the code:

    Code: (c) [Select]
    static void fselect_purge_cb(unsigned int btn) {
        struct fitem  *ptr, *ptr2;
        int found=0;

        if (btn != MBOX_BTN_YES) return;

        for (ptr=head; ptr; ptr=ptr->next) {
            if ((ptr->name[0] == 'C' || ptr->name[9] == 'C') && ptr->attr != 0xFF && !(ptr->attr & DOS_ATTR_DIRECTORY)) {
                for (ptr2=head; ptr2; ptr2=ptr2->next) {
                    if ((ptr->name[4] == ptr2->name[4] && ptr->name[5] == ptr2->name[5] &&
                         ptr->name[6] == ptr2->name[6] && ptr->name[7] == ptr2->name[7]) && !(ptr2->name[0] == 'C' || ptr2->name[9] == 'C')) {
                        started();
                        found=1;
                    }
                }           
                if (found == 0) {
                    sprintf(selected_file, "%s/%s", current_dir, ptr->name);
                    remove(selected_file);
                    finished();
                }
                else {
                    found=0;
                    finished();
                }
            }
        }
        selected_file[0]=0;
        gui_fselect_read_dir(current_dir);
        gui_fselect_redraw = 2;
    }

    I didn't go with the original idea of erasing each time in playback mode because I was making a mess in the gui.c. An additional function in the gui_fselect file would be a "cleaner" approach. Also there could be problems like different folders as PhyrePhox said or when the camera does erase the RAW file as GrAnd mentioned.

    I was concerned about the two nested "for" loops but it looks like there is not noticeable difference with the normal erasing function. I was underestimating the processor.

    I used the word purge to differentiate with delete but I don't know if it applies. For me, this word means "get rid of" but a native English speaker can correct me.

    I made several tests and it always worked as expected, the RAW files must have the prefix CRW or CR2 or file extension. Remember you are ERASING ORIGINAL DATA. Test the function first and use it if you feel confident.

    I include the diff file and the S3 build in case some S3 eager-hand user (yes you Barney (if you have time of course)) wants to try it.

    Thank you guys for your ideas. Have fun!

  • Publish
    Re: Erase JPG along with corresponding CRW - done
    « Reply #9 on: 25 / February / 2008, 10:41:26 »
    Deleted
    « Last Edit: 22 / April / 2008, 12:54:10 by Barney Fife »
    [acseven/admin commented out: please refrain from more direct offensive language to any user. FW complaints to me] I felt it imperative to withdraw my TOTAL participation. Nobody has my permission, nor the right, to reinstate MY posts. Make-do with my quoted text in others' replies only. Bye

     

    Related Topics