NOW: Exiftool usage lessons .. WAS: Bug ?? in Auto CHDK to Exif - page 2 - General Help and Assistance on using CHDK stable releases - CHDK Forum
« previous next »
Pages: 1 [2] 3   Go Down

NOW: Exiftool usage lessons .. WAS: Bug ?? in Auto CHDK to Exif

  • 22 Replies
  • 13831 Views
*

Offline Sow.N.Reap

  • *
  • 23
  • Canon S5is
Re: Bug ?? in Auto CHDK to Exif
« Reply #10 on: 10 / August / 2009, 17:38:15 »
Advertisements
Hooray !! It may have taken most of the day (yeah I'm a little command interpreter illiterate) but I finally figured out the right place for the quotes & spaces to get this to work:

exiftool.exe -k "-canon:ExposureTime>EXIF:ExposureTime" "-canon:FNumber>EXIF:FNumber" "-canon:OwnerName>EXIF:Artist" "-canon:FocusDistanceUpper>EXIF:SubjectDistance" -P

I'm on my way now that I figured that out! :)

And got it as a shortcut so I can drag/drop files from iMatch to it so iMatch doesn't have to rescan every time a change is made outside that program.

Oh yeah ... I can see it is a little quicker than AutoCHDK now that I actually have the params right & incorporated into my work flow. And I can see all the possibilities it has to offer now too.

Thanks every one for all your help.
Logged
Canon S5is .. My Flickr page
*

Offline Anaglyphic

  • ***
  • 129
  • Anaglyphic lives!
Re: Bug ?? in Auto CHDK to Exif
« Reply #11 on: 11 / August / 2009, 12:32:19 »
Yay. :D

And while fe50 fixed that small problem with "Auto CHDK to Exif", it's still bad and broken. Specifying -e prevents exiftool from generating Composite tags. But it shouldn't be writing them anyway! "The values of the composite tags are Derived From the values of other tags. These are convenience tags which are calculated after all other information is extracted.". So they should not actually be set directly, but especially when you are calling the master of tagging tools that auto-generates them for you? I like this "application" less and less. That means that since this utility has been around, it's been wrong? Niiiice.

Seriously, exiftool is baadazz. (as much as a tagging program can be baadazz anyway)

Regarding sidecar files, I'll give you one:

exiftool -ext jpg -ext dng -tagsfromfile @ -srcfile %d%f.mie -r .

This will, from the starting directory, go though every image in every subdirectory with .jpg or .dng extension, and copy all the metadata to a same-named sidecar file in MIE format. The command block-copies every tag into the MIE file, even tags exiftool doesn't currently understand, so as the software improves, you'll always have the original tagging regardless of the various edits and saved formats you go though.

Another variant of that command can strip out the .dng previews and .jpg thumbnails from the sidecar file as it creates it, while dng previews are only ~2M each, and jpg thumbnails even smaller, still it adds up for 100,000 images (and you can always regenerate them.):

exiftool -ext jpg -ext dng -tagsfromfile @ --previewimage --thumbnailimage -srcfile %d%f.mie -r .

One warning: you know exiftool is powerful. This is a great example for the possibly-pending wiki article. Be aware that rearranging the sequence of parameters has relevance; if you put the tag removals (the --<tag>) in front of -tagsfromfile @, the @ is the token for referencing the file you're processing... what'll happen is you'll strip out all your thumbnails and previews from the actual images. Oops! (and you can always regenerate them here, too.)

Oh, and if you want to put it in a batch file, don't forget to double up the %'s.... %%d%%f.mie. I think. Try it. I rarely use Windows.
Logged
Since we cannot know all that there is to be known about anything,
 we ought to know a little about everything. -- Blaise Pascal
*

Offline Sow.N.Reap

  • *
  • 23
  • Canon S5is
Re: Bug ?? in Auto CHDK to Exif
« Reply #12 on: 12 / August / 2009, 06:05:16 »
Quote from: Anaglyphic on 11 / August / 2009, 12:32:19

Yay. :D

[snipped] That means that since this utility has been around, it's been wrong? Niiiice.

Yep you talked me into it. All that cajoling & yelling. :)

That's sort of why I started this new thread. I didn't notice any posts about this problem.
 
Quote from: Anaglyphic on 11 / August / 2009, 12:32:19
Seriously, exiftool is baadazz. (as much as a tagging program can be baadazz anyway)
Since I started reading & playing with it I realized I could do things I hadn't even thought about doing!

Quote from: Anaglyphic on 11 / August / 2009, 12:32:19
Regarding sidecar files, I'll give you one:

exiftool -ext jpg -ext dng -tagsfromfile @ -srcfile %d%f.mie -r .
This is what I was working on when my brain blow a fuse. :) I wanted to have backups before starting to use this on real photos .. not just my test ones. I hadn't quite figured out how to let it know the source file. So when it had "srcfile" in the example it meant to actually use that word not a path of some sort. Doh. That and I missed the "@" ... So thanks for that one!

I thought I was backing up the data by doing "exiftool.exe  -w EXT" but after looking at the file saw it didn't copy everything so starting working on the this one. So your example would be a good one for the Wiki.  

The next one I'll need to figure out (which shouldn't be to hard) is how to restore the metadata from the .mie file. Just in case ...

Quote from: Anaglyphic on 11 / August / 2009, 12:32:19

Another variant of that command can strip out the .dng previews and .jpg thumbnails from the sidecar file as it creates it, while dng previews are only ~2M each, and jpg thumbnails even smaller, still it adds up for 100,000 images (and you can always regenerate them.):

exiftool -ext jpg -ext dng -tagsfromfile @ --previewimage --thumbnailimage -srcfile %d%f.mie -r .

Dumb question .. but if it strips out the thumbnails from the .mie file how would you re-create them if needed .. for example. they got deleted from the actual photo? Is there something in a header or other data in the file that knows how to regenerate it?

Quote from: Anaglyphic on 11 / August / 2009, 12:32:19
One warning: you know exiftool is powerful. This is a great example for the possibly-pending wiki article. Be aware that rearranging the sequence of parameters has relevance; if you put the tag removals (the --<tag>) in front of -tagsfromfile @, the @ is the token for referencing the file you're processing... what'll happen is you'll strip out all your thumbnails and previews from the actual images. Oops! (and you can always regenerate them here, too.)

I read a couples examples of the difference that re-arranging the tags could mean & this is a very good example. And very important to know.

Quote from: Anaglyphic on 11 / August / 2009, 12:32:19
Oh, and if you want to put it in a batch file, don't forget to double up the %'s.... %%d%%f.mie. I think. Try it. I rarely use Windows.

Yep .. I remember reading that also. Batch files - another thing I'm not real familiar with writing. But something else I have to learn because I guess I'm at the limit on how many tags, parameters, etc I can put in the shortcut "target" line & I have a few more I need to add. I may need/want to adjust my workflow somewhat. But this tool will make it possible to eliminate 1 step in my current workflow & uninstall 1 other program that won't be needed anymore. So that's a good thing.

I most definitely appreciate your help & examples.
« Last Edit: 12 / August / 2009, 06:15:26 by Sow.N.Reap »
Logged
Canon S5is .. My Flickr page
*

Offline Anaglyphic

  • ***
  • 129
  • Anaglyphic lives!
Re: Bug ?? in Auto CHDK to Exif
« Reply #13 on: 12 / August / 2009, 16:16:16 »
Logged
Since we cannot know all that there is to be known about anything,
 we ought to know a little about everything. -- Blaise Pascal
*

Offline Sow.N.Reap

  • *
  • 23
  • Canon S5is
Re: Bug ?? in Auto CHDK to Exif
« Reply #14 on: 12 / August / 2009, 22:40:20 »
I sure don't mind being a guinea pig but I don't want to take advantage of your help either .. at least too much :). Maybe some of this thread can be useful towards writing that article.

I think more people would try exiftool if it didn't appear such a daunting task to learn it for those of us that are somewhat command line ignorant. I first looked at it & said Huh???

Quote from: Anaglyphic on 12 / August / 2009, 16:16:16

Two options, -overwrite_original and -overwrite_original_in_place exist. Obviously a bad idea if you have no backups of your images... just in case. :b The second one, overwrite in place, is the better of the two, it preserves all the original file attributes (date, time, permissions, etc.) but it's execution is slightly slower because of this. On a modern machine working on a local hard drive, this overhead is minimal.
I'm glad you clarified these differences. I"m not wanting to have a bunch of duplicate files which is why I was interested in backing up the exif & using the .mie file.

Just curious ...Is there another manual somewhere besides the HTML docs in the Perl download & the website info? I find it very hard to read the documentation in the command window & assumed it was similar to the other docs. Does it contain different info than the website? I haven't had much chance to read all the HTML docs mostly been using the website.

Quote from: Sow.N.Reap on 12 / August / 2009, 06:05:16
But something else I have to learn because I guess I'm at the limit on how many tags, parameters, etc I can put in the shortcut "target" line & I have a few more I need to add.
Quote from: Anaglyphic on 12 / August / 2009, 16:16:16
Ah! Here we go. Instructor time. Nope. There is no (reasonable!) limit. The biggest limitation is using the rename-the-exe-file method. There's so many characters that Windows can't allow as part of a filename. But there's multiple solutions:

Yep .. that's the problem I found. For reasons I won't go into I was wanting to use shortcuts for drag-n-drop. But now I realized I can do so much more with exiftool besides just writing the "correct" exposure time & Fnumber I decided to go a different route. So I starting reading about the config file and did see the execute option mentioned. Just haven't had enough time to study them in more detail. Didn't know I had homework to do before reading your message. :D Silly me was storm chasing to see if I could get some lightening pics.

You've given me a lot of options that I'll have to study in more detail.

Another dumb question ... are there significant differences between the Windows .exe  & the Perl version?

Quote from: Anaglyphic on 12 / August / 2009, 16:16:16
:blink: Longest. Post. Ever. :blink:  
(and terribly mis-titled!)
Have fun. Class dismissed!
I'm not sure whether to feel honored or sorry for you having to deal with me in order to have the longest post ever.   :)

Oh .. now it's recess time. I thought that was last night!

BTW .. as you can see I've changed the title. Hope that helps to more accurately reflect the new discussion.


« Last Edit: 13 / August / 2009, 00:16:22 by Sow.N.Reap »
Logged
Canon S5is .. My Flickr page
*

Offline Anaglyphic

  • ***
  • 129
  • Anaglyphic lives!
Re: NOW: Exiftool usage lessons .. WAS: Bug ?? in Auto CHDK to Exif
« Reply #15 on: 17 / August / 2009, 12:26:01 »
Quote from: Sow.N.Reap on 12 / August / 2009, 22:40:20
Is there another manual somewhere besides the HTML docs in the Perl download & the website info? I find it very hard to read the documentation in the command window
Sorry, I don't know about that. I'm reading from command line, perldoc Image::ExifTool under linux... it's basically the same as the man page (which will mean nothing if you don't use unix so I'll shut up now) Can't you just dump the docs into a text file then open it in notepad back on the desktop?

Quote from: Sow.N.Reap on 12 / August / 2009, 22:40:20
Silly me was storm chasing to see if I could get some lightening pics.
Ah, well - in case you didn't catch anything: 07-26-09: Glad I'm indoors! (storm had multiple strikes and fatalities.)

Quote from: Sow.N.Reap on 12 / August / 2009, 22:40:20
Another dumb question ... are there significant differences between the Windows .exe  & the Perl version?
Yes and no. For just tagging, not really, but Perl is very powerful and if you wanted to manipulate tag data in complex ways, it would be much easier in Perl than calling the compiled module (which is basically a runtime encoded with the Perl script, much the way Auto CHDK to Exif is a script with a runtime embedded.)

Quote from: Sow.N.Reap on 12 / August / 2009, 22:40:20
I'm not sure whether to feel honored or sorry for you having to deal with me in order to have the longest post ever.   :)
Actually... get this: Because of you, exiftool is getting modified. :D

I was working on the next bit of the "lesson" and found shortcoming in exiftool, talked to Phil Harvey, he helped me get around it, but found a bug. He fixed that bug, and I re-tabulated my data and in the process of going it, he figured out the Canon Maker Notes decoding related to PowerShot cameras was flawed. So thanks to you, not only will the next "lesson" show how to uniquely extract the ISO/FNumber/ExposureTime that we want for CHDK-created images, we're getting more accurate decoding for the data inside the tags. :b How's cool is that? (you can follow the thread here if it interests you: CPAN::Forum Image-ExifTool: Basics - extracting sub-sub-folder maker notes?
Logged
Since we cannot know all that there is to be known about anything,
 we ought to know a little about everything. -- Blaise Pascal
*

Offline Sow.N.Reap

  • *
  • 23
  • Canon S5is
Re: NOW: Exiftool usage lessons .. WAS: Bug ?? in Auto CHDK to Exif
« Reply #16 on: 18 / August / 2009, 01:18:41 »
Quote from: Anaglyphic on 17 / August / 2009, 12:26:01
Can't you just dump the docs into a text file then open it in notepad back on the desktop?
I starting doing that but notepad messed with the formatting so start doing it in OpenOffice. It got to be tedious since I can't just 'select all' & paste. I scanned through it it & I think it's pretty much the same as on online.  It's not a big deal .. just felt like I needed more info.

Part of the problem was I read through everything but it didn't sink in until I started working with Exiftool.  Then after re-reading the parts that were relevant to what I was doing, it started making more sense.   

I'm making progress. I've got a couple argfiles set up & working. Just had a little problem with 1 command (?? not sure that terminology is correct) working with one of my argfiles. I was actually trying to get the 2 options into 1 argfile but couldn't get that to work. I finally put the option then the argfile on the command line & got it to work using the -execute option. I could just put both options on the command line but I'm going to add more options to the argfile.

Quote from: Sow.N.Reap on 12 / August / 2009, 22:40:20
Silly me was storm chasing to see if I could get some lightening pics.
Quote from: Anaglyphic on 17 / August / 2009, 12:26:01
Ah, well - in case you didn't catch anything: 07-26-09: Glad I'm indoors! (storm had multiple strikes and fatalities.)
The pic in your link didn't show up. For me the hard part about taking lightening pictures is staying safe. I have a very healthy respect for lightening.

I didn't get any pics that night. The storm zigged & I zagged so I missed it. I don't get the kind of storms they get in the midwest just little cells that popup & quickly go away. Makes it hard to get in the right spot for pics.

Quote from: Sow.N.Reap on 12 / August / 2009, 22:40:20
I'm not sure whether to feel honored or sorry for you having to deal with me in order to have the longest post ever.   :)
Quote from: Anaglyphic on 17 / August / 2009, 12:26:01
Actually... get this: Because of you, exiftool is getting modified. :D

I was working on the next bit of the "lesson" and found shortcoming in exiftool, talked to Phil Harvey, he helped me get around it, but found a bug. He fixed that bug, and I re-tabulated my data and in the process of going it, he figured out the Canon Maker Notes decoding related to PowerShot cameras was flawed. So thanks to you, not only will the next "lesson" show how to uniquely extract the ISO/FNumber/ExposureTime that we want for CHDK-created images, we're getting more accurate decoding for the data inside the tags. :b How's cool is that? (you can follow the thread here if it interests you: CPAN::Forum Image-ExifTool: Basics - extracting sub-sub-folder maker notes?

Well that really is cool!! Sometimes we newbies do come in handy. :) I'll have to check out that thread.
Logged
Canon S5is .. My Flickr page
*

Offline Anaglyphic

  • ***
  • 129
  • Anaglyphic lives!
Re: NOW: Exiftool usage lessons .. WAS: Bug ?? in Auto CHDK to Exif
« Reply #17 on: 18 / August / 2009, 07:19:51 »
weird, someone else said that. Must be a direct linking to images thing, probably have to disable referrers... try the album view vs. directly linking, I guess: http://bill-p.tripod.com/CHDKLightning/ It's one of the crazy wiggles ones like a mad scientists lab, not a fractal looking cloud-to-ground bolt.... and like most things, loses it's drama when you have to futz with it.

Here I'll just attach it.
Logged
Since we cannot know all that there is to be known about anything,
 we ought to know a little about everything. -- Blaise Pascal
*

Offline Sow.N.Reap

  • *
  • 23
  • Canon S5is
Re: NOW: Exiftool usage lessons .. WAS: Bug ?? in Auto CHDK to Exif
« Reply #18 on: 19 / August / 2009, 02:05:06 »
That's some wicked looking lightening! I like it when they branch out everywhere. If you get a chance check out Discovery channel's "Raging Planet - Lightening" show .. has some pretty interesting stuff.
Logged
Canon S5is .. My Flickr page
*

Offline Anaglyphic

  • ***
  • 129
  • Anaglyphic lives!
Re: NOW: Exiftool usage lessons .. WAS: Bug ?? in Auto CHDK to Exif
« Reply #19 on: 20 / August / 2009, 11:53:45 »
Yea, that storm was intense. I got ~1000 images from that line of thunderstorms, and ~20 "keepers" (you'll start getting picky when your lightning collection passes 20 gigs). It occurred almost exactly one year from the last really intense storm with fatalities here. BTW did you explore the tags in the image? Specifically some URIs you might be interested in; http://www.struckbylightning.org/ (tracker) and http://www.lightning-strike.org/ (survivor support group).

Back to the topic of tagging, check out http://www.sno.phy.queensu.ca/~phil/exiftool/history.html, specifically #3 in version 7.88... that's ours. :b So you should upgrade to the current release. (as of posting this, 7.89) Grab both the full distribution as well as the Windows executable. In the full dist you'll get a bunch of support files, including all the .html files for a local copy of documentation, and 5 or 6 argfiles for converting tags to<--->from other tag formats (like exif2xmp.args).

Quote from: Sow.N.Reap on 18 / August / 2009, 01:18:41
Just had a little problem with 1 command (?? not sure that terminology is correct) working with one of my argfiles. I was actually trying to get the 2 options into 1 argfile but couldn't get that to work. I finally put the option then the argfile on the command line & got it to work using the -execute option. I could just put both options on the command line but I'm going to add more options to the argfile.
It's very straight forward. One argument per line. Even if it's part of the a "whole" command, you need to break it at spaces in the command.

For instance:
exiftool -a -e -s -G:4:3:2:1:0 -d %d-%b-%y <filename or directory> in an argfile would look like:

-a
-e
-s
-G:4:3:2:1:0
-d
%d-%b-%y

Notice that even though -d %d-%b-%y go together as a single command, they are two arguments.
Is this concept clearly explained, or do I need a better example of individual arguments vs. whole commands?

(fyi that date format string makes dates appear in military style, eg. 20-Aug-09)
Logged
Since we cannot know all that there is to be known about anything,
 we ought to know a little about everything. -- Blaise Pascal
Pages: 1 [2] 3   Go Up
« previous next »
 

Related Topics