General workflow question (including managing/emptying cards) - Hello, I'm a NEWBIE - HELP!! (Newbies assistance, User Guides and thank you notes) - CHDK Forum supplierdeeply
« previous next »
Pages: [1] 2   Go Down

General workflow question (including managing/emptying cards)

  • 10 Replies
  • 3514 Views
*

Offline DavidB

  • *
  • 40
General workflow question (including managing/emptying cards)
« on: 12 / August / 2010, 00:25:51 »
Advertisements
I did play with ancient versions of this on an A610 (waaay before the days of DNG) but right now do not have a CHDK-able camera in-hand. I'm ready to drop a bit of cash on one, but need to work out a few things first. So please bear with me if I'm asking stupid questions because I haven't actually played with it yet.
But hopefully the questions won't be so stupid: I am a professional photographer and teacher (I'm even Adobe Certified) who shoots RAW with my DSLRs all the time.

This post is quite wordy, but hopefully you'll see the fundamental questions in bold as you read through.


I'm trying to picture how I can fit a CHDK camera into my normal workflow. For me this camera will be an additional tool in my pocket. My primary purpose for CHDK will be to get DNG files straight from camera. On the computer side I'll be generating a lens profile for the camera so that Lightroom/Camera Raw can auto-correct distortions and aberrations, and dual-illuminant DNG profiles so the colours are consistent with all of our other cameras. All of that's fine for me, but I'm just trying to get my head around the processes of cycling cards between the camera and the computer. I've read through the documentation but there are some grey areas.

I'll need to have autoboot on: when I pick up the camera to photograph something I won't have time to tweak things, it has to be ready to shoot quickly. I know I'll be tuning various parameters (e.g. DNG mode, OSD settings, etc) and I'll obviously want those to be preserved between boots.
Fundamental question: Is the configuration of CHDK stored on the card so that it is persistent across card and battery changes?

For my normal cameras, when I fill up cards they end up being backed up either to a Mac via a card reader or to a ND2700 with internal hard drive. When I know the data has been backed up securely to multiple drives I put the card back into the camera and reformat. Erasing files rather than formatting is something I have no interest in doing except as maybe a one-off and reformatting the next time around. I've seen too many cards become corrupted after multiple erase cycles to trust anything other than fresh formatting.
In many environments I do have access to my computer at least at the end of each day, but sometimes (e.g. when bicycling through foreign countries for weeks on end) a couple of ND2700s is all I've had with me.

So, how would I deal with the cards from this camera? I presume that all cards to be used with this camera need to be prepped with the right config (making sure they all have the appropriate bad pixel map, etc) and identified separately from all the other cards in my card carriers.

Why does the card apparently need to be a <= 2GB FAT16 partition for autoboot? The camera's firmware update can happen from FAT32 cards, so is this just a limitation of the autoboot function (DISKBOOT.BIN)? I have an assortment of SD/SDHC cards from 1GB to 8GB.

If I set up an SDHC card with a (2MB) partition and [the rest] I need to set the write-protect tab to preserve the autoboot function. Do I need to duplicate the CHDK files on both partitions? Can I use the camera's format function to wipe the main partition yet preserve the CHDK config? I'm guessing not.

Do I have to go through a special "reformat" procedure on the cards at the end of each day (making sure that I have enough cards to get me through the day) or is there a simpler way that doesn't require the computer?
I'm guessing that the manual procedure would at least involve reformatting the big partition in the camera and then copying over CHDK files again (assuming the partitioning was still in place). Or are we prepared to trust that OSX's newfs_msdos produces something that the camera will be happy with? With most cameras the safest generic answer has always been to format in-camera and just read from the card via a card reader.
Also this would imply that when travelling with a CHDK camera you need to either have a computer with you or have enough flash storage to last the entire trip without reformatting.
Fundamental question: Does emptying a card require the use of a computer?

Thanks for your help!
« Last Edit: 12 / August / 2010, 00:33:30 by DavidB »
Logged
*

Offline fe50

  • ******
  • 3105
  • IXUS50 & 860, SX10 Star WARs-Star RAWs
    • fe50
Re: General workflow question (including managing/emptying cards)
« Reply #1 on: 12 / August / 2010, 01:55:04 »
Hello & welcome, David !

Quote from: DavidB on 12 / August / 2010, 00:25:51
Fundamental question: Is the configuration of CHDK stored on the card so that it is persistent across card and battery changes?
Yes, CHDK use a config file for this, stored in the \CHDK folder on the SD card (on the "data" partition).

Quote
So, how would I deal with the cards from this camera? I presume that all cards to be used with this camera need to be prepped with the right config (making sure they all have the appropriate bad pixel map, etc) and identified separately from all the other cards in my card carriers.
You can copy all CHDK-related files, including the configuration & badpixel file (note: the badpixel file is camera dependend), to multiple cards.
When using CHDK on different cameras (model or firmware version) you'll need to separate the prepared cards since the CHDK binary files (diskboot.bin & ps.fi2) must fit to the camera model & firmware version; also the badpixel file is only valid for the camera it was created for.
Non-Powershot cameras, i guess also your DSLR's, should ignore the CHDK files...

Quote
Why does the card apparently need to be a <= 2GB FAT16 partition for autoboot? The camera's firmware update can happen from FAT32 cards, so is this just a limitation of the autoboot function (DISKBOOT.BIN)? I have an assortment of SD/SDHC cards from 1GB to 8GB.
The autoboot mechanism is part of the Canon firmware update mechanism, it is limited to the FAT16 file system; the autoboot works with FAT16-formatted partitions up to 4 (four) GB (on 4GB partitions a cluster size of 4k is used, works fine with all Powershot cameras, Mac / Windows / Linux computers).

Quote
Do I need to duplicate the CHDK files on both partitions?
The first (small, FAT12 or FAT16) partition is only used to start CHDK, once started CHDK runs from the 2nd partition. The 1st partition need at least the diskboot.bin file...

Quote
Can I use the camera's format function to wipe the main partition yet preserve the CHDK config?
No.
The Canon format command deletes all partitions & creates a new one.
The partitioning of the card gets lost, also the CHDK-bootable flag; the Canon format command creates one partition with the full capacity & formats this partition with FAT16 (upt to 2GB) or FAT32 (on cards with 4GB or more)...

Quote
Do I have to go through a special "reformat" procedure on the cards at the end of each day (making sure that I have enough cards to get me through the day) or is there a simpler way that doesn't require the computer?
I'm guessing that the manual procedure would at least involve reformatting the big partition in the camera and then copying over CHDK files again (assuming the partitioning was still in place).
To format (and also to make CHDK-bootable & copy the CHDK files) you'll need a computer (or something similar); CHDK can create a new multipartitioned card, but we can't format...

Quote
Fundamental question: Does emptying a card require the use of a computer?
Formatting a card: yes
Deleting files: no - you can delete files (of any filetype) with CHDK

IMO deleting files would be ok for some days or weeks, most people run their CHDK installation for months without reformatting the card without problems or significant speed issues due fragmentation...
Logged
*

Offline reyalp

  • ******
  • 11998
Re: General workflow question (including managing/emptying cards)
« Reply #2 on: 12 / August / 2010, 01:59:46 »
I'll answer the bits I know
Quote from: DavidB on 12 / August / 2010, 00:25:51
Is the configuration of CHDK stored on the card so that it is persistent across card and battery changes?
The CHDK configuration (along with CHDK itself) is stored on the card, so it will persist across battery changes, but obviously not card changes.
Quote
I presume that all cards to be used with this camera need to be prepped with the right config (making sure they all have the appropriate bad pixel map, etc) and identified separately from all the other cards in my card carriers.

Yes.
Quote
so is this just a limitation of the autoboot function (DISKBOOT.BIN)? I have an assortment of SD/SDHC cards from 1GB to 8GB.
Yes. DISKBOOT.BIN appears to exist to support canon development or service, and they don't appear to care about using large cards in this application.
Quote
If I set up an SDHC card with a (2MB) partition and [the rest] I need to set the write-protect tab to preserve the autoboot function. Do I need to duplicate the CHDK files on both partitions?
DISKBOOT.BIN needs to go on the small partition. Everything else needs to go on the large partition.  In practice, I usually put PS.FI2 on both, so I can manual boot from either.
Quote
Can I use the camera's format function to wipe the main partition yet preserve the CHDK config? I'm guessing not.
No, the canon format will wipe the whole card (been a while since I tested this but I'm 99.9% sure)
Quote
Do I have to go through a special "reformat" procedure on the cards at the end of each day (making sure that I have enough cards to get me through the day) or is there a simpler way that doesn't require the computer?
There is no way to format the card and re-install CHDK without a computer or equivalent device (some smartphone/pda devices should be able to do it.) If you have a device that can completely clone USB mass storage disks (including the partition table) you could keep a master and clone it, but I'm not aware of anything like that which is pocket size.
Quote
I'm guessing that the manual procedure would at least involve reformatting the big partition in the camera and then copying over CHDK files again (assuming the partitioning was still in place). Or are we prepared to trust that OSX's newfs_msdos produces something that the camera will be happy with?
Canon cameras do not appear to have trouble PC formatted cards. Every CHDK multipartition install is formatted in a computer. You could automate the whole process with a shell script.
Quote
Also this would imply that when travelling with a CHDK camera you need to either have a computer with you or have enough flash storage to last the entire trip without reformatting.
Or you can erase the images without reformatting.
Quote
Fundamental question: Does emptying a card require the use of a computer?
If you insist on reformatting.

edit:
fe50 is faster than me  >:(
« Last Edit: 12 / August / 2010, 02:01:17 by reyalp »
Logged
Don't forget what the H stands for.
*

Offline DavidB

  • *
  • 40
Re: General workflow question (including managing/emptying cards)
« Reply #3 on: 12 / August / 2010, 02:33:38 »
Thanks guys!

At the moment I'm only expecting to be using one CHDK camera, but obviously if this expands I will have to manage the versions for different cameras. Actually, if I had multiple cameras of the same model the only thing I'd had to do to swap a card over would be to create a new bad pixel map in-camera, right?
ACR would default to using the same DNG profile and lens profiles for all those cameras (the PowerShots don't have the concept of a serial number) but that would be OK.

My DSLRs currently use CF cards, it's just my audio recorder and my infrared camera (a Lumix G1) that use SD. I should have no trouble setting some cards aside for the CHDK camera.

Sounds like I'll be able to manage things via a reformat script on my MacBook to set up the cards at the end of each day.



Hmmm..... thought for future development....
If the DNG-saving code can insert a programmable serial number then people could use multiple cameras of the same model and have one modified for IR with a 695nm filter, one for IR with an 830nm filter, one of visible light, etc.
Adobe Camera Raw can have different defaults set for each camera as long as it can see a serial number. Unfortunately no PowerShots put a serial # in their official JPG/CRW/CR2 files.
Logged
*

Offline reyalp

  • ******
  • 11998
Re: General workflow question (including managing/emptying cards)
« Reply #4 on: 12 / August / 2010, 03:09:33 »
Quote from: DavidB on 12 / August / 2010, 02:33:38
Actually, if I had multiple cameras of the same model the only thing I'd had to do to swap a card over would be to create a new bad pixel map in-camera, right?
If you can do badpixel removal in your workflow, you could use a dummy badpixel file on all the cameras.

Quote
If the DNG-saving code can insert a programmable serial number then people could use multiple cameras of the same model and have one modified for IR with a 695nm filter, one for IR with an 830nm filter, one of visible light, etc.
CHDK could fill something in the serial number field, no problem. The problem is all CHDK data is stored on the card, so it would go with the card rather than the camera, unless there is some unique identifier available to the code. You could probably save it in one of the onboard flash parameters (I believe there's one for owners name or something like that which can be set with canons software)

Logged
Don't forget what the H stands for.
*

Offline zeno

  • *****
  • 885
Re: General workflow question (including managing/emptying cards)
« Reply #5 on: 12 / August / 2010, 04:12:36 »
Since you have a Mac, you could use my SDMInst app to prepare cards - it will partition, format and copy a set of files for you. It works just as well with CHDK as it does with SDM. Once you have a folder set up on your Mac (copied from a card that works the way you want), it makes it easy to 'clone' that card.

See http://www.zenoshrdlu.com/clickpansdm/sdminst.html
Logged
A570, S100, Ixus 127
Author of ASSIST, STICK, WASP, ACID, SDMInst, LICKS, WICKS, MacBoot, UBDB, CFGEdit
*

Offline fe50

  • ******
  • 3105
  • IXUS50 & 860, SX10 Star WARs-Star RAWs
    • fe50
Re: General workflow question (including managing/emptying cards)
« Reply #6 on: 12 / August / 2010, 04:49:57 »
Quote from: reyalp on 12 / August / 2010, 03:09:33
Quote from: DavidB on 12 / August / 2010, 02:33:38
If the DNG-saving code can insert a programmable serial number then people could use multiple cameras of the same model and have one modified for IR with a 695nm filter, one for IR with an 830nm filter, one of visible light, etc.
CHDK could fill something in the serial number field, no problem. The problem is all CHDK data is stored on the card, so it would go with the card rather than the camera, unless there is some unique identifier available to the code. You could probably save it in one of the onboard flash parameters (I believe there's one for owners name or something like that which can be set with canons software)

Field "OwnerName" in the MakerNotes, can be set with the Canon software (ZoomBrowser / CameraWindow); on a Mac, this is possible with Canon's CameraWindow AIAIK...

Edit:
@reyalP: in PTP mode you'll get a "serial number" - is this a real, unique number ?
--> http://www.wirklemms.de/chdk/forum/viewtopic.php?p=18454#18454
« Last Edit: 12 / August / 2010, 05:23:50 by fe50 »
Logged
*

Offline DavidB

  • *
  • 40
Re: General workflow question (including managing/emptying cards)
« Reply #7 on: 17 / August / 2010, 11:51:30 »
Thanks folks! For now I've picked up an A470 to play with, and things are falling into place. I've set up a shell script to format a card with a bootable FAT12 boot partition and [the rest] as FAT32 and copy the appropriate files in from a template. Having automated that (and knowing I have to add the CCHDK.CFG to the template when I change it) it's all rather painless now. Only hassle is remembering to unlock the card when I want to reformat it!

Next I'll be calibrating the colours with a new dual-illuminant DNG profile and playing with it in brighter conditions (only got it late this afternoon). So far so good!
Logged
*

Offline reyalp

  • ******
  • 11998
Re: General workflow question (including managing/emptying cards)
« Reply #8 on: 18 / August / 2010, 01:53:30 »
Quote from: DavidB on 17 / August / 2010, 11:51:30
Having automated that (and knowing I have to add the CCHDK.CFG to the template when I change it) it's all rather painless now. Only hassle is remembering to unlock the card when I want to reformat it!
Many cheap card readers ignore the lock, or can be modified to do so. The lock position is just sensed by a switch in the reader (or camera), and reported to software to enforce.
Logged
Don't forget what the H stands for.
*

Offline DavidB

  • *
  • 40
serial number in metadata
« Reply #9 on: 19 / August / 2010, 01:58:16 »
Quote from: reyalp on 12 / August / 2010, 03:09:33
Quote from: DavidB on 12 / August / 2010, 02:33:38
Actually, if I had multiple cameras of the same model the only thing I'd had to do to swap a card over would be to create a new bad pixel map in-camera, right?
If you can do badpixel removal in your workflow, you could use a dummy badpixel file on all the cameras.
My workflow is unlikely to change from taking CR2/RW2/DNG files from the cameras and feeding these directly into Lightroom/ACR.


Quote
Quote
If the DNG-saving code can insert a programmable serial number then people could use multiple cameras of the same model and have one modified for IR with a 695nm filter, one for IR with an 830nm filter, one of visible light, etc.
CHDK could fill something in the serial number field, no problem. The problem is all CHDK data is stored on the card, so it would go with the card rather than the camera, unless there is some unique identifier available to the code. You could probably save it in one of the onboard flash parameters (I believe there's one for owners name or something like that which can be set with canons software)
If I accept the premise that I need to have each card tied down to a particular camera (because not all cameras will be the same model [and thus have different CHDK builds] although some may be, as well as the above issue of badpixel maps) then having a serial-number config file that goes along with the badpixel.bin as part of the per-camera config seems quite reasonable. The absence of that config file could just result in no serial number, so no new menu items would be needed.

It might be possible to overload the serial number into a suffix to the OwnerName field (which if I remember my CRW/MakerNote decoding correctly is a null-terminated string and thus maybe the "serial number" could be inserted at the end, although the Canon utility software will presumably not allow this over USB) but care should be taken to not interfere with normal use of this field (e.g. in ACR it provides a default value for the IPTC Creator Name field).
Logged
Pages: [1] 2   Go Up
« previous next »
 

Related Topics