ISO Override feature tested on A570 IS - Creative Uses of CHDK - CHDK Forum

ISO Override feature tested on A570 IS

  • 5 Replies
  • 8223 Views
ISO Override feature tested on A570 IS
« on: 25 / March / 2009, 10:44:34 »
Advertisements
First, a little background.

A couple of weeks ago I discovered the Clarkvision.com site for the second time and, this time, I studied his article on measuring Sensor Noise, Dynamic Range, and Full Well Capacities for the Canon 1D in detail.

Clark uses an incandescant light globe shining on a sheet of copy paper to generate an even light source for measuring the performance of cameras and their sensors.
http://www.clarkvision.com/imagedetail/evaluation-1d2/index.html

I was keen to use the CHDK ISO override feature to get a slower ISO than stock on my A570 IS and it struck me that Clark's was a method that I could use to try and calibrate the ISO override feature for my camera. I highly recommend Clark's site, check it out.


The Test Setup.


I clipped 3 pieces of copy paper to a book so that I could stand them up vertical.
I warmed up an incandescent lamp for >30 minutes and then set it up about 2' away from the copy paper to start with.
I set up the camera on a tripod focused on the centre of the sheet of paper.
- Auto-everything off in the Canon menu
- Manual Mode
- Incandescant WB
- Colours Off
- Aperture f/8
- Manual focus zoomed in to the maximum (23.2mm)
- Exposure 1/60 sec. The AC power in Oz is 60Hz so I wanted to make sure I always captured the light from one full AC cycle in case the lamp brightness varies during the cycle.
- CHDK RAW enabled. I kept the JPEG files as well so that I had a record of the settings for each shot and to cut down on the typing I had to do to setup the charts in the spreadsheet later on.
- CHDK Overrride "Off"

I set the ISO to1600, which is an option in Manual Mode on the A570, and adjusted the distance of the lamp from the paper until the histogram on half-press showed no overexposure.

Before I took any pictures I also turned off all the fluorescent lights in the room.

After the setup, all I change for the test photos is either the ISO setting or the ISO override value.


The Test Photos


I took a first set of photos with the Canon ISO settings to compare against the shots taken later using ISO Overrride and also to check how reproducible the method is using an incandescent lamp + copy paper as my light source. I took a photo at all the Canon ISO settings (50, 80, 100, 200, 400, 800, 1600) and repeated this 4 times.

I took a second set of photos with the ISO Override enabled at 10x (2, 3, 4, 5, 6, 7, 8, 9) and 100x (1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15). I only repeated this 3 times because my SD card filled up.

Finally, I disabled the ISO Override and I took a 5th set of photos at all the Canon ISO settings (50, 80, 100, 200, 400, 800, 1600) again so I could compare these results at the end of the session with the ones at the start of the session.


The Processing/Analysis


Convert the raw file to a TIFF using "dcraw -D -4 -j -t 0 -T ". The object here is to get the raw values without any adjustments or conversions.
-D   Shows the totally raw data with no interpolation and no colour scaling.
-4   Write 16-bit linear samples (fixed white level, no gamma).
-t 0   By default, dcraw applies the flip specified by the camera. -t 0 disables all flipping.
-j   this option guarantees that each output pixel corresponds to one raw pixel.
-T   Write TIFF

Get the mean value for a 200x200 pixel selection at the centre of each picture. Clark uses Photoshop to do this step, which I don't own. A google search for some free "photo mean std deviation" software quickly found ImageJ and, not only could I get the min, max, mean and std deviation for a selection using a menu option but I could write a script to open the image, get the image height & width, calculate the position of 200x200 pixels at the centre, select that range, call a function to get the mean for the current selection, open the matching JPEG image and get the EXIF data (using someone else's plugin), write all this info to a log window in a tab delimited format (that I could later save to disk and open in OpenOffice Calc) and run the script for all the pictures in a directory. If you haven't seen ImageJ yet it is worth checking out.


The result is good news and bad.

The good news is that this test gave a pretty unambigous picture of what possible range of ISO values my camera is capable of. It also showed that the ISO override feature allows you to choose from a bigger set of ISO values. eg 100, 125, 160, 320, 500, 640, etc instead of just the stock values 80, 100, 200, 400 and 800.

The bad news is that the lowest ISO I achieved using the ISO Override feature on my A570 IS was equivalent to the stock ISO80 setting and the highest was only fractionally higher than the stock ISO800 setting. This highest effective ISO800 also applies to the stock ISO1600 that you can choose under Manual mode.

I've included charts of my results as attachments to this post so you'll need to be logged in to see what I'm talking about.

Stock ISO Chart.JPG plots the mean of the raw data values plotted against the stock ISO settings. It shows two things:
- That the incandescent lamp did provide a reliable, reproducible light source for testing, at least at the 1/60 sec exposure I used.
- That ISO1600 is exactly the same as the ISO800 on my A570 IS

ISO Override Chart 1.JPG and ISO Override Chart 2.JPG are the results for the ISO override settings.

ISO Override Chart 1.JPG plots the mean raw data values plotted against the ISO override value. It show:
- That the lowest ISO bottoms out at a mean raw data value of around 62. This is exactly the same value that you get with the stock ISO80
- That the highest raw data value is around 360. This is only slightly higher than the stock ISO800 which gives a mean raw data value around 330.
- That for an ISO override which gives a value of ISO80 in the JPEG EXIF the the raw data value of the image is a good match with the stock ISO80 setting.
- That for an ISO override which gives a value of ISO800 in the JPEG EXIF the the mean raw data value of the image is slightly higher than the stock ISO800 setting. The stock ISO800 gives a mean raw data value around 325 - 330 while an ISO override that gave an ISO800 in the JPEG EXIF had a mean raw data value of 330 - 350.

ISO Override Chart 2.JPG plots the mean of the raw data values plotted against the ISO that is recorded in the JPEG EXIF data. This is easier to compare with Stock ISO Chart.JPG because some ISO override values I used produce the same ISO in the EXIF data. You can see this in ISO Override Chart 2.JPG where some ISO values have 6 data points instead of 3.

The mapping between the ISO override values I used and the ISO the camera recorded in the JPEG EXIF data was:

ISO           ISO
override     in
value         EXIF
10x   2          32
        3          50
        4          64
        5          80
        6         100
        7         100
        8         125
        9         125
100x 1         160
        2         320
        3         500
        4         640
        5         800
        6        1000
        7        1000
        8        1250
        9        1250
       10       1600
       11       1600
       12       2000
       13       2000
       14       2000
       15       2500
« Last Edit: 27 / March / 2009, 07:16:36 by flenser »

Re: ISO Override feature tested on A570 IS
« Reply #1 on: 27 / March / 2009, 09:37:41 »
Wow! What an indepth look at the ISO override function. This is something I have wanted to try for a long time with my a710is and have only done so with limited success. Perhaps I won't bother now! Have you thought of contacting DPreview for a job!

Thanks for sharing all your hardwork, I find it interesting and conclusive!

Nick
a710is

*

Offline fudgey

  • *****
  • 1705
  • a570is
Re: ISO Override feature tested on A570 IS
« Reply #2 on: 28 / March / 2009, 08:12:15 »
It's always nice to see some careful testing done  :). A couple of comments:

1) What the camera calls Tv 1/60 may not be exactly 1/60 second exposure and may slightly differ from your mains frequency (likely very close to 60 Hz at all times). To see if that matters you could take a series of test shots with identical settings and see if there's significant variation from shot to shot.

A DC powered lamp or a long exposure would be a brute force solution of course.

2) Your a570 has an ISO 50 preset? Or was that a typo? Mine only goes down to 80 (sub 100e). The ISO range of my camera is 80 to 800, artificially extended up to 1600 for JPEG output, just as you've found with yours.

3) You can see the real ISO values in EXIF Canon MakerNotes section, maybe even CHDK MISC OSD when enabled, and definitely via CHDK scripts from their respective propcases. As you've found, the real ISO values are often slightly different from the common "Market" values visible in Canon menus.

Re: ISO Override feature tested on A570 IS
« Reply #3 on: 29 / March / 2009, 09:52:12 »
1) What the camera calls Tv 1/60 may not be exactly 1/60 second exposure and may slightly differ from your mains frequency (likely very close to 60 Hz at all times). To see if that matters you could take a series of test shots with identical settings and see if there's significant variation from shot to shot.

That's exactly what I did. The Stock ISO Chart.JPG is the same set of Stock ISO settings tested 5 times each with all other settings kept identical. The other two charts are the same set of ISO Override Values tested 3 times each with all other settings kept identical and with all the other settings identical to the ones that were used for the Stock ISO tests.

The small amount of variation between each of these test sets indicates to me that the variation in the lamp brightness across the AC cycle is not very great. More on this later.

A DC powered lamp or a long exposure would be a brute force solution of course.

A well regulated high power 12V power supply would definitely be better, but now I think that the AC lamp brightness is actually steady enough for this type of testing.

I've done a similar set of testing for shutter speed override (I'll post that next). If the AC lamp brightness varied significantly I would have expected to see it clearly for fast shutter speeds as wildly varying mean values measured for the same shutter speed, but I did not see anything like this.

2) Your a570 has an ISO 50 preset? Or was that a typo? Mine only goes down to 80 (sub 100e). The ISO range of my camera is 80 to 800, artificially extended up to 1600 for JPEG output, just as you've found with yours.

I may have confused people by including this table. When you shoot with the ISO overridden the Canon software puts an ISO value into JPG files EXIF data and I captured these in my spreadsheet to see what the correlation was, if any between it and the ISO Override Value I used (and thats why these values go much higher and much lower than the stock values). If I found override values higher & lower than stock worked then I was hoping to be able to identify what ISO Override Value had been used from the ISO value in the JPG EXIF data.

3) You can see the real ISO values in EXIF Canon MakerNotes section, maybe even CHDK MISC OSD when enabled, and definitely via CHDK scripts from their respective propcases. As you've found, the real ISO values are often slightly different from the common "Market" values visible in Canon menus.

I wasn't interested in the actual ISO values. I wanted to measure what the ISO Override Values gave as raw data values and then compare that with the raw data values that the stock Canon ISOs gave, when shot with all other settings the same. eg. if a low ISO Override Value showed a significantly lower raw data value then the Canon stock ISO80 I could infer it was causing the camera to operate at a lower ISO than the lowest stock setting of 80.

I did have second thoughts about the low values for the mean raw data values at the lowest ISO Override Values. I wanted to make sure that the low values for mean raw data value (around 62) wasn't some sort of "floor" for the measurements  at the settings I was using.

ISO Override Chart 3.jpg is a repeat of the tests but only for ISO Override Values between 2 and 100 and with lamp moved in as close as I could to get brighter exposures. Manual Mode, F8.0, 1/60s, ISO100, Tugsten WB, Manual Focus at infinity, Lens zoomed in to max (23.2mm), Flash Off, RAW+Small JPEG, Auto-everything Off in the Canon menus, IS shoot-only.
It shows the same levelling off of the mean raw data value at ISO Override Values of 50 and below so that leveling out is not due to the raw data values being low.

Then I continued on and took photos of ISO Override Values 2,4,6,8 while only changing the shutter speed for speeds of 1/125, 1/250, 1/500, 1/1000, 1/2000. This is ISO Override Chart 4.jpg, the values on the legend are the shutter speeds. Once the ISO Override Value is 50 or lower the sensor is operating at its lowest possible ISO value.

Greg.
« Last Edit: 29 / March / 2009, 10:11:37 by flenser »

*

Offline fudgey

  • *****
  • 1705
  • a570is
Re: ISO Override feature tested on A570 IS
« Reply #4 on: 30 / March / 2009, 17:27:09 »
So we're getting ISO50 after all? Hm.

One thing you may not be aware of: if you want to use JPEGs for actual math in a future test for one reason or another, you should set custom WB, because it's been found Canon firmware likes to fine tune WB with any other setting. AUTO WB is just a whole lot more automatic compared to the others.

As for EXIF, the override settings basically go to MakerNotes, converted to ISO by the camera (although ISO override in CHDK menu may end up rounding differently as it's put into effect using a propcase that handles APEX Sv values, not ISO values directly). The normal EXIF tags can't be trusted when using overrides. When using manual ISO, the normal EXIF tag can't be fully trusted either if you wish to compare results with those from overrided ISO, because EXIF ISO is a user-friendly rounded up 'market' value.

Try `exiftool -EXIF:ISO -MakerNotes:ISO` for a batch of JPEGs to find the relation for each image. Note that there are more ISO fields in EXIF than those two and that in AUTO mode it may behave a bit differently (it's been quite while since I've touched those so I may remember something incorrectly). From some old images, to me it looks like manual ISO80 is actually ISO82.

Re: ISO Override feature tested on A570 IS
« Reply #5 on: 31 / March / 2009, 09:45:20 »
Fudgey,

Sorry mate, I didn't get an ISO50 setting for the A570 IS.

The three charts Stock ISO Chart.JPG, ISO Override Chart 1.JPG and ISO Override Chart 2.JPG were all shot in the same session with all the same settings (shutter speed, aperture, etc) except for the ISO. I did this so that I would be able to compare the raw pixel levels I got for the stock ISO settings with the raw pixel levels that I got with the ISO Override settings.

The raw pixel level in chart Stock ISO Chart.JPG for the lowest stock setting of ISO80 was in the range 61-63.

The lowest raw pixel level in the charts ISO Override Chart 1.JPG and ISO Override Chart 2.JPG, representing the lowest ISO I got using the ISO Override feature, was exactly the same 61 - 63 for all ISO Override values <= 50.

So the lowest ISO value I got using the ISO Override was exactly the same as the stock Canon ISO80 setting.

Greg

 

Related Topics


SimplePortal © 2008-2014, SimplePortal