Real vs market ISO

waterwingz

12138

« on: 05 / July / 2013, 23:23:07 »

Advertisements

Given all the posts about this in the last seven years, I'm not sure I want to go here but ..

A quick look at the menu strings tells me that ISO values as an input are only used three places. ISO override and ISO bracketing expect "real" ISO values. Custom Auto ISO requires "Market" ISO. This appears to have been the case for a long time now : http://chdk.wikia.com/wiki/CHDK_firmware_usage/AllBest#Show_.27Real.27_ISO

We get a lot of complaints about real vs market ISO and quite frankly, while I understand the APEX calculation reasons, its still a PITA to have to enter values that will appear different in my image EXIF's. I don't think anyone would argue that.

So can we break with tradition and make all ISO values "market" values for overrides & bracketing? And then convert to "real" when they are used. (I'm assuming that scripts will continue to work with "real" ISO for APEX96 calculations - they have access to both values in any case).

As I understand it, the issue is the lack of a precise relationship between market and real iso that works for all cameras? Could we handle this by extending the iso_table[] in each cam's shooting.c to include the market values for each entry as well as the real value? Then converting market values to the internal real values is just a table lookup & interpolation exercise. We might have to set some approximate values for cameras that are not calibrated. Or maybe we could add a startup function that prompts the user to be allowed to calibrate ISO and then goes through the process of filling the table by setting real iso values and reading back the associated market values.

This is a bit of work but would correct one of the remaining UI bits of stress in CHDK.

Thoughts ?

Logged

Ported : A1200 SD940 G10 Powershot N G16

reyalp

14079

Re: Real vs market ISO

« Reply #1 on: 05 / July / 2013, 23:36:10 »

Quote from: waterwingz on 05 / July / 2013, 23:23:07

So can we break with tradition and make all ISO values "market" values for overrides & bracketing? And then convert to "real" when they are used.

I'm fine with this if there is a reliable way to do which we can implement without excessive effort. However if it requires touching every camera, it might be better to punt it to 1.3.

Quote

Could we handle this by extending the iso_table[] in each cam's shooting.c to include the market values for each entry as well as the real value?

Unless we can find some automated way to extract it from the firmware, it would take us a very long time to get all the cameras filled in, and it's not clear what we would do with the ones that weren't filled in yet.

Quote

Or maybe we could add a startup function that prompts the user to be allowed to calibrate ISO and then goes through the process of filling the table by setting real iso values and reading back the associated market values.

I don't think this is a good solution. What happens if they haven't filled it in yet?

Quote

Thoughts ?

There must be functions or lookup tables in the firmware that handle the conversion. If we can automate finding them, that's probably the best option.

I completely agree that overrides etc should be entered as the same (market) values that appear in the canon UI.

Logged

Don't forget what the H stands for.

waterwingz

12138

Re: Real vs market ISO

« Reply #2 on: 05 / July / 2013, 23:49:50 »

Quote from: reyalp on 05 / July / 2013, 23:36:10

Quote from: waterwingz on 05 / July / 2013, 23:23:07
So can we break with tradition and make all ISO values "market" values for overrides & bracketing? And then convert to "real" when they are used.
I'm fine with this if there is a reliable way to do which we can implement without excessive effort. However if it requires touching every camera, it might be better to punt it to 1.3.

The shooting_recalc_conf_autoiso_values() code in core/autoiso.c converts market to real values. It just reads current market & real values from propcase values and calculates a slope value ( offset is assumed to be zero - i.e ISO 0 is the same for real & market values).

It seems to work pretty well on my G10 over a wide range of parameters. Haven't tried it on my other cams yet but this functionality for AutoISO has been in CHDK for a quite a while. What happens if we just use it for ISO override and ISO bracketing? I'm thinking that even if its off by a bit, it will still look "right" to the end user.

« Last Edit: 05 / July / 2013, 23:52:06 by waterwingz »

Logged

Ported : A1200 SD940 G10 Powershot N G16

reyalp

14079

Re: Real vs market ISO

« Reply #3 on: 06 / July / 2013, 00:48:58 »

Quote from: waterwingz on 05 / July / 2013, 23:49:50

The shooting_recalc_conf_autoiso_values() code in core/autoiso.c converts market to real values. It just reads current market & real values from propcase values and calculates a slope value ( offset is assumed to be zero - i.e ISO 0 is the same for real & market values).

Digging into those functions, I don't have much confidence it works reliably all the time:

Code: [Select]

short shooting_get_base_sv96()
{
    short dsv,sv;
    if (shooting_get_canon_iso_mode()<50)
    {
        get_property_case(PROPCASE_DELTA_SV, &dsv, sizeof(dsv));
        get_property_case(PROPCASE_SV, &sv, sizeof(sv));
        sv96_base = (sv-dsv);
    }
    return sv96_base;
}

So sv96_base may or may not be initialized?

shooting_get_canon_iso_mode() will be 0 in canon auto ISO, otherwise selected iso value (i.e. >= 50)

Code: [Select]

short shooting_get_iso_base()
{
    sv96_base = shooting_get_base_sv96();
    if (sv96_base != 0)
        return shooting_get_iso_from_sv96(sv96_base);
    return 0;
}

iso base may be 0?

Code: [Select]

short shooting_get_iso_market_base()
{
    if (iso_market_base==0)
    {
        if (iso_table[1-iso_table[0].id].prop_id == 50) iso_market_base=50;
        else iso_market_base=CAM_MARKET_ISO_BASE;
    }
    return iso_market_base;
}

Huh ?

I'd much rather have to work in inconvenient values than have the values I enter be interpreted unpredictably.

« Last Edit: 06 / July / 2013, 00:51:17 by reyalp »

Logged

Don't forget what the H stands for.

waterwingz

12138

Re: Real vs market ISO

« Reply #4 on: 06 / July / 2013, 10:16:21 »

Quote from: reyalp on 06 / July / 2013, 00:48:58

So sv96_base may or may not be initialized?

It's a static short defined at the start of the file and initialized to zero.

Quote

iso base may be 0?

Quote

Huh ?

I started out assuming this was put in by tsvstar but the code appear to go back to the Allbest/grand/EWAVR builds circa rev 243-246. It been in there a while and used in Custom Auto ISO ever since then.

tsvstart appears to check in shooting_recalc_conf_autoiso_values() for goofy returned values from these functions.

Quote

I'd much rather have to work in inconvenient values than have the values I enter be interpreted unpredictably.

If I understand this correctly, what they are trying to do here is capture a [market:real] pair from the camera propcases, and thus they need to be sure that a valid pair exists. Once they have that, the code is written to not try again.

So there may be better ways to do this. Perhaps an expicit user request to allow a "fake shoot" the first time an ISO value is entered from the GUI. Store the result on the SD card, read that back at startup, and you are good to go. Or if we don't want to get fancy, add a "ISO scaling factor" menu entry to the CHDK Settings Menu and default it to "80/50" (which seems to be a common value people post about).

My real question is whether the resulting straight line approximation is "good enough"? I guess I have four cams where a simple script could set a range of real ISO values, half shoot, read back the market value and print to a file. I could even post it here (or maybe its time for a new thread on this subject) to collect more data.

This is starting to feel like a 1.3.0 topic if we are not willing to take the time to look at it now. To me, its kind of a big deal and I think from your comments that you agree. It would be a nice touch to add to 1.2.0 from the perspective of our "user base" if we delay to do so. Besides, I'm only half way through the first draft of the 1.2.0 User Manual conversion. And once that's done, some edits to add brief rational behind the functionality are probably in order.

« Last Edit: 06 / July / 2013, 10:22:53 by waterwingz »

Logged

Ported : A1200 SD940 G10 Powershot N G16

reyalp

14079

Re: Real vs market ISO

« Reply #5 on: 06 / July / 2013, 16:29:13 »

Quote from: waterwingz on 06 / July / 2013, 10:16:21

If I understand this correctly, what they are trying to do here is capture a [market:real] pair from the camera propcases, and thus they need to be sure that a valid pair exists. Once they have that, the code is written to not try again.

Right, but my question is, what happens if it doesn't get one? Then what does your override/auto-iso etc do?

I did some experimenting last night and the various propcases do different things depending whether you have manual or automatic ISO set in the canon firmware. ISO mode updates immediately (and can be set outside of half shoot) while the others (mostly?) update on half press. I also found some previously undocumented ISO related propcases.... I'll try to summarize the results in a bit.

Quote

So there may be better ways to do this. Perhaps an expicit user request to allow a "fake shoot" the first time an ISO value is entered from the GUI. Store the result on the SD card, read that back at startup, and you are good to go.

I don't really like this, it seems complicated and hassles the user. If the calibration also only works with certain canon settings that would make it worse.

Quote

Or if we don't want to get fancy, add a "ISO scaling factor" menu entry to the CHDK Settings Menu and default it to "80/50" (which seems to be a common value people post about).

Hmm, having a default and allowing overriding it in platform code might be good enough. New ports can actually measure, existing ones that we can't find someone to calibrate will have a fairly reasonable default.

This would be a lot simpler than the current mess in the code, and still be reasonably close.

Really the difference between say ISO 80 and ISO 85 isn't going to make a huge real world difference.

Quote

I guess I have four cams where a simple script could set a range of real ISO values, half shoot, read back the market value and print to a file. I could even post it here (or maybe its time for a new thread on this subject) to collect more data.

In my experience, the lowest value is quite different, and all the rest have exactly the offset (for a given camera). See http://chdk.setepontos.com/index.php?topic=8613.msg90280#msg90280

Quote

This is starting to feel like a 1.3.0 topic if we are not willing to take the time to look at it now. To me, its kind of a big deal and I think from your comments that you agree. It would be a nice touch to add to 1.2.0 from the perspective of our "user base" if we delay to do so.

I'm a bit torn on it too. It would be really nice to get that mess sorted out, but it seems like kind of a big change to do right before release. But we don't have any actual deadline, so I'm tempted to do it now if we find a decent solution.

Logged

Don't forget what the H stands for.

waterwingz

12138

Re: Real vs market ISO

« Reply #6 on: 06 / July / 2013, 19:49:07 »

Quote from: reyalp on 06 / July / 2013, 16:29:13

Hmm, having a default and allowing overriding it in platform code might be good enough. New ports can actually measure, existing ones that we can't find someone to calibrate will have a fairly reasonable default.

Even better, if the default was 1:1 then you would be defaulting to "real" ISO. Exactly what we have today.

Quote

Really the difference between say ISO 80 and ISO 85 isn't going to make a huge real world difference.

Especially as Canon seems to round vallues to the nearest default when it picks the value to go in the EXIF and OSD.

Quote

In my experience, the lowest value is quite different, and all the rest have exactly the offset (for a given camera). See http://chdk.setepontos.com/index.php?topic=8613.msg90280#msg90280

I'll take a look at my cameras.

Quote

I'm a bit torn on it too. It would be really nice to get that mess sorted out, but it seems like kind of a big change to do right before release. But we don't have any actual deadline, so I'm tempted to do it now if we find a decent solution.

If we leave this to 1.3.0, I'm guessing it will tempt "us" to push for an early release of 1.4.0 to get this out onces its solved.

Logged

Ported : A1200 SD940 G10 Powershot N G16

reyalp

14079

Re: Real vs market ISO

« Reply #7 on: 06 / July / 2013, 20:08:04 »

Quote from: waterwingz on 06 / July / 2013, 19:49:07

Even better, if the default was 1:1 then you would be defaulting to "real" ISO. Exactly what we have today.

I don't think this is good, it would get us back to randomly having different behavior on different models.

The normal offset seems to be about 2/3 of a stop, which is pretty noticeable. Assuming they all go in the same direction (market ISO > real ISO) we should be able to make the standard approximation cut it down to something fairly insignificant.

Logged

Don't forget what the H stands for.

waterwingz

12138

Re: Real vs market ISO

« Reply #8 on: 06 / July / 2013, 21:12:24 »

Quote from: reyalp on 06 / July / 2013, 20:08:04

Quote from: waterwingz on 06 / July / 2013, 19:49:07
Even better, if the default was 1:1 then you would be defaulting to "real" ISO. Exactly what we have today.
I don't think this is good, it would get us back to randomly having different behavior on different models.

I agree its a "half assed" solution but it would not have different behavior on different models. All models would ship using "real" ISO only. The end user can tweak to use "market" ISO for input by adjusting the ratio (or a couple of discrete ratios) if he/she chooses.

Quote

The normal offset seems to be about 2/3 of a stop, which is pretty noticeable. Assuming they all go in the same direction (market ISO > real ISO) we should be able to make the standard approximation cut it down to something fairly insignificant.

Maybe a two or three breakpoint table ?

Logged

Ported : A1200 SD940 G10 Powershot N G16

reyalp

14079

Re: Real vs market ISO

« Reply #9 on: 06 / July / 2013, 21:55:06 »

Quote from: waterwingz on 06 / July / 2013, 21:12:24

I agree its a "half assed" solution but it would not have different behavior on different models. All models would ship using "real" ISO only. The end user can tweak to use "market" ISO for input by adjusting the ratio (or a couple of discrete ratios) if he/she chooses.

Oh, I misunderstood. Not sure about that, but at least it would be consistent.

What I was thinking of was just to have a fixed default real/market conversion in the code and allowing overriding it on a per platform basis. I would want to have the real/market values for a decent sample of cameras before doing this, but assuming they were all fairly similar, I think it would be ok.

Logged

Don't forget what the H stands for.