CHDKPTP - PC Remote Control Performance Analysis - page 42 - RAW Shooting and Processing - CHDK Forum
supplierdeeply
« previous next »
Pages: 1 ... 37 38 39 40 41 [42] 43 44 45 46 47   Go Down

CHDKPTP - PC Remote Control Performance Analysis

  • 465 Replies
  • 119660 Views
*

Offline SticK

  • *****
  • 779
Re: CHDKPTP - PC Remote Control Performance Analysis
« Reply #410 on: 09 / October / 2012, 22:41:46 »
Advertisements
Quote "This is exactly matches what I've been trying to describe for the last several pages."

Last several pages?  Holey moley -- I must look totally dense to everyone reading this, like an old stump of petrified wood.  But wait .....  since I caught this ISO override gremlin over the weekend did we not have our first conversation on this just yesterday?  http://chdk.setepontos.com/index.php?topic=8613.msg91720#msg91720  If not, well then, according to your mathematics, I've lived nine hundred years!  Since you have evoked such intense pity in me I will buy you a new pen when this is all over.  // signed // Joda.

WRT deeper analysis of the 3200 issue, this is one time I jumped ahead of you and did a number of tests taht actually included your suggestions early this morning but have had to divert my attention to some other matters today.  The results are quite interesting and surprising / not surprising.  Will report later, if not tonight then tomorrow.

Logged
*

Offline SticK

  • *****
  • 779
Re: CHDKPTP - PC Remote Control Performance Analysis
« Reply #411 on: 10 / October / 2012, 14:31:02 »
"ISO3200 Special" Native CR2 + JPG  test results

These were shot with the same parameters and conditions as the DNG+JPG in my previous post.  To decode CR2s I used RawTherapee.  Fig 1 is CR2 left column and parallel JPG right column.  Shooting parameters are displayed in the thumbnails.

Figure 1

A note on barrel distortion first.

 >> Before RT decodes a CR2 (ISO 400, no green checkmark), it displays the original thumbnail from the camera.  Because then incoming thumbnail is barrel-corrected, this demonstrates that the EXIF thumbnail is a small version of the JPG, which we know now is a barrel-corrected CCD raw, done in the camera.

 >> After decoding a CR2 (green checkmarks), RT updates the thumbnail.  You can see the barrel distortion and more subject area captured in the CR2 raw image data when compared to its JPG.

 >> Hence we can conclude that CR2 raw data as emitted by the camera, is not consistent with the thumbnail.  So if one is using Canon PC software to decode and view CR2s, decoded CR2s get barrel-corrected on-the-fly on the PC, giving one the illusion that they are the same // nice trick.  Hence this new data confirms that the PC software must know the camera model to apply the appropriate algorithm.

The foregoing raises the question: does the ISO3200 Special undergo a similar process?  What really happens inside in the camera?

ISO3200 Special

  >> In Figure 1, ISO brightness stepping from row-to-row is consistent.  So we can firmly establish that after RT decodes the CR2, image exposure also remains unchanged.

  >> Hence in that respect, ISO3200 CR2 brightness is as originally given by Canon, and is the same as the JPG, leading one to conclude there are no brightness correction-specific exposure transforms embedded in EXIF.

This begs the question then ... what is special about this high ISO mode?  Does Canon expose at 1600 and contrast-push the exposure?  Is this a true electronically-increased VGA gain setting?  A combination?  In Figure 2 seeks to find some answers.

Figure 2

Is Canon ISO3200 Special a fake?  Here I expose the scene at ISO 3200 top left, and 1600, top right.  Bottom is a color-balanced tonecurve-pushed version of the ISO 1600 I did by hand so that the wall background and keyboard areas have increased intensity by very close to 1EV, on the plus side // if you look carefully you see that the push is slightly more than 1EV, done intentionally to be on the safe side as you will see below.  Hence if one part of the scene is exposed equally across the two images (wall and keyboard), all parts should follow (computer screen).  That means if Canon indeed creates a 3200 by shooting at electronic 1600 and then does a trivial intensity change in firmware, the bottom image should be identical to the top left.

It is not identical ... observing the yellow zones, it is clear that the highlights are considerably higher intensity (around an additional 1EV) in the ISO3200 than the pushed ISO1600, despite the EV1+ on the 1600.  I also find that the vertical wall transitions borders appear sharper in the ISO3200.  And what of the noise and SNR?  By observation noise appears roughly the same between the pushed ISO1600 and the ISO3200.

Some conclusions

That result leans towards a preliminary observational interpretation that the 1EV+ ISO 1600, noise seems to increase as expected, but in the ISO3200, the scene signal strength does appear to be higher, thus one could conclude the Canon "ISO3200 Special" is unique and not a fake.  In my early empirical analysis of this camera I paid much attention to this setting.  Results then were consistent with these, but with very dark scenes, full iris and very long exposures.  I could even say that ISO3200 performs better at the longer exposures: overall, after taking many shots at 15 seconds it was clear to me that the 3200s had better SNR and image rendition than pushed-1600s under the same environmental and setting conditions.  This means that with dark-subtracted long exposures (these here are 1/4s not dark-subtracted), ISO3200 Special performs very well.  Perhaps Canon does change VGA amplifier gain for the 3200, but it is not exactly 2X over 1600, after which a sophistical noise-reduction algorithm handles the rest.  It could be a combination of factors that give excellent results.
Logged
*

Offline reyalp

  • ******
  • 14079
Re: CHDKPTP - PC Remote Control Performance Analysis
« Reply #412 on: 11 / October / 2012, 00:00:10 »
Quote from: SticK on 10 / October / 2012, 14:31:02
That result leans towards a preliminary observational interpretation that the 1EV+ ISO 1600, noise seems to increase as expected, but in the ISO3200, the scene signal strength does appear to be higher, thus one could conclude the Canon "ISO3200 Special" is unique and not a fake.
We know that the data in CHDK raw/DNG is ultimately the input for the jpeg process, because we can modify it and see the effects in the jpeg. So if there is no significant difference in the pixel values of a DNG at ISO 1600 and ISO 3200, we can be confident that any difference in the jpeg is from processing later.
Logged
Don't forget what the H stands for.
*

Offline SticK

  • *****
  • 779
Re: CHDKPTP - PC Remote Control Performance Analysis
« Reply #413 on: 11 / October / 2012, 00:58:02 »
Quote "We know that the data in CHDK raw/DNG is ultimately the input for the jpeg process, because we can modify it and see the effects in the jpeg. So if there is no significant difference in the pixel values of a DNG at ISO 1600 and ISO 3200, we can be confident that any difference in the jpeg is from processing later."

Yes you could say that 1600 DNG and 3200 DNG are for all practical purposes identical.  The results are conclusive.  But you can't ignore the results from the Canon CR2s either.  In the Canon 3200 CR2 there likely is some noise SNR enhancement process going on (agreeing with the 2nd part of you statement).  Do the they expose their in-camera 3200 at 1600 electrical?  They may not.  We really don't know.  It is entirely possible they are still gaining up the amplifier by a non-standard step value, even going to a larger value than 2X, like 4X, and then doing firmware noise and signal level reductions  ... Assuming that very real possibility, my question is this ... if I say -sv=3200 in PTP (or shoot a DNG at 3200) that because I am bypassing the Canon 3200 process function, I may not have the same *amplifier gain* if I had pressed the Canon button with the camera set at 3200??  For the existing ISO call you intercept, Canon may be doing something like amp_gain = min(my_iso, 1600).  Perhaps the Canon ISO3200 is another kind function call which invokes a non-standard amplification gain?  Do you see where I'm going?

edit:
The point I am trying to make is this ... it is definite that when I say -sv=3200 via PTP the Canon 3200 doesn't happen.  Even if it is a fake (Canon actually exposes at the 1600 we are seeing in the DNG and *only* does intensity manipulation, not both) .... unless the Canon 3200 function call is tested by CHDK to produce the same CR2 created by the on-camera button would we only know then (by inspecting the DNG), whether any CCD amplifier gain different from 1600 was applied and how much: the DNG will tell us the truth.  If there is an amplifier gain change, whether it is 2X or higher or lower, it would be a useful CHDK feature to have.  But that can only be discovered with 100% certainty if CHDK implements the feature.  It would be interesting if you could locate the hook so we can find out.
« Last Edit: 11 / October / 2012, 09:02:04 by SticK »
Logged
*

Offline SticK

  • *****
  • 779
Re: CHDKPTP - PC Remote Control Performance Analysis
« Reply #414 on: 11 / October / 2012, 11:40:45 »
Quote "As far as I know, there is no practical difference between "PTP ISO override" and the CHDK menu ISO override, so I can only assume there is some other difference between the tests, or some observational error."

... So I carefully repeated the tests by going back to 2163-test-1 (Fig 1) and then again to present 2201-reyalp-test-1 (Fig 2).  I did exactly the same sequence but now from PTP: shoot -tv=1/4 -av=4 -sv=X, where X was 400, 800, 1600 and 3200.  Canon is set to ISO 3200 in all of these.

Conclusion: 2201-test-1 works and 2163-test-1 doesn't.  Can you check it in?

CHDKPTP REBOOT WARNING: Note that the last 8 file numbers are the same in Fig 1 and Fig 2.  That's because when I use REBOOT from CHDKPTP (PLAY, REBOOT, CONNECT), the image counter is not saved in permanent memory overwriting the last files since the last real PDN.  If I use SHUTDOWN and then the power switch, the next file is correct, ie IMG_6208.  That means shooting after REBOOT overwrites files and would be good to fix.
Logged
*

Offline SticK

  • *****
  • 779
Re: CHDKPTP - PC Remote Control Performance Analysis
« Reply #415 on: 12 / October / 2012, 00:46:39 »
The S90_101a has unofficially passed Phase II acceptance and has been moved to Phase III electromechanical modifications.

In that respect, I am inviting you to visit my new sideshow thread with an introductory post embarking on an adventure called "hacking out the CCD."
http://chdk.setepontos.com/index.php?topic=8801.msg91863#msg91863

I do remain committed to this present thread in parallel, at least until Phase III is complete.


Logged
*

Offline waterwingz

  • ******
  • 12138
    • Waterwingz
Re: CHDKPTP - PC Remote Control Performance Analysis
« Reply #416 on: 12 / October / 2012, 10:00:35 »
Did you ever publish a formal definition of what all these "Phases" you keep referencing include ?
Logged
Ported :   A1200    SD940   G10    Powershot N    G16
*

Offline SticK

  • *****
  • 779
Re: CHDKPTP - PC Remote Control Performance Analysis
« Reply #417 on: 12 / October / 2012, 10:15:47 »
Logged
*

Offline waterwingz

  • ******
  • 12138
    • Waterwingz
Re: CHDKPTP - PC Remote Control Performance Analysis
« Reply #418 on: 12 / October / 2012, 12:18:23 »
Quote from: SticK on 12 / October / 2012, 10:15:47
Yes.

http://chdk.setepontos.com/index.php?topic=8613.msg91048#msg91048, 3rd para.
http://chdk.setepontos.com/index.php?topic=8613.msg91295#msg91295

Hope this helps.
Thanks ... I think.   From what I can see,  neither of those links points to a post that says something clear like :

Phase1 :  The objective of phase 1 is  .....   Work done will include ....   Completion of phase 1 is when ....
Phase2 :  The objective of phase 2 is  .....   Work done will include ....   Completion of phase 2 is when ....
Phase3 :  The objective of phase 3 is  .....   Work done will include ....   Completion of phase 3 is when ....
Phase4 :  The objective of phase 4 is  .....   Work done will include ....   Completion of phase 4 is when ....
Phase5 :  The objective of phase 5 is  .....   Work done will include ....   Completion of phase 5 is when ....
Phase5 :  The objective of phase 6 is  .....   Work done will include ....   Completion of phase 6 is when ....
Phase6 :  The objective of phase 7 is  .....   Work done will include ....   Completion of phase 7 is when ....
etc etc etc


They just seem to give vague references to some activities that might occur during the course of the project and attach those activities to a numberer phase without reference to an overall plan.
Logged
Ported :   A1200    SD940   G10    Powershot N    G16
*

Offline reyalp

  • ******
  • 14079
Re: CHDKPTP - PC Remote Control Performance Analysis
« Reply #419 on: 12 / October / 2012, 23:17:06 »
Quote from: SticK on 11 / October / 2012, 00:58:02
If there is an amplifier gain change, whether it is 2X or higher or lower, it would be a useful CHDK feature to have.  But that can only be discovered with 100% certainty if CHDK implements the feature.  It would be interesting if you could locate the hook so we can find out.
I don't understand. If you set the ISO using Canon menus and compare DNGs shot at 1600 and 3200, than that will definitively tell you whether there is a difference in amplifier gain. As I said before, the DNG data is the input to the Canon jpeg process. If the raw pixels values don't differ between 1600 and 3200, then we can be 100% certain that the difference is from later software processing. This doesn't mean it is entirely "fake" in the sense that the 3200 jpegs have no additional information, because Canon could do the conversion from 12 bit to jpeg differently.

Anyway, speaking for myself I don't have any idea how to make overrides work for these "fake" ISO values, and have very little interest in spending my time to find out. Maybe some other developer feels differently.
Quote
That's because when I use REBOOT from CHDKPTP (PLAY, REBOOT, CONNECT), the image counter is not saved in permanent memory overwriting the last files since the last real PDN.  If I use SHUTDOWN and then the power switch, the next file is correct, ie IMG_6208.  That means shooting after REBOOT overwrites files and would be good to fix.
I've updated to the wiki to reflect the fact the counter is one of the settings that isn't saved.
Logged
Don't forget what the H stands for.
Pages: 1 ... 37 38 39 40 41 [42] 43 44 45 46 47   Go Up
« previous next »
 

Related Topics