CHDKPTP - PC Remote Control Performance Analysis - page 47 - RAW Shooting and Processing

SticK

779

Re: CHDKPTP - PC Remote Control Performance Analysis

« Reply #460 on: 25 / October / 2012, 14:22:03 »

Advertisements

Quote "The top one is the "superfine". Good eye"

Thanks. You made me step out of my comfort zone temporarily ... but ... I do appreciate the challenge. Normally in the way I described my lab process, reference and target images are always coregistered pixel-to-pixel. In your case I did not want do that due to possible pixel-shift reason explained earlier. So here's how I did it: I tone curved up your PNG, selected three contrast-transition locations, put them side by side as you see in the Figure. I found the differences to be consistent in all three and made the call.

What's telling about my pushed version is that it demonstrates very well the kind of imaging quality I rely on: Often I go in at the pixel level as you see here and rely on the contrast performance of the superfine.

Quote "although why not use raw in that case ?"
Good question and very relevant to everything. The fact that a CR2 is bigger is not much the issue. It's a matter of photographic throughput and ease of image set manipulation. JPGs are essentially instant-view in full format and quick zoom if needed. For the RAWs ... even Canon's own CR2s in Canon Zoombrowser software, require a manual decoding step and file saving manipulation, adding time and unnecessary steps which detract from the scientific focus: thinking about imaged specimen characteristics rather than having to be bogged down in computational resources. Even though RAWTherapee has a good GUI, it hogs RAM because RAWs need to remain open, among a host of other drawbacks in my instrument as a potential active part of the operating environment. Hence you could say 99%+ of my acquisitions are superfine JPG viewed in Zoombrowser, and where the performance of RAWs is absolutely necessary, then yes I spend the time to get those done.

So you can see that these kinds of differences are important to me. Thus in your opinion, do you think it's worth for me to research out your ...
Quote "=return get_prop(require('propcase').QUALITY)
=set_prop(require('propcase').QUALITY,0)"
to determine if superfine is supported by firmware? If so, it is in my high interest to explore the possibility. Alternatively, can you look at the S90 firmware to find out if you can spot "familiar" superfine code?

edit:
Perhaps the entry point on the S90 has a different address ??

[109] D10 Superfine vs fine TC+.PNG (30.24 kB, 807x428 - viewed 139 times.)

« Last Edit: 25 / October / 2012, 14:34:23 by SticK »

Logged

SticK

779

Re: CHDKPTP - PC Remote Control Performance Analysis

« Reply #461 on: 25 / October / 2012, 15:07:51 »

Hmmm .. propcase signalling ... something odd but very interesting // working on it // hope it's not a flash in the pan.

Logged

reyalp

14080

Re: CHDKPTP - PC Remote Control Performance Analysis

« Reply #462 on: 25 / October / 2012, 15:52:24 »

Quote from: SticK on 25 / October / 2012, 14:22:03

Perhaps the entry point on the S90 has a different address ??

There is no "entry point" involved here, only a propcase number and propcase value.

We know for certain the propcase functions are correct, otherwise the port wouldn't work at all.

We know the propcase number and value are correct, because you reported that the the exif records images made using the override as superfine. So either it is in fact working and the difference was just undetectable in your test, or this camera does something completely different from other (older and if I'm not mistaken newer) cameras.

There's really not much room for a simple bug in the implementation. I suppose it is possible that there is another propcase which controls actual quality, and the one we are setting only affects the exif. If this is the case (I doubt it) searching for propcases that always have the same value as current PROPCASE_QUALITY should solve it.

Note my image was taken at ISO 80 (Canon "market" UI value). At higher ISO levels, much of the high frequency detail will most likely be eliminated by the cameras noise reduction, making "superfine" have less effect on both files size and detail.

Logged

Don't forget what the H stands for.

SticK

779

Re: CHDKPTP - PC Remote Control Performance Analysis

« Reply #463 on: 25 / October / 2012, 16:39:41 »

NOT ALL SUPERFINEs are created EQUAL

Here's what I could gather til now. If Canon CR2 is ON, then the CHDK quality override doesn't work on the JPG. However, I found that if I set Canon to JPG only, then yes, the override works and the result is very different // very nice indeed. Thus Canon RAW ON was my trap. Have a look:

a) Normal 700 KB
b) Fine 1.8 MB
c) Superfine 2.8 MB (not avail in Canon)

There is a caveat. Although we see a definite improvement, for a 10Mpx, superfine is nowhere near as good as the SX110 which is around 7 MB for a 9Mpx. A typical fine on the SX110 is 3 MB. So roughly a superfine on an S90 is a bit worse than a fine on the SX110. For comparison a superfine on the 5mpx S50 is around a sensible 4 MB.

This raises two interesting questions:
1) Can the override be made to work with Canon CR2 ON ?
2) Is there a hard-coded %JPEG (probably 0->1) quality parameter for each three settings somewhere in the camera that they set specific to a camera model ?

Of the two questions, the 2nd is more intriguing. If there is, I would be immensely grateful to you if you can find a way set that parameter via CHDK.

Logged

reyalp

14080

Re: CHDKPTP - PC Remote Control Performance Analysis

« Reply #464 on: 25 / October / 2012, 23:59:58 »

Quote from: SticK on 25 / October / 2012, 16:39:41

If Canon CR2 is ON, then the CHDK quality override doesn't work on the JPG.

That's a useful bit of information.

Quote

This raises two interesting questions:
1) Can the override be made to work with Canon CR2 ON ?
2) Is there a hard-coded %JPEG (probably 0->1) quality parameter for each three settings somewhere in the camera that they set specific to a camera model ?

Reverse engineering is required to answer either of these questions.

#1 Given that you could use CHDK raw/DNG instead, or generate a jpeg with whatever quality setting you want from the raw, I see little value in expending any effort on this. Especially considering how few CHDK cameras support native raw.

#2 A way was found to override quality in mjpeg encoding. Maybe a similar thing exists for still jpeg encoding. Looking for calls to propcases functions with the PROPCASE_QUALITY might be a place to start. I had a look through my dump of propcase related calls on d10, but in the most likely area I found, the value gets shuffled through too many pointers to easily follow.

(d10 100a sub_FF93664C__SsCreateJpeg_c__1904 -> sub_FF8A6FAC -> sub_FF8A65A4__DevelopModule_c__0 -> FF8A6670, if anyone is interested)

I'm not interested in putting a lot of effort on this one either. My feeling is that Canon discarded superfine because the small, high MP sensors are so noisy that the higher jpeg quality levels are basically just preserving more noise. Yes, you can detect the difference, but you don't gain a whole lot. If you really want every last bit of noise, use raw.

Logged

Don't forget what the H stands for.

SticK

779

Re: CHDKPTP - PC Remote Control Performance Analysis

« Reply #465 on: 26 / October / 2012, 10:38:28 »

Quote "#1 Given that you could use CHDK raw/DNG instead, or generate a jpeg with whatever quality setting you want from the raw, I see little value in expending any effort on this. Especially considering how few CHDK cameras support native raw."

I agree, not a showstopper.

Quote "My feeling is that Canon discarded superfine because the small, high MP sensors are so noisy that the higher jpeg quality levels are basically just preserving more noise."

Noise is the first thing I studied in these cameras for potential CCD donors so you can count on my findings here. I demonstrated early on that the noise behavior in the SX110 and the S90 are at least 3Ev and 4Ev *lower* compared to the S50, so contrary to your assessment it makes all the sense to preserve real superfine. As obvious evidence, the SX110 certainly does preserve real superfine (8MB) despite a smaller and denser CCD, while the S90 and your D10 don't, and worse, we know now those two demote the qualities by 1 step:
SX110 superfine = S50 superfine ==>> real superfine
S90 #superfine# = D10 #superfine# = SX110 fine ==> real fine
S90 #fine# and D10 #fine# = real normal (if one follows the trend from the data in my last post)
Hence there must be another reason why Canon demoted qualities across-the-board for our two cameras. In my previous post I offered the availability of RAW in the S90 for the removal of #superfine#. But that does not explain the demotions in both our cameras. So one possibility for the demotions I thought of in our DIGIC IVs could be marketing ... people complained that they can't shoot fast enough with the PowerShots... that is, an improvement in speed-shooting because a small-size file to save // makes sense to me. And for the S90 #superfine# removal, I still think the reason is available RAW.

Quote "the value gets shuffled through too many pointers to easily follow."

Thank you for looking into your D10. I see it does look problematic, so too bad because it would really help. I know it doesn't have luster and pizazz, but personally I would put this solution into the basic operational parameter category rather than of course the fancy feature category.

Quote "If you really want every last bit of noise, use raw."

I demonstrated noise cannot be the reason for compression quality manipulation by Canon. In fact this is especially true in the S90 as it has the lowest noise of all the cameras I tested: S50, S70, S90 and SX110. So like the SX110, the S90 (and your D10) should have the opposite: "a very high superfine," that produces an 8MB file. As I explained in my previous post, using RAW for 99% of the shots is an operational kludge when trying to focus on science, when fast-performing excellent JPEGs will suffice. That's why control over the quality parameter would be very very useful for scientific applications of these high-end CCDs and signal processors.

Logged