Custom processing for JPEG (Tone curve, CA ...)

[toinech]

Difference between CV and CVF curves:
- CV curves are the standard RGB curves
- CVF curves are my own version of the curves which
  intends to avoid color shift side effect of the RGB curves.
  They are an approximation of the luminance curves.

Except the SYSCURVES.CVF file. It is recommended to
put the curves files in the curve directory.

Auto DR with Zebra is intended to be use for the direct 
application of the curves to the shot - and not for post
processing (the feature is just mimic of the Fuji S100
+200, +400 Dynamic Range). The flow is:

1. Activate Zebra display
2. Activate Auto DR
3. Use the camera EV negative compensation to reduce the
    Zebra overblown highlights shown on screen.
    I used the (+/-) button on the back of the camera.
4. Take the shot. You will directly get the JPG without post
    processing. The luminance of the darker areas will be
    raised while the highlight is compressed.

Your workflow is correct. However during the RAW develop
process, you need to set the EV comp to the value that you
used to reduce the high light blown out when you took the RAW.
I don't use much in camera develop since it is a tricky art there
(specially for WB issues).

If the scene does not require you to reduce exposure to avoid
blown highlight, the Auto DR feature does not do anything.

I am not sure what the Velvia effect is. Won't be able to help there.
I have attached some curve samples (which include a Provia effect).

[toinech]

More about CV versus CVF...

- CV curves are RGB.
  To oversimplify the process: If a pixel has the following VRi, VGi, VBi values.
  The new VRo, VGo, VBo value are derived from a table lookup based on the VRi, VGi, VBi.
  The proportion of R,G,B value may not be kept after the transformation.

  VRo = T(VRi)
  VGo  = T(VGi)
  VBo = T(VBi)

- CVF curves are approximation of luminance curve
  Again to oversimplify it: A luminance value is L derived from the VRi, VGi, VBi.
  For a specific L a factor K(L) is looked up in a table and multiplied to the VRi, VGi, VBi.

  VRo = K(L) * VRi
  VGo = K(L) * VGi
  VBo = K(L) * VBi

  Color proportion is kept since we multiply with the same factor.

[littlejohn]

Hi,

On my 860IS, it took a few seconds to complete one shot if a curve is applied.
If post processing on each JPEG/RAW file is allowed (by applying different curves), that would be great.

[toinech]

If post processing on each JPEG/RAW file is allowed (by applying different curves), that would be great.

feature is already there...
1. Take a pic with save RAW
2. Activate the curve
3. Develop RAW

Curve does not apply on the JPEG though but only to the RAW.

[dovciak]

  Hello,

  I am trying to use the curves as well. The custom curve loading works but the "Auto DR", "+1EV" and "+2EV" features does not!

  I have Canon Powershot S5 IS. In my filebrowse in CHDK I can see the filenames only in DOS 8.3 filename convention. Can this be the cause of the problem?
The file SYSCURVES.CFG is thus shown as SYSCUR~1.CFG. Is there some workaround?

  I think the "Auto DR" feature is really a must have! Can anyone help, please?

  Michal.

--> I am sorry, please, ignore my message - I made a mistake. The filename should be SYSCURVES.CVF (mistake in extension), and it works even if the filebrowser still shows SYSCUR~1.CVF!

--> Great work, no more overexposed sky!

[littlejohn]

Hi,

Check this out:
CHDK/MoreBest - CHDK Wiki
You have to put the file "SYSCURVES.CVF" in your CHDK directory.

Edit: I'm also interested in the Auto DR feature.
If you have any great picture using Auto DR, could you please post it here?
I'm not sure if I'm using the right way. 

[toinech]

I may need to adjust the SYSCURVES a little bit to get the desired effect.
As yet they are not aggressive enough to reduce the blown out.
Still need to do some more study here.

[hiker_jon]

Hi,

I just tried the "Collaborative Build" and see that several curves are built in: +1,2 ev and hdr.  That is great, glad to see it in there.  I have a request: How about adding -2,-1 ev and perhaps also the .5 ev steps.  That would give some nice exposure choices when developing raws.  I didn't look at the code, but these choices do not need a pre-calculated curve.  It can be easily calculated on the fly.

Jon

[ewavr]

I also tried curves and found some strange things (see attached picture, 100% crop at right border of image):
- Some unprocessed vertical band at right side of image. Maybe, skippings of last 40 columns is too much.
- Multiple black points on image. Maybe (only maybe!) this is defect pixels on CCD, stored in RAW with value 0. After curve processing this pixels must be stored back with the same value 0 (not black level (31)).
The same things reported in russian forum user of S3IS, his image:

[hotvedt]

Some days ago i made this post: http://chdk.setepontos.com/index.php/topic,265.msg21410.html#msg21410

The black pixels is present in my cam too (A540), also the same things as ewavr mentions.

[toinech]

I also tried curves and found some strange things (see attached picture, 100% crop at right border of image):
- Some unprocessed vertical band at right side of image. Maybe, skippings of last 40 columns is too much.

I guess not skipping any column at all would be good for all cameras. The column skipping is only to save some ridiculous processing time on a610. It was tested with a A610 which does not use those columns at all.

- Multiple black points on image. Maybe (only maybe!) this is defect pixels on CCD, stored in RAW with value 0. After curve processing this pixels must be stored back with the same value 0 (not black level (31)).

Why store back 0? Unless the camera JPEG processing engine is doing something with it.

[toinech]

I just tried the "Collaborative Build" and see that several curves are built in: +1,2 ev and hdr.  That is great, glad to see it in there.  I have a request: How about adding -2,-1 ev and perhaps also the .5 ev steps.  That would give some nice exposure choices when developing raws.  I didn't look at the code, but these choices do not need a pre-calculated curve.  It can be easily calculated on the fly.

What will be the use of -2, -1 curves?
Calculation on the fly may not be optimal for a real time development to JPG if done at development time. It can be done at selection time though.

I had a non public build with 1/3 EV step (in the Auto DR mode) in order to follow the camera EV steps. Nice but - I still user test it. I have not feel that this is really useful.

[hiker_jon]

What will be the use of -2, -1 curves?
Calculation on the fly may not be optimal for a real time development to JPG if done at development time. It can be done at selection time though.

I find it very useful.  I often get overexposed images (in fact that is a characteristic of Canon Powershots) that need a little -ev compensation.

I wrote code to do this on the fly (i.e. calculate the curve when it is needed.)  The amount of computation is minimal.  Here is the beginning of my routine.
Jon

Code:

#define CURVE_SIZE 1024
unsigned short curve[CURVE_SIZE];

void raw_change_ev() {
double mult;
double black;
short i,j;
unsigned short pixVal0, pixVal1, pixVal2;
unsigned char *src;

black = 0;
for(i=10;i<20;i++) black += get_raw_pixel(i,1);
black = black / 10;// calculate the black value using pixels that are always black

mult = 1;
if(conf.raw_ev == 0) return;
if(conf.raw_ev == 1) mult = 1.5;//ev+.5
if(conf.raw_ev == 2) mult = 2;//ev +1
if(conf.raw_ev == 3) mult = 3;//ev+1.5
if(conf.raw_ev == 4) mult = 4;//ev+2
if(conf.raw_ev == 5) mult = .25;//ev-2
if(conf.raw_ev == 6) mult = .375;//ev-1.5
if(conf.raw_ev == 7) mult = .5;//ev-1
if(conf.raw_ev == 8) mult = .75;//ev-.5

for(i=0;i<CURVE_SIZE;i++) {
j = (short) ((i * mult) + black - mult * black);
if(j > 1023) j = 1023;
if(j < 0) j = 0;
curve[i] = j;
}

[toinech]

Hum...
I've always been able to reduce the blown out with in camera negative EV adjustment.
Applying curves always incurs some data loss due to quantification. That's why I always
avoid using digital level gain/attenuation if the in camera EV adjustment does the job.

Then I would not use the digital gain/attenuation outsideof the following context:
- Increase apparent ISO beyond current camera ISO capability to maintain same shutter speed.
- Decrease apparent ISO beyond current camera ISO capability (eg. Electronic ND filter).

The +1, +2 EV syscurves provided has a factor of +1 or +2 EV with the lower luminance
and reducing the slope for higher luminance value to reduce highlight blown out when
combined with in camera -1, -2EV - creating a non linear response curve.

[hiker_jon]

Hi Toinech,

I think I don't entirely understand how you use this.  Do you use it while shooting?  That's not how I use it.

I use curves when developing raw in camera that I have already shot.  You can't change the ev using the camera button in that situation. The on-camera ev button changes the exposure when taking a picture, but makes no difference if you are using CHDK to redevelop a raw that has already been captured.  So I use curves to adjust exposure on raws that I have already taken.  In that case  brightening or darkening the image is a common need and can't be done in camera any other way.

Going from 10bit data to 8 bit display must unavoidably involve some loss of data.  Giving people flexibility in how this is done is good.   +/- ev changes to the raw data are straightforward and useful.
Jon