Black and White RAW counting routine for HDR purpose - General Discussion and Assistance - CHDK Forum
« previous next »
Pages: [1] 2   Go Down

Black and White RAW counting routine for HDR purpose

  • 18 Replies
  • 6839 Views
*

Offline Lebeau

  • ***
  • 187
Black and White RAW counting routine for HDR purpose
« on: 14 / May / 2011, 16:03:34 »
Advertisements
Bonjour,

For HDR purpose, I made a little routine to count, in raw buffer,
 - black pixels (pixels lower than twice black level),
 - white pixels (pixels greather than half white level)
 - masked pixels (standard deviation from black level in masked area)

.. per color :)

I integrate this routine in raw.c plus a function, to get result, and display it in replacement to Farenheit temperature OSD (i.e. in gui_osd.c). I know it's not the right place but for troubleshooting purpose ... :)

On my A650is, it take about 7 to 8 secs to compute this. Now, I have to define a LUA function to get these data usable from Lua.

Any suggestion, or best practice?

P.S.: I use "twice" black and "half" white to consider gamma correction without scaling these numerous data. Standard deviation of masked area, if define, indicates the level of sensor noise without any exposure.
Logged
*

Offline reyalp

  • ******
  • 14082
Re: Black and White RAW counting routine for HDR purpose
« Reply #1 on: 14 / May / 2011, 16:33:13 »
I suggest you use shot_histogram as an example, it's quite similar. For lua, you might want to return a table with all the different results. You can see an example of this in get_buildinfo().
Quote
On my A650is, it take about 7 to 8 secs to compute this.
You might hit watchdog or other timeouts in some cases. You can optimize reading pixels if you know you are going to read them all, especially if you are willing to write arm ASM ;)

shot_histogram doesn't use every pixel, it just samples some.
Logged
Don't forget what the H stands for.
*

Offline Bernd R

  • ****
  • 430
Re: Black and White RAW counting routine for HDR purpose
« Reply #2 on: 15 / May / 2011, 04:15:56 »
this sound great, where is the source of it, so i can test ?

newer cameras are faster.when can see after a shot the RAW is correct, good to be sure to have best picture possible

If you can place the routine in the DNG Pixelswap it can maybe done for free time cost, because memory read write is slow, and if CPU do during read and write 10-20 more asm intstructions is not notice, only CPU need not so long wait for mem access 
Logged
Ixus 1000 HS
*

Offline Lebeau

  • ***
  • 187
Re: Black and White RAW counting routine for HDR purpose
« Reply #3 on: 15 / May / 2011, 15:45:49 »
I didn't know about shot_histogram  ;) I will get a look to.

The draft version (using get_raw_pixel with "<" and ">" compare) was consuming about 21 secs without any watchdog (?) problem. The optimize version (using direct mem access with "bit &" operation) is consuming about 7 secs, so a 3 to 1 ratio.

I have the good will to write ARM assembler but missing time to learn ARM architecture, programming and setup  :'(

I am convince that my optimize version could be well implemented within DNG backward little-endian -> big-endian conversion but my time and my knowledge is missing.

Here is my routine:


P.S.: in get_raw_stats function, remove scale factor operations since they are erronous :(
« Last Edit: 15 / May / 2011, 18:33:15 by Lebeau »
Logged
*

Offline Lebeau

  • ***
  • 187
Re: Black and White RAW counting routine for HDR purpose
« Reply #4 on: 17 / May / 2011, 07:59:06 »
Ok ! I got the shot_histogram point :( I really didn't know about since the build_shot_histogram were in well known raw.c :( and accessible from Lua via shot_histogram_enable and get_histo_range functions :( :( :(  :'(

Here is a corrected version concerning result scaling:
Logged
*

Offline Lebeau

  • ***
  • 187
Re: Black and White RAW counting routine for HDR purpose
« Reply #5 on: 20 / May / 2011, 10:20:12 »
I follow "shot_histo_enable" way to insert into luascript.c and I can't found any reference to my lua functions in dump file.

What I missed ?
Logged
*

Offline reyalp

  • ******
  • 14082
Re: Black and White RAW counting routine for HDR purpose
« Reply #6 on: 20 / May / 2011, 13:19:16 »
Quote from: Lebeau on 20 / May / 2011, 10:20:12
I follow "shot_histo_enable" way to insert into luascript.c and I can't found any reference to my lua functions in dump file.

What I missed ?
Hard to say without seeing the code. Did you add the FUNC macro to register your function ?
Logged
Don't forget what the H stands for.
*

Offline Lebeau

  • ***
  • 187
Re: Black and White RAW counting routine for HDR purpose
« Reply #7 on: 20 / May / 2011, 13:38:18 »
I just find it and saw your reply when I came back  >:( I feel stupid  :blink:

I will do performance test and come back  :xmas
Logged
*

Offline Bernd R

  • ****
  • 430
Re: Black and White RAW counting routine for HDR purpose
« Reply #8 on: 21 / May / 2011, 14:54:26 »
I try to compile and i get this errors.i think your code should work without this values, because below the errors is  that line

#if defined CAM_MASKED_AREA_X1

and it use diffrent code

raw.c:82: error: `CAM_MASKED_AREA_Y2' undeclared (first use in this function)
raw.c:82: error: (Each undeclared identifier is reported only once
raw.c:82: error: for each function it appears in.)
raw.c:82: error: `CAM_MASKED_AREA_Y1' undeclared (first use in this function)
raw.c:82: error: `CAM_MASKED_AREA_X2' undeclared (first use in this function)
raw.c:82: error: `CAM_MASKED_AREA_X1' undeclared (first use in this function)
raw.c: In function `generate_raw_stats':
raw.c:132: error: `CAM_RAW_ROWLEN' undeclared (first use in this function)

I change to this and it get less compile errors.

#if defined CAM_MASKED_AREA_X1

   int factor = ( stats_domain = RAW_STATS_MASKED
               ? ( (CAM_MASKED_AREA_Y2-CAM_MASKED_AREA_Y1) * (CAM_MASKED_AREA_X2-CAM_MASKED_AREA_X1) )
               : ( (CAM_ACTIVE_AREA_Y2-CAM_ACTIVE_AREA_Y1) * (CAM_ACTIVE_AREA_X2-CAM_ACTIVE_AREA_X1) ) );
#else
 int factor = ( stats_domain = RAW_STATS_MASKED
               ? ( (CAM_ACTIVE_AREA_Y2-CAM_ACTIVE_AREA_Y1) * (CAM_ACTIVE_AREA_X2-CAM_ACTIVE_AREA_X1) )
               : ( (CAM_ACTIVE_AREA_Y2-CAM_ACTIVE_AREA_Y1) * (CAM_ACTIVE_AREA_X2-CAM_ACTIVE_AREA_X1) ) );
#endif

Now i have only this error left

raw.c:132: error: `CAM_RAW_ROWLEN' undeclared (first use in this function)

i see in code for CAM_MASKED_AREA_X1 is no CAM_RAW_ROWLEN used and function getrawpixel is used.

is code that use no  getrawpixel is faster ?
Logged
Ixus 1000 HS
*

Offline Lebeau

  • ***
  • 187
Re: Black and White RAW counting routine for HDR purpose
« Reply #9 on: 22 / May / 2011, 09:07:50 »
-------------------------
In my camera.h, I added CAM_MASKED_AREA_... values (pointing to Canon Masked Area)
Here is my values:
Code: [Select]
/* lebeau :
The Canon raw data contains a masked area and an active are which contains the JPEG area.

The mask area could be usefull for some software, to evaluate black level.
In fact, the masked area reflect a quasi Gaussian distribution of black values,
between 0 to, about, twice the black level, making an average of black level.

Mask Area Values are set in camera.h and could be approximative.
Select an area at the left of the active area, not so near the overall raw area.
All numbers shall be even for CFA pattern

example for my A650, but not tested on other cam !!!
   CAM_MASKED_AREA_X1 >= 12
   CAM_MASKED_AREA_Y1 = CAM_ACTIVE_AREA_Y1
   CAM_MASKED_AREA_X2 <= ( CAM_ACTIVE_AREA_X1 - 4 )
   CAM_MASKED_AREA_Y2 = CAM_ACTIVE_AREA_Y2
*/
    #define CAM_MASKED_AREA_X1 12
    #define CAM_MASKED_AREA_Y1 14
    #define CAM_MASKED_AREA_X2 48
    #define CAM_MASKED_AREA_Y2 3030
/* Masked Area
This tag contains a list of non-overlapping rectangle coordinates of fully masked pixels, which can be optionally used by DNG readers to measure the black encoding level.
The order of each rectangle's coordinates is: top, left, bottom, right.
If the raw image data has already had its black encoding level subtracted, then this tag should not be used, since the masked pixels are no longer useful.
Note that DNG writers are still required to include estimate and store the black encoding level using the black level DNG tags. Support for the MaskedAreas tag is not required of DNG readers.
*/

-------------------------
CAM_RAW_ROW_LEN were in raw.h and I move it to camera.h because it is usefull, from the begining, for other #define AND CAM_stuff are defined there. Therefore, I also modified RAW_ROW_LEN to CAM_RAW_ROW_LEN
Code: [Select]
"were in raw.h"
//#define RAW_ROWLEN ((CAM_RAW_ROWPIX*CAM_SENSOR_BITS_PER_PIXEL)/8)
"move to camera.h, with prefixed CAM_"

-------------------------
Yes, I made a junior error writing "stats_domain = RAW_STATS_MASKED" instead of "stats_domain == RAW_STATS_MASKED", sorry

Also, I don't know why that code section is compile (and generate error) if your "CAM_MASKED_AREA_X1" is not defined" ? May be because of my junior error?
-------------------------
Yes, direct check instead of getrawpixel is really faster since it conditionnally access less addr[val] and, also, compute less often the addr value.

-------------------------
I will come back with a Lua function interface as soon as tested which will look like that.
Code: [Select]
// selectively set hdr stats mode, called before capturing raw data
static int luaCB_set_hdr_stats_mode( lua_State* L )
{
set_hdr_stats_mode ( luaL_checknumber(L,1) ); // HDR_STATS_NONE (=0), HDR_STATS_BLACK (=1), HDR_STATS_WHITE (=2) or HDR_STATS_BLACK_WHITE (=3)
return 0;
}

// return count of the active area cells, upon requested color
static int luaCB_get_hdr_stats( lua_State* L )
{
int hdr_stats = (luaL_checknumber(L,1)); // HDR_STATS_BLACK (=1) or HDR_STATS_WHITE (=2)
int stats_color = (luaL_checknumber(L,2)); // STATS_ALL, STATS_RED, STATS_GREEN1, STATS_GREEN2 or STATS_BLUE
lua_pushnumber( L, get_hdr_stats ( hdr_stats, stats_color ) ); // return counts of black or white region, upon requested color
return 1;
}

#if defined CAM_MASKED_AREA_X1
// set noise stats mode, called before capturing raw data
static int luaCB_set_noise_stats_mode( lua_State* L )
{
set_noise_stats_mode ( luaL_checknumber(L,1) ); // false (=0) or true (=1)
return 0;
}

// return standard deviation of the masked area cells, upon requested color
static int luaCB_get_noise_stats( lua_State* L )
{
int stats_color = (luaL_checknumber(L,1)); // STATS_ALL, STATS_RED, STATS_GREEN1, STATS_GREEN2 or STATS_BLUE
lua_pushnumber( L, get_noise_stats ( stats_color ) );
return 1;
}
#endif
« Last Edit: 22 / May / 2011, 09:19:54 by Lebeau »
Logged
Pages: [1] 2   Go Up
« previous next »
 

Related Topics