Suggestion for DNG optimisation in raw.c

philmoz

3449

« on: 14 / December / 2010, 23:07:17 »

Advertisements

I noticed that when creating DNG files the function get_raw_pixel is called once for every pixel in the thumbnail and nine times for each bad pixel. Also set_raw_pixel is called once for each bad pixel.

On my SX30 this amounts to ~300,000 calls to get_raw_pixel and set_raw_pixel. Each of these calls is in turn calling get_raw_image_addr to get the raw buffer memory address.

Since get_raw_image_addr will return the same value for each call (while processing a given image) all these extra calls to get_raw_image_addr are redundant.

Instead I suggest calling get_raw_image_addr once at the start of raw_savefile and then passing the returned value to each of the functions that need the raw buffer address:
- get_bad_count_and_write_file
- create_thumbnail
- set_raw_pixel
- get_raw_pixel
- patch_bad_pixel
- patch_bad_pixels
- patch_bad_pixels_b

Passing the value as a parameter is probably more efficient than storing it in a global variable as it will be passed around in a register and will save a memory access.

The function build_shot_histogram in core\shot_histogram.c also needs to be updated since it calls get_raw_pixel as well.

Regards,
Phil.

Logged

CHDK ports:
sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
g12 (1.00c, 1.00e, 1.00f & 1.00g)
sx130is (1.01d & 1.01f)
ixus310hs (1.00a & 1.01a)
sx40hs (1.00d, 1.00g & 1.00i)
g1x (1.00e, 1.00f & 1.00g)
g5x (1.00c, 1.01a, 1.01b)
g7x2 (1.01a, 1.01b, 1.10b)

reyalp

14003

Re: Suggestion for DNG optimisation in raw.c

« Reply #1 on: 14 / December / 2010, 23:23:52 »

Have you measured the difference ?

As I recall, the entire bad pixel processing is quite insignificant in comparison to the other parts of raw processing. I'm not against doing this, but I doubt it will have a noticeable benefit.

Logged

Don't forget what the H stands for.

philmoz

3449

Re: Suggestion for DNG optimisation in raw.c

« Reply #2 on: 15 / December / 2010, 03:30:53 »

Quote from: reyalp on 14 / December / 2010, 23:23:52

Have you measured the difference ?

As I recall, the entire bad pixel processing is quite insignificant in comparison to the other parts of raw processing. I'm not against doing this, but I doubt it will have a noticeable benefit.

The previous change and the optimised versions of get_raw_pixel and set_raw_pixel below shave 80ms of the DNG save time on my SX30 - not huge; but every little bit helps.

Optimised get_raw_pixel for 12 bit raw (eliminates a multiplication & 2 divisions, and decreases memory accesses for two of the cases.

Code: [Select]

unsigned short get_raw_pixel(unsigned char* rawadr,unsigned int x,unsigned  int y){
#if CAM_SENSOR_BITS_PER_PIXEL==10
 unsigned char* addr=rawadr+y*RAW_ROWLEN+(x/8)*CAM_SENSOR_BITS_PER_PIXEL;
 switch (x%8) {
  case 0: return ((0x3fc&(((unsigned short)addr[1])<<2)) | (addr[0] >> 6));
  case 1: return ((0x3f0&(((unsigned short)addr[0])<<4)) | (addr[3] >> 4));
  case 2: return ((0x3c0&(((unsigned short)addr[3])<<6)) | (addr[2] >> 2));
  case 3: return ((0x300&(((unsigned short)addr[2])<<8)) | (addr[5])); 
  case 4: return ((0x3fc&(((unsigned short)addr[4])<<2)) | (addr[7] >> 6)); 
  case 5: return ((0x3f0&(((unsigned short)addr[7])<<4)) | (addr[6] >> 4)); 
  case 6: return ((0x3c0&(((unsigned short)addr[6])<<6)) | (addr[9] >> 2)); 
  case 7: return ((0x300&(((unsigned short)addr[9])<<8)) | (addr[8])); 
 }
#elif CAM_SENSOR_BITS_PER_PIXEL==12
 unsigned char* addr=rawadr+y*RAW_ROWLEN+(x&0xfffffffc)+((x>>1)&0xfffffffe);
 switch (x&3) {
  case 0: return ((unsigned short*)addr)[0] >> 4;
  case 1: return ((unsigned short)(addr[0] & 0x0F) << 8) | (addr[3]);
  case 2: return ((unsigned short)(addr[2]) << 4) | (addr[5] >> 4);
  case 3: return ((unsigned short*)addr)[2] & 0xfff;
 }
#else 
 #error define get_raw_pixel for sensor bit depth
#endif
 return 0;
}

Optimised set_raw_pixel for 12 bit raw (eliminates a multiplication & 2 divisions, and decreases memory accesses for two of the cases.

Code: [Select]

void set_raw_pixel(unsigned char* rawadr,unsigned int x, unsigned int y, unsigned short value){
#if CAM_SENSOR_BITS_PER_PIXEL==10
 unsigned char* addr=rawadr+y*RAW_ROWLEN+(x/8)*CAM_SENSOR_BITS_PER_PIXEL;
 switch (x%8) {
  case 0: addr[0]=(addr[0]&0x3F)|(value<<6); addr[1]=value>>2;                  break;
  case 1: addr[0]=(addr[0]&0xC0)|(value>>4); addr[3]=(addr[3]&0x0F)|(value<<4); break;
  case 2: addr[2]=(addr[2]&0x03)|(value<<2); addr[3]=(addr[3]&0xF0)|(value>>6); break;
  case 3: addr[2]=(addr[2]&0xFC)|(value>>8); addr[5]=value;                     break;
  case 4: addr[4]=value>>2;                  addr[7]=(addr[7]&0x3F)|(value<<6); break;
  case 5: addr[6]=(addr[6]&0x0F)|(value<<4); addr[7]=(addr[7]&0xC0)|(value>>4); break;
  case 6: addr[6]=(addr[6]&0xF0)|(value>>6); addr[9]=(addr[9]&0x03)|(value<<2); break;
  case 7: addr[8]=value;                     addr[9]=(addr[9]&0xFC)|(value>>8); break;
 }
#elif CAM_SENSOR_BITS_PER_PIXEL==12
 unsigned char* addr=rawadr+y*RAW_ROWLEN+(x&0xfffffffc)+((x>>1)&0xfffffffe);
 switch (x&3) {
  case 0: ((unsigned short*)addr)[0] = (((unsigned short*)addr)[0]&0x0F) | (value << 4);  break;
  case 1: addr[0] = (addr[0]&0xF0) | (unsigned char)(value >> 8);  addr[3] = (unsigned char)value;         break;
  case 2: addr[2] = (unsigned char)(value >> 4);  addr[5] = (addr[5]&0x0F) | (unsigned char)(value << 4);  break;
  case 3: ((unsigned short*)addr)[2] = (((unsigned short*)addr)[2]&0xF00) | value; break;
 }
#else 
 #error define set_raw_pixel for sensor bit depth
#endif
}

Logged

CHDK ports:
sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
g12 (1.00c, 1.00e, 1.00f & 1.00g)
sx130is (1.01d & 1.01f)
ixus310hs (1.00a & 1.01a)
sx40hs (1.00d, 1.00g & 1.00i)
g1x (1.00e, 1.00f & 1.00g)
g5x (1.00c, 1.01a, 1.01b)
g7x2 (1.01a, 1.01b, 1.10b)

Microfunguy

4037

Re: Suggestion for DNG optimisation in raw.c

« Reply #3 on: 15 / December / 2010, 05:11:36 »

Just for the record, I have just finished adding a 'save raw image strip' to SDM.

Bad-pixel patching and byte reversal is done on the PC.

Sensor data and DNG header (including white-balance as shot) are saved, in less than one second (for full-size image).

David

Logged

Bernd R

430

Re: Suggestion for DNG optimisation in raw.c

« Reply #4 on: 15 / December / 2010, 11:08:09 »

@Microfunguy
>Just for the record, I have just finished adding a 'save raw image strip' to SDM.
>Bad-pixel patching and byte reversal is done on the PC.

this sound great, so there is no need anymore to have correct code for the Sensor ?
The Ixus 1000 and Ixus 300 Port have wrong color in RAW, is it possible to try out in PC to get color working ?

so this code is not need ?
If so, can you upload your full source, i like test if Ixus 1000 work better with RAW.

""""""
#if cam_CFAPattern==0x02010100 // Red Green Green Blue
   r=gamma[get_raw_pixel((x/2)*2,(y/2)*2)>>(CAM_SENSOR_BITS_PER_PIXEL-8)]; // red pixel
   g=gamma[6*(get_raw_pixel((x/2)*2+1,(y/2)*2)>>(CAM_SENSOR_BITS_PER_PIXEL-8))/10]; // green pixel
   b=gamma[get_raw_pixel((x/2)*2+1,(y/2)*2+1)>>(CAM_SENSOR_BITS_PER_PIXEL-8)]; //blue pixel
#elif cam_CFAPattern==0x01000201 // Green Blue Red Green
   r=gamma[get_raw_pixel((x/2)*2,(y/2)*2+1)>>(CAM_SENSOR_BITS_PER_PIXEL-8)]; // red pixel
   g=gamma[6*(get_raw_pixel((x/2)*2,(y/2)*2)>>(CAM_SENSOR_BITS_PER_PIXEL-8))/10]; // green pixel
   b=gamma[get_raw_pixel((x/2)*2+1,(y/2)*2)>>(CAM_SENSOR_BITS_PER_PIXEL-8)]; //blue pixel
"""""

about speed:

How many MHZ have this ARM CPU on Canon Camera ?

lets say the CPU have at least 500 MHZ.300 000 calls with each function cost 30 clock cycle cost then 9 Million clock cycles.

so the 500 MHZ CPU need 0.02 sec to process this.

« Last Edit: 15 / December / 2010, 11:44:54 by Bernd R »

Logged

Ixus 1000 HS