Attempting to find color matrix from firmware

fudgey

1705
a570is

« on: 12 / January / 2009, 17:27:25 »

Advertisements

For vxworks cameras, look for the string COLORMATRIX.
In a570is there's code like this, but its found in other vxworks cameras just the same with different addresses:

Code: (asm) [Select]

ffe0b498: 	e1a00004 	mov	r0, r4
ffe0b49c: 	ebfdf08c 	bl	ffd876d4 <_binary_PRIMARY_BIN_start+0x1876d4 -540104>
ffe0b4a0: 	e1a01004 	mov	r1, r4
ffe0b4a4: 	e3a0203c 	mov	r2, #60	; 0x3c
ffe0b4a8: 	e59f000c 	ldr	r0, [pc, #12]	; ffe0b4bc: (ffe0b378)  **"COLORMATRIX"

ffd876d4: 	e52de004 	push	{lr}		; (str lr, [sp, #-4]!)
ffd876d8: 	e49de004 	pop	{lr}		; (ldr lr, [sp], #4)
ffd876dc: 	ea0003a7 	b	ffd88580 <_binary_PRIMARY_BIN_start+0x188580 +3748>

ffd88580: 	e92d4010 	push	{r4, lr}
ffd88584: 	e1a04000 	mov	r4, r0
ffd88588: 	ebffff4a 	bl	ffd882b8 <_binary_PRIMARY_BIN_start+0x1882b8 -720>

  ffd882b8: 	e59f3004 	ldr	r3, [pc, #4]	; ffd882c4: (000066a0) 
  ffd882bc: 	e5930000 	ldr	r0, [r3]
  ffd882c0: 	e1a0f00e 	mov	pc, lr

ffd8858c: 	e5902080 	ldr	r2, [r0, #128]
ffd88590: 	e5842000 	str	r2, [r4]

etc. The code that continues here reads 15 values from memory and changes the sign of 4 of them. The number of coefficients is similar to 18 values in a color matrix, which also has exactly 4 negative coefficients.

A small Lua script to print those values (don't run it if you don't understand this, without changing the address it's only for a570).

Code: (lua) [Select]

--[[
@title peek for colormatrix
@param a script enable
@default a 0
--]]

-- prevent accidental run by careless people
if a>0 then
  addr=0x66a0
  offs=peek(addr)
  print(addr.."="..offs)   print(tostring(peek(offs+128)),"-",tostring(peek(offs+128)),"-",tostring(peek(offs+132)),tostring(peek(offs+132)),tostring(peek(offs+136)))
  print(tostring(peek(offs+140)),tostring(peek(offs+144)),tostring(peek(offs+148)),tostring(peek(offs+152)),"-",tostring(peek(offs+152)))
  print("-",tostring(peek(offs+156)),tostring(peek(offs+156)),tostring(peek(offs+456)),tostring(peek(offs+480)),tostring(peek(offs+504)))
  wait_click(0)
end

The result is always 1 -1 -170 170 2 166 4 201 4 -4 -138 138 4 0 0, which doesn't look like a color matrix even though as PlasmaHH says: "matrix converts rawRGB to XYZ and not to sRGB so the values might be unusual to us".
I tried switching ISO and WB and still/movie mode, always shooting after changing things. I didn't try to read that during shooting.

There are a couple of promising function names (RegistColorMatrix and UnregistColorMatrix) but they appear to be hard to grasp.

Any ideas on how to find those or find use for the values above or anything else related for that matter?

edit: Fixed the script, see more recent posts for more.

« Last Edit: 24 / January / 2009, 05:16:47 by fudgey »

Logged

whoever

280
IXUS950

Re: Attempting to find color matrix from firmware

« Reply #1 on: 14 / January / 2009, 03:49:10 »

The numbers in Ixus950 are identical to yours, so they cannot be the sensor color matrix. Otherwise, as you mentioned, it is hard to understand what the code does and where it is called from. Perhaps it is impossible to understand without the access to DSP code, which is certainly unfathomable -- where is it anyway?

Logged

ewavr

1057
A710IS

Re: Attempting to find color matrix from firmware

« Reply #2 on: 14 / January / 2009, 13:37:48 »

Quote from: whoever on 14 / January / 2009, 03:49:10

Perhaps it is impossible to understand without the access to DSP code, which is certainly unfathomable -- where is it anyway?

IMO, this code is located not far from string:
for ixus950: " jp56_firmware build#0094 2005/11/12 Saku Hiwatashi, Satoshi Naito, Yukio Chiba"
for a710: "jp52_firmware build#0015 2006/04/04 Akihiro Uehara, Satoshi Naito, Yukio Chiba"

Logged

sharky

65

Re: Attempting to find color matrix from firmware

« Reply #3 on: 23 / January / 2009, 03:32:01 »

I have been looking at the COLORMATRIX strings and code in the firmware of my ixus85

What I have found is that there seems to be a boot procedure which seems to load some compressed (?) data from 0xfffa0000 into the RAM. This is then indexed from several functions, including a function that seems to load 0x3c (60) bytes from the uncompressed data, and then calls a function with

some_function("COLORMATRIX", buffer, sizeof(buffer) /* 0x3c */);

When I dump this data, I get 15 integers 867 -867 -215 215 867 215 848 279 848 -848 -279 279 76 129 55

I guess the first 12 could be a color matrix, with the last three being multipliers (76, 129, 55 looks like it COULD be inverse multipliers for R, G, B)

I have made some assumptions (I am obviously unsure of my function names), which then shows:

Code: [Select]

ROM:FFA80EE4 RegisterColorMatrix                     ; DATA XREF: ROM:off_FFA856B8o
ROM:FFA80EE4                                         ; ROM:FFB20D58o
ROM:FFA80EE4
ROM:FFA80EE4 var_4           = -4
ROM:FFA80EE4 arg_4           =  4
ROM:FFA80EE4
ROM:FFA80EE4                 STR     LR, [SP,#var_4]!
ROM:FFA80EE8                 SUB     SP, SP, #0x3C
ROM:FFA80EEC                 MOV     R0, SP
ROM:FFA80EF0                 MOV     R1, #0x3C
ROM:FFA80EF4                 BL      MemZero
ROM:FFA80EF8                 MOV     R0, SP
ROM:FFA80EFC                 BL      j_GetCurrentMatrix
ROM:FFA80F00                 LDR     R0, =paColorMatrix
ROM:FFA80F04                 MOV     R2, #0x3C
ROM:FFA80F08                 LDR     R0, [R0]
ROM:FFA80F0C                 MOV     R1, SP
ROM:FFA80F10                 BL      SetSensorParam
ROM:FFA80F14                 ADD     SP, SP, #0x3C
ROM:FFA80F18                 LDR     PC, [SP],#arg_4 ; ret
ROM:FFA80F18 ; End of function RegisterColorMatrix

Logged

sharky

65

Re: Attempting to find color matrix from firmware

« Reply #4 on: 23 / January / 2009, 04:38:51 »

Quote from: fudgey on 12 / January / 2009, 17:27:25

Code: [Select]
if a>0 then offs=0x66a0

Aren't you missing an indirection here? In my ROM, I read a value from 0xb344, take that pointer (normally 1b7404), then add 0x80, 0x84, etc to THAT pointer.

Indeed, that is what your firmware is doing too

Code: [Select]

  ffd882b8: 	e59f3004 	ldr	r3, [pc, #4]	; ffd882c4: (000066a0) 
  ffd882bc: 	e5930000 	ldr	r0, [r3]
  ffd882c0: 	e1a0f00e 	mov	pc, lr
  ffd8858c: 	e5902080 	ldr	r2, [r0, #128]

« Last Edit: 23 / January / 2009, 09:36:50 by sharky »

Logged

sharky

65

Re: Attempting to find color matrix from firmware

« Reply #5 on: 23 / January / 2009, 14:24:36 »

I have found something that may be a coincedence, and may be something

When I apply the last three values posted before, 76 129 55, as inverse factors over the, R, G, B sensor data, it scales the values neatly to reach an almost exact full range of values; so

Code: [Select]

R = Rsensor * (255/76)
G = Gsensor * (255/129)
B = Bsensor * (255/55)

The basically brings all the levels up to the correct values. (normally B and R are quite a bit too dark)

This is not a full color matrix obviously, but it looks like it could be right. Is this what they call 'daylight multipliers' maybe ?

Logged

whoever

280
IXUS950

Re: Attempting to find color matrix from firmware

« Reply #6 on: 23 / January / 2009, 14:37:53 »

Quote from: sharky on 23 / January / 2009, 04:38:51

In my ROM, I read a value from 0xb344, take that pointer (normally 1b7404), then add 0x80, 0x84, etc to THAT pointer.

Ah yes... In which case, in my RAM I read [0x8D30]=0x1F77CC, and then [0x1F77CC+0x80], etc., gives 821,-30,331,181,202,310,-155,-994,821,-30,331,181,202,310,-155,...

Nice... But what do these numbers mean? It may well be that "COLORMATRIX" notion of Canon FW has no counterpart in any human language...

Quote from: sharky on 23 / January / 2009, 14:24:36

Is this what they call 'daylight multipliers' maybe ?

Why don't you take a shot of gray card in daylight? Then, "daylight multipliers" are the inverse of the camera R,G,B values.

Logged

sharky

65

Re: Attempting to find color matrix from firmware

« Reply #7 on: 23 / January / 2009, 14:52:06 »

Quote from: whoever on 23 / January / 2009, 14:37:53

Nice... But what do these numbers mean? It may well be that "COLORMATRIX" notion of Canon FW has no counterpart in any human language...

Well, I think they must use a DSP chip to do various color matrix mapping + RGB -> YUV conversion to save the JPG; the ARM processor can never generate a 10MP JPG as quickly as it does. So there must be some kind of semi-logical interface between the firmware and the DSP chip. I'm guessing that the RegisterColorMatrix is sending the color matrix to the DSP system, and presumably that's in some kind of format that makes some sense

Logged

whoever

280
IXUS950

Re: Attempting to find color matrix from firmware

« Reply #8 on: 23 / January / 2009, 15:01:37 »

Right. But what kind of matrix is it, what does it do? There's a lot of possible permutations of the 15 values that you find... And I'm afraid that the DSP code may be undecipherable, almost surely custom made Canon proprietary...

Edit:
Now, here is a nice matrix for my cam that converts a column D65 XYZ vector to camera RGB vector, as per DNG specification:
1.4573 -0.5482 -0.1546 -0.1266 0.9799 0.1468 -0.1040 0.1912 0.3810
It has 9 numbers, of which 8 are significant, the overall luminosity being a free parameter. Now, the matrix will be different for any other reference white point. It also will be different if presented as transposed or inversed. It can be presented as a transform to/from CIE Lab rather than XYZ. Etc., etc. That all makes me rather skeptical about interpreting the numbers in FW. But, with certain skills and luck...

« Last Edit: 23 / January / 2009, 15:40:45 by whoever »

Logged

sharky

65

Re: Attempting to find color matrix from firmware

« Reply #9 on: 24 / January / 2009, 02:52:10 »

I'm still optimistic that we'll be able to find something.

Even if you provided me with your matrix as just 8 numbers in inverse or transposed I would be able to see that quite easily since the matrix is always not THAT far from a identity matrix, since the sensor is not THAT far from being direct XYZ anyway.. So we're always looking for something like

Code: [Select]

1 0 0
0 1 0
0 0 1

Your matrix matches that quite nicely.

Anyway, time will tell whether we can find something like that

Logged