Movie mode framebuffers

  • 33 Replies
  • 4035 Views
*

Offline srsa_4c

  • ******
  • 3165
Re: Movie mode framebuffers
« Reply #20 on: 15 / November / 2011, 15:23:20 »
Advertisements
Just to be clear, you have taken the live buffer 411 image and saved it as a JPG image ?
I figured out (most of) the parameters of the function that instructs the co-processor to take an (yuv, uncompressed) image buffer and compress its content to another buffer. As this function is called from the movie code, it of course emits an mjpeg frame. I simply saved the result (with also adding back that missing "table").

The Y411 example's chroma looks bad because it hasn't been interpolated.
Quote
I know nothing about Huffmann tables,
I'm also not an expert of image processing, so I've looked it up http://en.wikipedia.org/wiki/JPEG
Quote
was that data on the stack and did it simply need attaching to one end of the 411 data ?
No, but I was curious, and found this: http://alexmogurenko.com/blog/programming/mjpeg-to-jpeg-cpp-csharp-delphi/

Take the Wikipedia article, look at the start of a .jpg file with a hex editor (skip the exif), and you'll see the "segments" and their "markers".

...what srsa_4c is doing with mjpeg compression here is light years ahead
Uhhh. I wouldn't say that. I just found that function call. It's in a part of the movie code that "we" regularly use in the ports.
Quote
There's also edmac (no idea what this is).
My guess is that it's a nickname of the compressor engine. The first google hit I get for this is a company website that sells air compressor parts. Maybe I'm wrong, and it's just a coincidence :)
Quote
Maybe this also helps a tiny bit:
http://magiclantern.wikia.com/wiki/ASM_Zedbra
Thx. Now I have a lot to read.

update: Edmac: not an air compressor, it's probably Engine DMA Controller. Inspired by http://magiclantern.wikia.com/wiki/DMA
« Last Edit: 07 / January / 2012, 20:42:15 by srsa_4c »

Re: Movie mode framebuffers
« Reply #21 on: 15 / November / 2011, 16:22:28 »
Checkout patents by Canon employees Satoshi Naito and Yukio Chiba. For example :-

http://www.google.com/patents?id=Hf1_AAAAEBAJ&printsec=abstract

*

Offline srsa_4c

  • ******
  • 3165
Re: Movie mode framebuffers
« Reply #22 on: 18 / November / 2011, 20:31:51 »
Small update.

Contrary to what I wrote earlier here, I have been unsuccessful trying to crash the camera for days now. I can shoot hundreds of pictures in a row (or doing it for hours taking one picture every 15 seconds), and then record a short movie or switch modes without any problem.
The code seems to have become stable when I added the function call to shutdown the jpeg engine.
There was another problem with some pictures getting corrupted: it seems to be gone now (the code now waits for 25 mseconds instead of 20).
I have also found the source for the "clock" that makes the camera add a new video frame when recording a movie: The movie code has a routine to send a "6" (along with the address of the current source buffer and the source image's dimensions) to the main movie loop. This routine's address is written into a pointer in Liveimage.c at start of a movie recording. The liveimage code then calls it for every new frame. Also in the liveimage code, I have found the active buffer's address - it turned out to be the same as our vid_get_viewport_live_fb(). But using that resulted in heavy tearing, probably because my routine is not synchronized to the liveimage code. Using another buffer made the tearing completely disappear. Will continue testing...

Re: Movie mode framebuffers
« Reply #23 on: 18 / November / 2011, 22:06:15 »
Very interesting, good work.

Quote
it turned out to be the same as our vid_get_viewport_live_fb()

When you have time a detailed explanation of how you found that would be of interest.



*

Offline srsa_4c

  • ******
  • 3165
Re: Movie mode framebuffers
« Reply #24 on: 19 / November / 2011, 09:40:23 »
@Microfunguy:
Please note: this is for A420, DIGIC II, VxWorks, with its shared movie / liveview buffers, the code may be different in later models
Code: [Select]
movierecordtask's main loop
...
STMFD  SP!, {R4,LR}
SUB    SP, SP, #4      @ one local variable
MOV    R4, SP
B      loc_FFD1BD84
 loc_FFD1BCE0:
LDR    R3, =0x72060
LDR    R2, [R3]
CMP    R2, #0
BNE    loc_FFD1BD70
SUB    R3, R12, #2
CMP    R3, #9
LDRLS  PC, [PC,R3,LSL#2]
B      loc_FFD1BD70
.long  loc_FFD1BD30
.long  loc_FFD1BD48
.long  loc_FFD1BD50
.long  loc_FFD1BD58
.long  loc_FFD1BD38
.long  loc_FFD1BD60
.long  loc_FFD1BD40
.long  loc_FFD1BD70
.long  loc_FFD1BD68
.long  loc_FFD1BD28
 loc_FFD1BD28:           @ msg 0xb, first msg on a new recording
BL     sub_FFD1BE20    @ see below
B      loc_FFD1BD6C
 loc_FFD1BD30:           @ msg 0x2, 2nd msg on a new recording
BL     unlock_optical_zoom
BL     sub_FFD1BFD4_my @ to time limit override
B      loc_FFD1BD6C
 loc_FFD1BD38:           @ msg 0x6, on every frame when recording
BL     sub_FFD1C3E4_my @ usual chdk override
B      loc_FFD1BD6C
 loc_FFD1BD40:
BL     sub_FFD1C8C4
B      loc_FFD1BD6C
 loc_FFD1BD48:           @ msg 0x3, first msg on stopping a recording
BL     sub_FFD1C6E0
B      loc_FFD1BD6C
 loc_FFD1BD50:           @ msg 0x4, next msg on stopping a recording (after a 0x6)
BL     sub_FFD1CA30
B      loc_FFD1BD6C
 loc_FFD1BD58:           @ msg 0x5, last msg on stop, .thm file creation
BL     sub_FFD1CBF4
B      loc_FFD1BD6C
 loc_FFD1BD60:
BL     sub_FFD1C94C
B      loc_FFD1BD6C
 loc_FFD1BD68:
BL     sub_FFD1C730
 loc_FFD1BD6C:
LDR    R1, [SP]
 loc_FFD1BD70:
LDR    R3, =0x71F94
MOV    R2, #0
STR    R2, [R1]
LDR    R0, [R3]
BL     sub_FFC104DC    @ postmessagequeue
 loc_FFD1BD84:
LDR    R3, =0x71F90    @ the main movie loop reads its messages from this msg queue
MOV    R1, R4          @ sp
LDR    R0, [R3]
MOV    R2, #0
BL     sub_FFC100C4    @ receivemessagequeue
LDR    R0, [SP]
LDR    R12, [R0]
CMP    R12, #0xC
MOV    R1, R0
BNE    loc_FFD1BCE0
...

With some debugging, I could find out the message structure movierecordtask's main loop gets:
5 words: (msg ID), (source buffer's address), (source picture width in pixels), (source picture height in pixels), 0?
The 2nd...5th words have probably other meaning when the message ID is not 6

sub_FFD1BEE8:   STMFD   SP!, {R4-R8,LR}
...
LDR    R7, =0x72078
LDR    R1, =0x720B0
LDR    R12, [R7]
LDR    R3, =0x720CC
LDR    LR, =0x720D0
LDR    R5, [R1,R12,LSL#2] @ source movie buffer's address
LDR    R6, [R3]         @ source picture width in pixels
ADD    R12, R12, #1
LDR    R4, [LR]         @ source picture height in pixels
MOV    R3, #6
CMP    R12, #3
STR    R3, [R0]         @ message ID (6)
MOV    R2, R8
MOV    R1, R0           @ message "location"
STMIB  R0, {R5,R6}      @ message's addr +4, +8
STR    R4, [R0,#0xC]    @ message's addr + 12
STR    R12, [R7]
STRGT  R8, [R7]
LDR    R3, =0x71F90     @ memory holding the queue ID
LDR    R0, [R3]
BL     sub_FFC104DC     @ postmessagequeue, 6 gets posted to the movierecord loop: new frame has arrived
...

Problem: the above 0x720xx addresses are only populated during movie recording
so, let's search for 0x72078 and 0x720b0 in the code
relevant places:

 sub_FFD1BE20:   STMFD  SP!, {R4,R5,LR}
LDR    LR, =0x72064
MOV    R3, #0
LDR    R12, =locret_FFD1C854
LDR    R5, =0x72078       @ !
LDR    R4, =0x7212C
STR    R3, [LR]
LDR    R2, =sub_FFD1BEE8 @ this is the address of the routine above !
LDR    R0, =sub_FFD1BE84 @ notes the active buffer for sub_ffd1bee8
LDR    R1, =0x720F0
STR    R3, [R5]
STR    R12, [R4]
BL     sub_FFC8DF04       @ see below
LDR    R2, =0x7207C
MOV    R3, #1
STR    R3, [R2]
LDMFD  SP!, {R4,R5,PC}


 sub_FFD1BE84:   STMFD  SP!, {R4-R8,LR} @ r0=0x34148 (it's known, see below)
LDR    R3, =0x72078    @ !
LDR    R8, [R0,#8]
LDR    R4, [R0]        @ 0x720f0
LDR    R2, [R0,#4]
LDR    R7, [R3]
LDR    R12, [R0,#0xC]
LDR    R6, =0x720B0    @ !
LDR    R5, =0x720EC
LDR    LR, =0x720CC
LDR    R1, =0x720D0
LDR    R0, =0x72064
MOV    R3, #1
STR    R4, [R6,R7,LSL#2]
STR    R2, [LR]
STR    R12, [R5]
STR    R8, [R1]
STR    R3, [R0]
LDMFD  SP!, {R4-R8,PC}


 sub_FFC8DF04:   STR    LR, [SP,#-4]!   @ in liveimage.c
LDR    R3, =0x34144
LDR    R12, =0x34148
LDR    LR, =0x3414C
STR    R0, [R3]        @ [0x34144] will hold sub_FFD1BE84's address
STR    R1, [R12]       @ [0x34148] will hold 0x720F0
STR    R2, [LR]        @ [0x3414C] will hold sub_FFD1BEE8's address
LDR    PC, [SP],#4


 sub_FFC8C054:   STMFD   SP!, {R4-R11,LR}
...
LDR    R3, =0x4FF0
LDR    R2, =0x340D8
LDRB   R0, [R3]
LDRB   R1, [R2]        @ it's 3 in a memdump, can be set to 3 or 4 by the firmware
ADD    R0, R0, R1
SUB    R0, R0, #1      @ r0=r0+3-1
BL     __modsi3        @ r0=r0 mod 3
LDR    R7, =0x34148    @ !
LDR    R3, =0x4FE0     @ 4 buffer addresses are here (copied there from the ROM, hint: ["start of data"] - 0x8 + 0x4fe0)
LDR    R1, [R7]        @ r1=[0x34148]
LDR    R2, [R3,R0,LSL#2] @ r2 = the selected buffer's address
LDR    R4, =0x34108
STR    R2, [R1]        @ [0x720f0]=r2
...
LDR    R3, =0x34144    @ !
LDR    R3, [R3]        @ sub_FFD1BE84 during active movie record
CMP    R3, #0          @ NULL pointer check
BEQ    loc_FFC8C3DC
LDR    R0, [R7]        @ r0=[0x34148]
MOV    LR, PC
MOV    PC, R3          @ call it!
 loc_FFC8C3DC:   LDR    R3, =0x340D4
LDR    R2, [R3]
CMP    R2, #1
BNE    loc_FFC8C404
LDR    R3, =0x3414C    @ !
LDR    R3, [R3]        @ sub_FFD1BEE8 during active movie record
CMP    R3, #0          @ NULL pointer check
BEQ    loc_FFC8C404
MOV    LR, PC
MOV    PC, R3          @ call it!
...
LDRNE  R0, =0xFFC8BBA8 @ "LiveImage.c" -> we're here (string for debugassert)
...
LDMFD  SP!, {R4-R11,PC}

So, to summarize: If you'd only like to find vid_get_viewport_live_fb(), you better follow waterwingz's advice here.

Re: Movie mode framebuffers
« Reply #25 on: 19 / November / 2011, 17:26:37 »
@Microfunguy:
Please note: this is for A420, DIGIC II, VxWorks

I have a pair of A620's.

I am also interested in the relevance of the JPG clock, whatever that is used for.

I am just wondering if any functions can change the frequency for the DSP that somehow determines the frame-rate.

Ideally, I would like to bring two cameras into synch during movie mode.


*

Offline srsa_4c

  • ******
  • 3165
Re: Movie mode framebuffers
« Reply #26 on: 19 / November / 2011, 18:46:42 »
I have a pair of A620's.
I am also interested in the relevance of the JPG clock, whatever that is used for.
I am just wondering if any functions can change the frequency for the DSP that somehow determines the frame-rate.
Ideally, I would like to bring two cameras into synch during movie mode.
I should have expressed myself a bit more clear and not use the word "clock". The so-called clock determines the movie framerate here. The movie's frame compression has to be synchronized to the liveview. That is what my post is about.

What you need is to finally catch the point where the CCD's and A/D converter's settings change.  My suspicion is that somehow imagesensortask should be involved. But that's it, I don't know what happens between my keypress and the movie format change... It could be inter-task messages, but it could also be some other mechanism (like some indirect function call).

You've probably noticed, that when you change the "movie" format (resolution and/or framerate), the liveview halts for a short time. How about switching back & forth between a still image and a movie mode? The mode switch causes some kind of initialization, which may affect liveview's phase.

And another thing: I don't think the DSP's and A/D & CCD's clocks are related. The DSP's clock is only activated when compression/decompression begins, and gets shut down right after it finishes (function names like "EnableEngineClocks" and "DisableEngineClocks").

*

Offline srsa_4c

  • ******
  • 3165
Re: Movie mode framebuffers
« Reply #27 on: 26 / November / 2011, 09:26:06 »
Just a quick note:
In case of the A420, MovieRecordTask and AviWriteTask are absent in movie mode when no recording is in progress, so no conflict with those can happen...


Re: Movie mode framebuffers
« Reply #28 on: 26 / November / 2011, 10:07:40 »
MovieRecordTask and AviWriteTask are absent

How do you detect that ?

*

Offline srsa_4c

  • ******
  • 3165
Re: Movie mode framebuffers
« Reply #29 on: 26 / November / 2011, 10:31:23 »
MovieRecordTask and AviWriteTask are absent

How do you detect that ?
Well, there's already an on-screen tasklist as a debug utility in CHDK (which is only implemented for VxWorks ATM), I have made another function from its code that dumps the tasklist to a file. I've also modified the debug shortcut code, so now I have DmpTsks besides DmpRAM etc.
Now I can dump the names of the tasks which are currently active.
For DryOS, one could hack the code of the overridden tasks to signal when they start / quit.

 

Related Topics