Movie mode framebuffers

  • 33 Replies
  • 3944 Views
*

Offline srsa_4c

  • ******
  • 3132
  • Publish
    Re: Movie mode framebuffers
    « Reply #20 on: 15 / November / 2011, 15:23:20 »
    Advertisements
    Just to be clear, you have taken the live buffer 411 image and saved it as a JPG image ?
    I figured out (most of) the parameters of the function that instructs the co-processor to take an (yuv, uncompressed) image buffer and compress its content to another buffer. As this function is called from the movie code, it of course emits an mjpeg frame. I simply saved the result (with also adding back that missing "table").

    The Y411 example's chroma looks bad because it hasn't been interpolated.
    Quote
    I know nothing about Huffmann tables,
    I'm also not an expert of image processing, so I've looked it up http://en.wikipedia.org/wiki/JPEG
    Quote
    was that data on the stack and did it simply need attaching to one end of the 411 data ?
    No, but I was curious, and found this: http://alexmogurenko.com/blog/programming/mjpeg-to-jpeg-cpp-csharp-delphi/

    Take the Wikipedia article, look at the start of a .jpg file with a hex editor (skip the exif), and you'll see the "segments" and their "markers".

    ...what srsa_4c is doing with mjpeg compression here is light years ahead
    Uhhh. I wouldn't say that. I just found that function call. It's in a part of the movie code that "we" regularly use in the ports.
    Quote
    There's also edmac (no idea what this is).
    My guess is that it's a nickname of the compressor engine. The first google hit I get for this is a company website that sells air compressor parts. Maybe I'm wrong, and it's just a coincidence :)
    Quote
    Maybe this also helps a tiny bit:
    http://magiclantern.wikia.com/wiki/ASM_Zedbra
    Thx. Now I have a lot to read.

    update: Edmac: not an air compressor, it's probably Engine DMA Controller. Inspired by http://magiclantern.wikia.com/wiki/DMA
    « Last Edit: 07 / January / 2012, 20:42:15 by srsa_4c »

  • Publish
    Re: Movie mode framebuffers
    « Reply #21 on: 15 / November / 2011, 16:22:28 »
    Checkout patents by Canon employees Satoshi Naito and Yukio Chiba. For example :-

    http://www.google.com/patents?id=Hf1_AAAAEBAJ&printsec=abstract

    *

    Offline srsa_4c

    • ******
    • 3132
  • Publish
    Re: Movie mode framebuffers
    « Reply #22 on: 18 / November / 2011, 20:31:51 »
    Small update.

    Contrary to what I wrote earlier here, I have been unsuccessful trying to crash the camera for days now. I can shoot hundreds of pictures in a row (or doing it for hours taking one picture every 15 seconds), and then record a short movie or switch modes without any problem.
    The code seems to have become stable when I added the function call to shutdown the jpeg engine.
    There was another problem with some pictures getting corrupted: it seems to be gone now (the code now waits for 25 mseconds instead of 20).
    I have also found the source for the "clock" that makes the camera add a new video frame when recording a movie: The movie code has a routine to send a "6" (along with the address of the current source buffer and the source image's dimensions) to the main movie loop. This routine's address is written into a pointer in Liveimage.c at start of a movie recording. The liveimage code then calls it for every new frame. Also in the liveimage code, I have found the active buffer's address - it turned out to be the same as our vid_get_viewport_live_fb(). But using that resulted in heavy tearing, probably because my routine is not synchronized to the liveimage code. Using another buffer made the tearing completely disappear. Will continue testing...

  • Publish
    Re: Movie mode framebuffers
    « Reply #23 on: 18 / November / 2011, 22:06:15 »
    Very interesting, good work.

    Quote
    it turned out to be the same as our vid_get_viewport_live_fb()

    When you have time a detailed explanation of how you found that would be of interest.



    *

    Offline srsa_4c

    • ******
    • 3132
  • Publish
    Re: Movie mode framebuffers
    « Reply #24 on: 19 / November / 2011, 09:40:23 »
    @Microfunguy:
    Please note: this is for A420, DIGIC II, VxWorks, with its shared movie / liveview buffers, the code may be different in later models
    Code: [Select]
    movierecordtask's main loop
    ...
    STMFD  SP!, {R4,LR}
    SUB    SP, SP, #4      @ one local variable
    MOV    R4, SP
    B      loc_FFD1BD84
     loc_FFD1BCE0:
    LDR    R3, =0x72060
    LDR    R2, [R3]
    CMP    R2, #0
    BNE    loc_FFD1BD70
    SUB    R3, R12, #2
    CMP    R3, #9
    LDRLS  PC, [PC,R3,LSL#2]
    B      loc_FFD1BD70
    .long  loc_FFD1BD30
    .long  loc_FFD1BD48
    .long  loc_FFD1BD50
    .long  loc_FFD1BD58
    .long  loc_FFD1BD38
    .long  loc_FFD1BD60
    .long  loc_FFD1BD40
    .long  loc_FFD1BD70
    .long  loc_FFD1BD68
    .long  loc_FFD1BD28
     loc_FFD1BD28:           @ msg 0xb, first msg on a new recording
    BL     sub_FFD1BE20    @ see below
    B      loc_FFD1BD6C
     loc_FFD1BD30:           @ msg 0x2, 2nd msg on a new recording
    BL     unlock_optical_zoom
    BL     sub_FFD1BFD4_my @ to time limit override
    B      loc_FFD1BD6C
     loc_FFD1BD38:           @ msg 0x6, on every frame when recording
    BL     sub_FFD1C3E4_my @ usual chdk override
    B      loc_FFD1BD6C
     loc_FFD1BD40:
    BL     sub_FFD1C8C4
    B      loc_FFD1BD6C
     loc_FFD1BD48:           @ msg 0x3, first msg on stopping a recording
    BL     sub_FFD1C6E0
    B      loc_FFD1BD6C
     loc_FFD1BD50:           @ msg 0x4, next msg on stopping a recording (after a 0x6)
    BL     sub_FFD1CA30
    B      loc_FFD1BD6C
     loc_FFD1BD58:           @ msg 0x5, last msg on stop, .thm file creation
    BL     sub_FFD1CBF4
    B      loc_FFD1BD6C
     loc_FFD1BD60:
    BL     sub_FFD1C94C
    B      loc_FFD1BD6C
     loc_FFD1BD68:
    BL     sub_FFD1C730
     loc_FFD1BD6C:
    LDR    R1, [SP]
     loc_FFD1BD70:
    LDR    R3, =0x71F94
    MOV    R2, #0
    STR    R2, [R1]
    LDR    R0, [R3]
    BL     sub_FFC104DC    @ postmessagequeue
     loc_FFD1BD84:
    LDR    R3, =0x71F90    @ the main movie loop reads its messages from this msg queue
    MOV    R1, R4          @ sp
    LDR    R0, [R3]
    MOV    R2, #0
    BL     sub_FFC100C4    @ receivemessagequeue
    LDR    R0, [SP]
    LDR    R12, [R0]
    CMP    R12, #0xC
    MOV    R1, R0
    BNE    loc_FFD1BCE0
    ...

    With some debugging, I could find out the message structure movierecordtask's main loop gets:
    5 words: (msg ID), (source buffer's address), (source picture width in pixels), (source picture height in pixels), 0?
    The 2nd...5th words have probably other meaning when the message ID is not 6

    sub_FFD1BEE8:   STMFD   SP!, {R4-R8,LR}
    ...
    LDR    R7, =0x72078
    LDR    R1, =0x720B0
    LDR    R12, [R7]
    LDR    R3, =0x720CC
    LDR    LR, =0x720D0
    LDR    R5, [R1,R12,LSL#2] @ source movie buffer's address
    LDR    R6, [R3]         @ source picture width in pixels
    ADD    R12, R12, #1
    LDR    R4, [LR]         @ source picture height in pixels
    MOV    R3, #6
    CMP    R12, #3
    STR    R3, [R0]         @ message ID (6)
    MOV    R2, R8
    MOV    R1, R0           @ message "location"
    STMIB  R0, {R5,R6}      @ message's addr +4, +8
    STR    R4, [R0,#0xC]    @ message's addr + 12
    STR    R12, [R7]
    STRGT  R8, [R7]
    LDR    R3, =0x71F90     @ memory holding the queue ID
    LDR    R0, [R3]
    BL     sub_FFC104DC     @ postmessagequeue, 6 gets posted to the movierecord loop: new frame has arrived
    ...

    Problem: the above 0x720xx addresses are only populated during movie recording
    so, let's search for 0x72078 and 0x720b0 in the code
    relevant places:

     sub_FFD1BE20:   STMFD  SP!, {R4,R5,LR}
    LDR    LR, =0x72064
    MOV    R3, #0
    LDR    R12, =locret_FFD1C854
    LDR    R5, =0x72078       @ !
    LDR    R4, =0x7212C
    STR    R3, [LR]
    LDR    R2, =sub_FFD1BEE8 @ this is the address of the routine above !
    LDR    R0, =sub_FFD1BE84 @ notes the active buffer for sub_ffd1bee8
    LDR    R1, =0x720F0
    STR    R3, [R5]
    STR    R12, [R4]
    BL     sub_FFC8DF04       @ see below
    LDR    R2, =0x7207C
    MOV    R3, #1
    STR    R3, [R2]
    LDMFD  SP!, {R4,R5,PC}


     sub_FFD1BE84:   STMFD  SP!, {R4-R8,LR} @ r0=0x34148 (it's known, see below)
    LDR    R3, =0x72078    @ !
    LDR    R8, [R0,#8]
    LDR    R4, [R0]        @ 0x720f0
    LDR    R2, [R0,#4]
    LDR    R7, [R3]
    LDR    R12, [R0,#0xC]
    LDR    R6, =0x720B0    @ !
    LDR    R5, =0x720EC
    LDR    LR, =0x720CC
    LDR    R1, =0x720D0
    LDR    R0, =0x72064
    MOV    R3, #1
    STR    R4, [R6,R7,LSL#2]
    STR    R2, [LR]
    STR    R12, [R5]
    STR    R8, [R1]
    STR    R3, [R0]
    LDMFD  SP!, {R4-R8,PC}


     sub_FFC8DF04:   STR    LR, [SP,#-4]!   @ in liveimage.c
    LDR    R3, =0x34144
    LDR    R12, =0x34148
    LDR    LR, =0x3414C
    STR    R0, [R3]        @ [0x34144] will hold sub_FFD1BE84's address
    STR    R1, [R12]       @ [0x34148] will hold 0x720F0
    STR    R2, [LR]        @ [0x3414C] will hold sub_FFD1BEE8's address
    LDR    PC, [SP],#4


     sub_FFC8C054:   STMFD   SP!, {R4-R11,LR}
    ...
    LDR    R3, =0x4FF0
    LDR    R2, =0x340D8
    LDRB   R0, [R3]
    LDRB   R1, [R2]        @ it's 3 in a memdump, can be set to 3 or 4 by the firmware
    ADD    R0, R0, R1
    SUB    R0, R0, #1      @ r0=r0+3-1
    BL     __modsi3        @ r0=r0 mod 3
    LDR    R7, =0x34148    @ !
    LDR    R3, =0x4FE0     @ 4 buffer addresses are here (copied there from the ROM, hint: ["start of data"] - 0x8 + 0x4fe0)
    LDR    R1, [R7]        @ r1=[0x34148]
    LDR    R2, [R3,R0,LSL#2] @ r2 = the selected buffer's address
    LDR    R4, =0x34108
    STR    R2, [R1]        @ [0x720f0]=r2
    ...
    LDR    R3, =0x34144    @ !
    LDR    R3, [R3]        @ sub_FFD1BE84 during active movie record
    CMP    R3, #0          @ NULL pointer check
    BEQ    loc_FFC8C3DC
    LDR    R0, [R7]        @ r0=[0x34148]
    MOV    LR, PC
    MOV    PC, R3          @ call it!
     loc_FFC8C3DC:   LDR    R3, =0x340D4
    LDR    R2, [R3]
    CMP    R2, #1
    BNE    loc_FFC8C404
    LDR    R3, =0x3414C    @ !
    LDR    R3, [R3]        @ sub_FFD1BEE8 during active movie record
    CMP    R3, #0          @ NULL pointer check
    BEQ    loc_FFC8C404
    MOV    LR, PC
    MOV    PC, R3          @ call it!
    ...
    LDRNE  R0, =0xFFC8BBA8 @ "LiveImage.c" -> we're here (string for debugassert)
    ...
    LDMFD  SP!, {R4-R11,PC}

    So, to summarize: If you'd only like to find vid_get_viewport_live_fb(), you better follow waterwingz's advice here.

  • Publish
    Re: Movie mode framebuffers
    « Reply #25 on: 19 / November / 2011, 17:26:37 »
    @Microfunguy:
    Please note: this is for A420, DIGIC II, VxWorks

    I have a pair of A620's.

    I am also interested in the relevance of the JPG clock, whatever that is used for.

    I am just wondering if any functions can change the frequency for the DSP that somehow determines the frame-rate.

    Ideally, I would like to bring two cameras into synch during movie mode.


    *

    Offline srsa_4c

    • ******
    • 3132
  • Publish
    Re: Movie mode framebuffers
    « Reply #26 on: 19 / November / 2011, 18:46:42 »
    I have a pair of A620's.
    I am also interested in the relevance of the JPG clock, whatever that is used for.
    I am just wondering if any functions can change the frequency for the DSP that somehow determines the frame-rate.
    Ideally, I would like to bring two cameras into synch during movie mode.
    I should have expressed myself a bit more clear and not use the word "clock". The so-called clock determines the movie framerate here. The movie's frame compression has to be synchronized to the liveview. That is what my post is about.

    What you need is to finally catch the point where the CCD's and A/D converter's settings change.  My suspicion is that somehow imagesensortask should be involved. But that's it, I don't know what happens between my keypress and the movie format change... It could be inter-task messages, but it could also be some other mechanism (like some indirect function call).

    You've probably noticed, that when you change the "movie" format (resolution and/or framerate), the liveview halts for a short time. How about switching back & forth between a still image and a movie mode? The mode switch causes some kind of initialization, which may affect liveview's phase.

    And another thing: I don't think the DSP's and A/D & CCD's clocks are related. The DSP's clock is only activated when compression/decompression begins, and gets shut down right after it finishes (function names like "EnableEngineClocks" and "DisableEngineClocks").

    *

    Offline srsa_4c

    • ******
    • 3132
  • Publish
    Re: Movie mode framebuffers
    « Reply #27 on: 26 / November / 2011, 09:26:06 »
    Just a quick note:
    In case of the A420, MovieRecordTask and AviWriteTask are absent in movie mode when no recording is in progress, so no conflict with those can happen...


  • Publish
    Re: Movie mode framebuffers
    « Reply #28 on: 26 / November / 2011, 10:07:40 »
    MovieRecordTask and AviWriteTask are absent

    How do you detect that ?

    *

    Offline srsa_4c

    • ******
    • 3132
  • Publish
    Re: Movie mode framebuffers
    « Reply #29 on: 26 / November / 2011, 10:31:23 »
    MovieRecordTask and AviWriteTask are absent

    How do you detect that ?
    Well, there's already an on-screen tasklist as a debug utility in CHDK (which is only implemented for VxWorks ATM), I have made another function from its code that dumps the tasklist to a file. I've also modified the debug shortcut code, so now I have DmpTsks besides DmpRAM etc.
    Now I can dump the names of the tasks which are currently active.
    For DryOS, one could hack the code of the overridden tasks to signal when they start / quit.

     

    Related Topics