Movie mode framebuffers - page 2 - General Discussion and Assistance - CHDK Forum

Movie mode framebuffers

  • 33 Replies
  • 10408 Views
*

Offline srsa_4c

  • ******
  • 4430
Re: Movie mode framebuffers
« Reply #10 on: 13 / November / 2011, 07:34:30 »
Advertisements
Small update, since my goal was to capture live images.

Getting the framebuffer's content (and saving it into a file) is not a problem, when doing it uncompressed.
I also tried to use the mjpeg engine (actually, I've started with that). The firmware's movie record code calls some functions, which live in MjpegEncPass.c . The actual call that starts the encoding process is a few calls before the reference to the "JPEGTimeOut" and "JPEGICError" debug strings.
It can be found easily as it takes a lot of parameters. Unfortunately, that doesn't mean that it's independent from the rest of the movie code.
First, after figuring out most of its parameters, tried to call it (with parameters, from a custom function I wired into lua). That resulted in a nice crash. Then I had to realize, it needs some initialization (because using it after recording a short clip was possible). So I started to look around references to functions in MjpegEncPass.c, found some with initialization intent. Put those into my function, and things seemed to work. Until I tried to record a movie afterwards, which crashed the camera. Then I found out, that the engine is only started when the encode function is called, and it also needs to be stopped. The function that does this was a few lines down in the movie code (sub_FFC81050). I thought, that's it. Unfortunately, not. My function is pretty stable when used few times in a row (I call it from a one-shot script, saves the result to the card). But then, when it comes to switching modes or actual movie recording, the camera tends to crash here and there with various signs. Most of the romlogs seem to refer to imagesensortask, I even got an E16 (?) ccdfifo error once, and one time the crash made the camera to stop responding with screwed up & frozen liveview. My suspicion is that something interrupt-related is going on. That's the current status of my trial. I haven't used anything related to semaphores in my code, used simply a call to msleep() to wait for the compressor to finish.

Conclusion: the mjpeg encode function appears to depend on the code of movierecordtask too much, calling it seems to be a pretty complex job.

Some other notes: as the co-processor (how should I refer to it?) can only write to 32bit-aligned addresses, the movie code has to pull back the frame by 2 bytes when unaligned. The compressor generates correct frames by itself, but the movie code overwrites some of the frame headers (reorders some jpeg sections). The destination address points to the place of the next avi video frame. The encoder function chain decides about the compressed frame's resolution (there's a table in the ROM for that, but not very obvious).

The parameters of sub_FFC80EE4 (320x240 movie mode, while recording), might contain mistakes
r0: address of the current framebuffer
r1: 360, framebuffer width in pixels
r2: 191, framebuffer height in pixels
r3: destination address (compressed frame to be placed at, the co-processor can only write to 32bit aligned addresses!), useful mjpeg data begins at start+0x18 or start+0x1a with the current canon code (alignment is 2 bytes when recording a movie)
//passed on caller's stack:
[sp+0x00]: 0x20000, probably compressed framesize limit
[sp+0x04]: 0 ...?
[sp+0x08]: 0 ...?
[sp+0x0c]: pointer to avi vidframe size (on caller's stack), to be filled by the encoder routine
[sp+0x10]: [0x720c8] = 0xffc7bb54, looks like a pair of quantization tables
[sp+0x14]: [0x720d4], compression quality (0..99?)
[sp+0x18]: caller's sp+0x1c, gets set to 0 when compression was ok


All of the above happened on the A420.

edit: fixed the parameter descriptions
« Last Edit: 18 / November / 2011, 19:58:30 by srsa_4c »

Re: Movie mode framebuffers
« Reply #11 on: 13 / November / 2011, 09:43:43 »
Well, as far as I know that is the first time anyone has published information regarding the MJPEG engine.

Are you saying you can save the live buffer as a JPG ?

I wonder if it works for newer cameras that do not use MJPEG for movies ?

*

Offline srsa_4c

  • ******
  • 4430
Re: Movie mode framebuffers
« Reply #12 on: 13 / November / 2011, 11:21:19 »
Are you saying you can save the live buffer as a JPG ?
Yes, but the camera becomes unstable because of some missing bits of the puzzle.
It's also not complete jpeg, but the missing Huffmann-table can be added after compression.
Quote
I wonder if it works for newer cameras that do not use MJPEG for movies ?
As all cameras can compress to jpeg, this might be possible. But you can forget about the original movie record routine, its "codec" is fixed, most probably.

A possible problem is the different colorspaces image formats. Liveview buffers are usually Y411, DIGIC II movie buffers (the ones I've seen) are also Y411. Still images get downsized (when the user instructs the camera to shoot in for example "S" size) in a different format (UYVY?). My DIGIC III SX100IS has UYVY movie buffers.
« Last Edit: 13 / November / 2011, 15:45:22 by srsa_4c »

*

Offline srsa_4c

  • ******
  • 4430
Re: Movie mode framebuffers
« Reply #13 on: 15 / November / 2011, 09:23:01 »
Some update.

Using my temporary function(s), I'm now testing scripts which "shoot" pictures in a loop. No crash so far (with 100 pictures in a row), when I don't attempt to change modes or record a movie. The issue with mjpeg (regular image viewers can't decode them) has been solved, my routine is now injecting the required Huffmann-table into the file being saved.
I've found another spot in the firmware which applies jpeg-compression. It's also in movierecordtask, at the end of a regular recording, the camera creates a .thm file with a thumbnail (160x120) image of the first video frame. The function (which starts compression) has much less parameters, as its duty is limited to create a single 160x120 jpeg image. It uses a 160x120 y411 source buffer with a 160x120 image in it (so, it's different from the usual buffers, and its location is not fixed). I've copied and hacked this functon to use a 720x568 buffer. It even works, but the resulting jpeg has strange chroma artifacts (I've only changed the obvious 160, 120 values to 720 and 568).
This function is simpler, calls less functions. The majority of those calls (they end up in EdMac.c EDmac.c, IIRC) can be found in the "mjpeg" encoder function. So, there isn't really such thing as separate mjpeg and jpeg engine. Using the thumbnail-compressor routine results in the same general unstability as using the mjpeg one.
« Last Edit: 17 / November / 2011, 08:58:52 by srsa_4c »


Re: Movie mode framebuffers
« Reply #14 on: 15 / November / 2011, 10:22:22 »
Good work !

Look forward to seeing some sample images.

*

Offline Coutts

  • *****
  • 538
  • www.flickr.com/couttsphotog
    • Flickr
Re: Movie mode framebuffers
« Reply #15 on: 15 / November / 2011, 12:57:01 »
this is implemented in magic lantern already :)

it's the silent pictures feature. we have located the image buffers and determined the resolution of the images in RAM using a script (i think indy wrote it) called img.py.

The silent pictures are saved in .422 format and converted with 422-jpg.py on a computer. In recent unified builds of magic lantern, alex has implemented a method to view the silent pictures in play mode (a sort of on-the-fly 422-jpg, only for preview purposes).


https://bitbucket.org/hudson/magic-lantern/src/aa106d604a99/src/shoot.c#cl-1200

That link will put you at the funtion used for the silent pictures. The actual dump code is at line 1222.
« Last Edit: 15 / November / 2011, 13:02:51 by Coutts »
Canon 5d
Canon 50mm f/1.8
Sigma 24mm f/1.8

Flickr

*

Offline Coutts

  • *****
  • 538
  • www.flickr.com/couttsphotog
    • Flickr
Re: Movie mode framebuffers
« Reply #16 on: 15 / November / 2011, 12:57:54 »
tutorial for using img.py (and where to get it) here:

http://magiclantern.wikia.com/wiki/VRAM/550D
Canon 5d
Canon 50mm f/1.8
Sigma 24mm f/1.8

Flickr

*

Offline srsa_4c

  • ******
  • 4430
Re: Movie mode framebuffers
« Reply #17 on: 15 / November / 2011, 13:46:07 »
@Microfunguy:
Y411 (chroma may
not look entirely correct)
720x568
compressed with the
mjpg code
640x480
compressed with the
.thm code (1)
720x568
compressed with the
.thm code (2)
720x568

it's the silent pictures feature. we have located the image buffers and determined the resolution of the images in RAM using a script (i think indy wrote it) called img.py.
Thanks for this info, I haven't expected the point&shoots and the DSLR's having this stuff (kind of) common. Your links look very useful.

Quote
The silent pictures are saved in .422 format and converted with 422-jpg.py on a computer.
That probably means the ML developers haven't tried to use the camera's compression engine for this purpose, I guess (haven't looked at the code yet).


Re: Movie mode framebuffers
« Reply #18 on: 15 / November / 2011, 14:27:09 »
Just to be clear, you have taken the live buffer Y411 image and saved it as a JPG image ?
I know nothing about Huffmann tables, was that data on the stack and did it simply need attaching to one end of the Y411 data ?

Normally, liveview images are 'double width'.

« Last Edit: 15 / November / 2011, 14:56:05 by Microfunguy »

*

Offline a1ex

  • *****
  • 671
  • ML dev
Re: Movie mode framebuffers
« Reply #19 on: 15 / November / 2011, 14:39:19 »
@coutts: what srsa_4c is doing with mjpeg compression here is light years ahead ML's implementation (which simply dumps the uncompressed image buffer on the card).

img.py is a small python script to help guessing the format of image buffers. In current DIGIC4 DSLRs, most things are YUV422 (i.e. UYVY).

The only exception I know is with SD monitors, where one of the buffers is in some strange format, but the luma component is the same as in YUV422.

In dSLRs there seems to be a MJPEG decoder (but not encoder); instead, there's a H.264 encoder. Its input is YUV422 and it seems the encoder outputs YUV420 (not sure where chroma loss takes place). There's also edmac (no idea what this is).

Maybe this also helps a tiny bit:
http://magiclantern.wikia.com/wiki/ASM_Zedbra

 

Related Topics