Movie mode framebuffers - General Discussion and Assistance - CHDK Forum
« previous next »
Pages: [1] 2 3 4   Go Down

Movie mode framebuffers

  • 33 Replies
  • 14922 Views
*

Offline srsa_4c

  • ******
  • 4451
Movie mode framebuffers
« on: 08 / November / 2011, 16:12:02 »
Advertisements
Movie mode framebuffers are different (edit: in size) from those used in still image mode and play mode. In case of DIGIC II, all of the above buffers have the pixelformat mentioned by reyalp here, in the "Live viewports" thread (according to the wikia article this is probably Y411).
This means, these (DIGIC II) cameras create their "movies" with about half horizontal color-resolution than their photos: four horizontal pixels get the same U and V values.

What could be the use of this information? I think, it would be possible to
- get decent :) resolution live pictures without moving the shutter
- use the DSP (via the Canon MJPEG routines) to compress those pictures
 - sort of custom (slower) movie mode, maybe
 - transfer them via PTP (much faster than uncompressed)

But there is a problem, which affects the new models (DIGIC 4, 5) without MJPEG codec: in case of these, the usual JPEG engine could be used - depending on how fast it is. Unfortunately, this is undiscovered area.

Also, according to these findings, it's NOT possible to have (much) greater resolution movies, even if one could set the CCD & A/D & timings to produce something bigger, the buffers would overflow. The buffer addresses are hardcoded in several parts of the firmware.

About the method I'm using:
- create dumps of the camera's (whole) RAM -> dump_memory() in /core/main.c needs fixing as it tries to dump 32MBytes of RAM fixed now in trunk
- inspect them visually with a hacked version of Hexplorer
 - with the "pixel view" window enabled, I browse through the dumps (I've implemented Y411, UYVY, ... viewing modes, source, binary is available on request attached below, also works with current versions of wine)


   
Visual inspection example (movie live viewports)   Pixel format menu in pixel view

Finally a question, maybe somebody can answer it:
Why am I not able to dump the RAM, when movie record is in progress? During record, the dumping process just hangs (haven't checked, where). Is write() blocked? Does VxWorks block the access of some memory ranges?

Have to break up the post, more to follow.

Edit: attached is that hacked Hexplorer version (sources, .exe) - get the official package first, link in the post.
... and almost forgot, that it contains win32 binaries, so, according to virustotal.com, it's clean and has the following characteristics:
MD5   : a7c95698ca1b17c24981fa6e048d6427
SHA1  : d2a09ad4e3c39bad6b992e81e0693c13046c5ed0
SHA256: ffd11ee82d6a0c73f0becbe55003ee26d41e61485455fff87f4e587120920dc8
« Last Edit: 04 / December / 2011, 20:15:08 by srsa_4c »
Logged
*

Offline srsa_4c

  • ******
  • 4451
Re: Movie mode framebuffers
« Reply #1 on: 08 / November / 2011, 16:14:30 »
Ixus 30 (first DIGIC II model, 3.2Mpixel)

There are three movie buffers. One (or all) of these is also used as live viewport (for display on the LCD or TV). The buffer size is 720x480 pixel, they follow each other in memory. When TV-out is active, the used area is reduced to 704 (or 352) pixel width, line ends are not used.
Upon entering movie mode (and later on changing the movie format from for example 640x480 to 320x240) the hardware is set to the desired framerate and resolution, instantly. The effect of inserting the TV-out jack is also instant.
The following table lists the movie modes, with the corresponding movie buffer picture resolution.
640x480 - 720x480 (704x480 with TV-out)
320x240 - 360x240 (352x240 with TV-out), also in high speed (60 fps) mode
160x120 - haven't checked

So, the "other side" of DIGIC has the duty of resizing and compressing the frames when record is activated (havent't found any sign of the resized uncompressed frames yet).


A420 (early DIGIC II model, no TV-out, 4Mpixel)

There are also three consecutive movie buffers. There isn't any separate liveview buffer in movie mode, so one or more of those movie buffers is/are used for that. The size of these buffers is 720x568 pixel. The used part is always top left.
Movie modes, buffer's picture resolution.
640x480, 10fps - 720x568
320x240, 30fps - 360x191 (!!! Canon's cheating)
160x120 - haven't checked

In a thread I wrote something about setting 30fps in 640x480 mode. It works (by setting the propcase), but the buffer's image resolution gets set to 360x191.
Seems like a severe hardware limitation. the used CCD or A/D converter is too slow. It's probably a hw limit of the CCD.
« Last Edit: 04 / December / 2011, 20:11:11 by srsa_4c »
Logged
*

Offline srsa_4c

  • ******
  • 4451
Re: Movie mode framebuffers
« Reply #2 on: 08 / November / 2011, 16:27:33 »
SX100IS (DIGIC III, 8Mpixel)

There are separate buffers (3) for liveview - buffer width 720 px, pixel format y411
In the lower resolution modes, these buffers are used as source for compression.
In VGA movie mode, there are additional 3 framebuffers: buffer width = picture width = 640px, height = 480px, pixel format uyvy. These serve as source in VGA mode.
The buffer dimensions for liveview are 720px x 480px. At most 240px is used as height.
The 3 liveview and the 3 special VGA buffers follow each other in memory.

movie res.   source buf.   source res.   pixel format   buffer kind
160x120720x480360x121y411liveview buffer
320x240720x480360x240y411liveview buffer
640x480640x480640x480uyvyspec. movie buffer

The above also means that the compression routine can accept either y411 or uyvy as source. The pixel format is also passed as a param (0 when y411, 1 when uyvy).
The compression subroutine is not only called from the movie task, the other caller comes from PCEvf.c . I've never tried, but this seems to indicate that in PC-controlled mode, the remote liveview is using MJPEG compression to save USB bandwidth.

Back to the Ixus 30: by accident I found out that in case of photos taken with less resolution, the camera places the resized picture back into the (only) raw buffer. The pixelformat seems too be UYVY (but I'm not sure, as CHDK's DNG routines were also involved). If new cameras also use this format for the resized picture AND their movie buffers are also the same format, then - as I wrote in the first post - using the JPEG engine for motion picures might be possible, when needed.


edit: the previous information regarding the SX100 here was a mistake, corrected
« Last Edit: 26 / January / 2012, 19:09:35 by srsa_4c »
Logged
*

Offline waterwingz

  • ******
  • 12140
    • Waterwingz
Re: Movie mode framebuffers
« Reply #3 on: 08 / November / 2011, 23:51:12 »
sorry srsa_4c  - tl;dr

was there a question in there somewhere ?   :)




EDIT : okay - I actually did read it.  This might be a better CHDK wiki page than a forum post ?
« Last Edit: 08 / November / 2011, 23:58:29 by waterwingz »
Logged
Ported :   A1200    SD940   G10    Powershot N    G16
*

Offline Microfunguy

  • ****
  • 4037
    • StereoData Maker
Re: Movie mode framebuffers
« Reply #4 on: 09 / November / 2011, 12:17:42 »
Quote from: srsa_4c on 08 / November / 2011, 16:12:02
Why am I not able to dump the RAM, when movie record is in progress?

No way  .....

The movie-record task will not tolerate any sort of pause, it has a lot of work to do.

Presumably, a hardware, external 29.97 Hz interrupt triggers the frame capture.

I gave-up playing with that.

Quote
I browse through the dumps (I've implemented Y411, UYVY, ... viewing modes

Sounds interesting, if I have sufficient time I may contact you about that.

« Last Edit: 09 / November / 2011, 12:21:12 by Microfunguy »
Logged
*

Offline srsa_4c

  • ******
  • 4451
Re: Movie mode framebuffers
« Reply #5 on: 09 / November / 2011, 12:50:20 »
Quote from: waterwingz on 08 / November / 2011, 23:51:12
was there a question in there somewhere ?   :)
Yes, it's buried in the first post: Why am I not able to dump the RAM, when movie record is in progress?
Quote
This might be a better CHDK wiki page than a forum post ?
I'll paste a link into the most appropriate wikia topic: Frame buffers.

Quote from: Microfunguy on 09 / November / 2011, 12:17:42
No way  .....

The movie-record task will not tolerate any sort of pause, it has a lot of work to do.
So, maybe task priorities are responsible for that?

Quote
Presumably, a hardware, external 29.97 Hz interrupt triggers the frame capture.
Interrupts, interrupt handling are black box for me...
Same goes for the communication in the 0XCxxxxxxx area.
Logged
*

Offline srsa_4c

  • ******
  • 4451
Re: Movie mode framebuffers
« Reply #6 on: 10 / November / 2011, 15:35:46 »
Slightly offtopic, but just noticed a "fun fact".

The above mentioned dump_memory() in /core/main.c doesn't work correctly (looking at the dump sizes):
The first part of the process (0x0-0x1900) fails, the actual dump starts at 0x1900, and is of course shorter.
Affected (my experience) :
SX100IS
A420
A410

The source applies a workaround for the IXUS65 (according to the comment, its 0x0 address can't be read). I'm really starting to think that a lot more than just that model needs that workaround...
Side note: In case of the IXUS 30, I had to use all uncached addresses there, the normal ones failed.

When I changed
Code: [Select]
write(fd, (void*)0, 0x1900);to
Code: [Select]
write(fd, (void*)0 | CAM_UNCACHED_BIT, 0x1900);it worked on the A420.

Could some of you devs try it?

If this one needs and gets a fix, individual cameras' RAM size should be defined and used there too.
Proposal:
Code: [Select]
#define CAM_RAM_SIZE 32 //camera's RAM size in MBytesin camera.h
« Last Edit: 10 / November / 2011, 15:41:13 by srsa_4c »
Logged
*

Offline Microfunguy

  • ****
  • 4037
    • StereoData Maker
Re: Movie mode framebuffers
« Reply #7 on: 10 / November / 2011, 20:52:34 »
Quote from: srsa_4c on 08 / November / 2011, 16:12:02
(I've implemented Y411, UYVY, ... viewing modes, source, binary is available on request attached below

Thanks, I look forward to trying  that.

It is a lot of files !
Logged
*

Offline reyalp

  • ******
  • 14126
Re: Movie mode framebuffers
« Reply #8 on: 10 / November / 2011, 23:43:18 »
Quote from: srsa_4c on 10 / November / 2011, 15:35:46
Slightly offtopic, but just noticed a "fun fact".

The above mentioned dump_memory() in /core/main.c doesn't work correctly (looking at the dump sizes):
The first part of the process (0x0-0x1900) fails, the actual dump starts at 0x1900, and is of course shorter.
Two things come to mind:
1) There could be a null pointer check, in which case you could simply start at 0x4. Using the uncached address should also be acceptable in this case.
2) The lowest 4k of address space maps to instruction TCM. It's possible that write() is trying to DMA directly from that and does not always have the desired effect. An fopen/fwrite might be OK in this case since it copies to a RAM buffer before sending to write(). In this case, uncached bit might get you actual RAM, which might not contain the same things as ITCM.
Logged
Don't forget what the H stands for.
*

Offline srsa_4c

  • ******
  • 4451
Re: Movie mode framebuffers
« Reply #9 on: 11 / November / 2011, 00:06:33 »
@reyalp
Thanks for the pointer, just found this: Instruction TCM accesses (ARM946E-S™ Technical Reference Manual).
Guess this means the routine really needs a fix (not that I've ever had to look at that address range, so its content may not have to be correct).

Quote from: Microfunguy on 10 / November / 2011, 20:52:34
It is a lot of files !
Yes, because it's not a proper release. Get the original from sourceforge, then replace the .exe with the one in this archive. I was able to compile the source with CBuilderX, a free (home use) product of Borland which is no longer officially available.

edit: grammar
« Last Edit: 13 / November / 2011, 05:54:40 by srsa_4c »
Logged
Pages: [1] 2 3 4   Go Up
« previous next »
 

Related Topics


SimplePortal © 2008-2014, SimplePortal