Display (bitmap overlay)

philmoz

@srsa_4c

In the current CHDK code the CHDK UI is drawn directly on the output YUV buffer so everything drawn is immediately visible.
This requires some way to notify CHDK that the screen needs to be redrawn which we have various solutions for.

If we move the CHDK UI to a new RGBA layer and insert it into the ximr context then our content won't be visible unless the firmware does a screen refresh.
So we need some way to force the firmware to update the display so our layer gets re-drawn.

Calling transfer_src_overlay seems to work; but I'm concerned their could be timing issues with the firmware and CHDK both calling this.

Have you come across any other way to tell the Canon code that a screen refresh is needed?

Phil.

kitor

You need [1] to at least match layer resolution (resolution in MARV structure + buffer size for it) to layer size of Canon code.
After that it is all about manipulating coordinates/size of "rectangle to be transfered" (i mean src_x/y, src_w/h and dst_w/h).

You will probably need to adjust coordinates based on active display output. EOSR does this "accidentally" in Canon code for me.

200D does not, and with "default" offsets (120,30) (720x480) (0,0) it will go to 120x30 of layer buffer, fetch 720x480 rectangle starting from that point and draw it from 0,0 of visible output area.
So for HDMI output one need to switch from (120,30) (720,480) (0,0) to (0,0) (960,640) (0,0), and vice versa.

For cameras with EVF (if they are any) you will have similar third case for EVF.

[1]
As I said a few posts before, layers are not upscaled separately to fit the screen, but there's a single "scaling ratio" defined for whole XimrContext.
Canon code is keeping this scaling ratio "right" for their layers (and this is the reason why all of them have one resolution).

Technically you should be fine with any lower resolution as long as you keep Ximr drawn rectangle coordinates in that buffer - but you won't be able to draw over full screen due to how scaling ratio is handled globally.

As for "telling Canon to refresh", we don't.
In EOS roms there's a flag that we call refresh_display_needed. If set to 1 and refreshVrmsSurface() is called (this is the one that does XimrExe call in EOS), it will redraw everything.

If you just set the flag, you will wait forever, so I initially created own redraw() which set the flag, did caches sync (iirc it was necessary) and called refreshVrmsSurface().

Of course this is different for your codebase, but you can just trace how to properly call that function which does XimrExe inside.

Later we switched into another approach.
We just "steal" winsys semaphore, and guard our own XimrExe call with pointer to "old" (existing) XimrContext structure (as the buffer location used by Ximr in advance). That's it, done.
Winsys semaphore is used in EOS by refreshVrmsSurface() to guard XimrExe too. Not sure if you have any equivalent, and maybe it is not really needed.

The worst it can happen is short visual glitch when switching outputs (as your code might race Canon one, and try to render before Canon updated (in your case regenerated) structure to match HDMI settings)

srsa_4c

philmoz
Calling transfer_src_overlay seems to work; but I'm concerned their could be timing issues with the firmware and CHDK both calling this.

Have you come across any other way to tell the Canon code that a screen refresh is needed?

I haven't found a variable that would do that. But I do have an idea for a workaround. If I recall right, most or all drawing operations are done by the CtrlSrv task. If we tell that task to execute transfer_src_overlay, chances for a clash should become pretty low. There is a firmware function for that, I called it exec_via_ctrlsrv in the m100 source. I don't think it's in the sigfinders yet, its address in m100 100a is 0xE033F5E8. Takes a function pointer and another argument which will be passed to the function in r0.
About the order of layers: I'm sure that can be fixed somehow (either by rearranging them or maybe there is priority/z-order info in the ximr struct).

edit:
We could also consider executing just ximrexe instead of the whole transfer_src_overlay.

kitor

maybe there is priority/z-order info in the ximr struct

@srsa_4c None that I'm aware of. Layers are always drawn starting from zero. They are always referenced by ID while being added to XimrContext in both EOS and what I saw in M10 ROM.

I guess you want to draw over LV (zebras, etc) but still allow Canon GUI to be drawn over that? As you already alter XimrContext by hand, reordering them should be simple.

[e]

We could also consider executing just ximrexe instead of the whole transfer_src_overlay.

This is almost exactly what we do currently in Magic Lantern, as mentioned in my previous post.
And it is also how our "April fools" Doom 200D port was redrawing screen in ~20 fps (render code was still the bottleneck)

Ant

kitor
Additionally, somewhere in XimrContext structure there is scaling ratio (or really two ratios - horizontal and vertical) set.

Yes. I upscaled my own layer. It also can be flipped, mirrored, shifted in 4 directions. But sometimes it disappears. Probably CHDK can draw in 360*240 buffer and then upscale it using XIMR.

kitor

@Ant are you sure you upscaled only your layer? I see duplicated Canon GUI is upscaled too.

According to my knowledge you can set scaling ratio (multiplier) once for all layers. This is kind of confirmed by XCMStatus on R+ which displays scaling ratio in Output Chunk section, not per layer.

Which offset in XimrContext you used for that upscaling (and which camera?)

For reference, XCMStatus output (part of XimrContextMaker which is "userspace tool" for XCM on R+):
This should give you an idea what is set "per layer" and what "per context" (those in Output Chunk, XCM can manage 4 contexts at once, that have the same input layers but different output chunk - that's why there are 4 outputs listed).

K424[1]>XCMStatus
--- XimrContextMaker 0x00bce800 ---
[Input Layer] No:0, Enabled:1
  VRMS Addr:0x02e18500, pImg:0x02c1e100, pAlp:0x00000000, W:960, H:540
  Color:=0x05040100, Range:FULL
  srcX:0120, srcY:0030, srcW:0720, srcH:0480, dstX:0000, dstY:0000
[Input Layer] No:1, Enabled:1
  VRMS Addr:0x03012a00, pImg:0x02e18600, pAlp:0x00000000, W:960, H:540
  Color:=0x05040100, Range:FULL
  srcX:0120, srcY:0030, srcW:0720, srcH:0480, dstX:0000, dstY:0000
[Input Layer] No:2, Enabled:0
[Input Layer] No:3, Enabled:0
[Input Layer] No:4, Enabled:0
[Input Layer] No:5, Enabled:0

[OutputChunk] No:0, Enabled:1
  VRMS Addr: 0x412ff900, pImg: 0x41100100, pAlp: 0x00000000, W:1024, H:682
  Color:=0x11060200
  LayerEnabledBit:ffffffff, isActualSize:0, isSetToContext:1
  Output Offset X:0000, Y:0000
  Resize fromX:0000, fromY:0000, outputW:1024, outputH:0682
  Horizontal 64/45, filter:0, Vertical 341/240, filter:0
[OutputChunk] No:1, Enabled:0
[OutputChunk] No:2, Enabled:0
[OutputChunk] No:3, Enabled:0

[LUT] Addr:0x00000000 Type:LUT_RANGE_FULL
  LUT is not enabled.

[Ximr Context] No:0, Addr:0x00bce810, Size:0x0000031c, Eanabled:1
//here goes XimrContext structure hex dump, so I skippped it

Ant

kitor
@Ant are you sure you upscaled only your layer? I see duplicated Canon GUI is upscaled too.

Sure. As you can see my upscaled layer is rendered over Canon's layer and over focus frame. I just copied Layer0 descriptor and put it after Layer1 descriptor.

Which offset in XimrContext you used for that upscaling (and which camera?)

This should be Byte+0x07 in InputLayer descriptor. 0x06 means both coordinates to be upscaling x2. EOS M3. LCD display type.

kitor

This should be Byte+0x07 in InputLayer descriptor. 0x06 means both coordinates x2 upscaling. EOS M3.

I need to test it on EOS, but this looks like a great find!

philmoz

Thanks @Ant - that's a great find 😀

This works on the G7X2 and works mostly ok on the G5X.

On the G5X when using the EVF the added RGBA layer gets clipped to 347 pixels wide.
In the attached screenshot the 4 blue boxes should all be the same size. The white line on the right is in column 346.

When using the LCD or HDMI the display is correct and the boxes are displayed fully.

Phil.

pigeonhill

@reylay

I read in 5868:

Fix for CHDK menus to resize and position correctly if screen size is changed while menu is displayed. Can happen if switching between LCD, EVF and HDMI.

What I notice on the M3 is that when you switch between LV and EVF, the CHDK histogram does not get drawn in the 'correct' place, ie it seems to remain at its absolute (LCD based) position, rather than its position being scaled.

Just an observation.

Cheers

Garry

philmoz

In case anyone is interested here is a patch of my current progress on using an RGBA Ximr layer for the CHDK UI.

I have implemented this for the G5X and G7X2 in this patch.
From what i have tested so far it works well - rock steady CHDK UI with no flickering.

I create a 360x240 (or 480x270 for HDMI) buffer inserted into the Canon display pipeline via the dry_memcpy patch.
This is then scaled 2x in both directions when rendered.

At the moment the CAM_DRAW_RGBA setting has to be in the makefile for the camera in order to build correctly - I'm not sure how to fix this at the moment.

I'm sure much of this could be done better so feedback is welcome.

Phil.

rgba_draw_v1.patch

36.64KB

pigeonhill

@philmoz

Looking forward to testing this on the M3.

philmoz

@pigeonhill

Here is a test build for M3 1.20f.

Note this is very experimental. It may not work, it may crash or cause issues in areas I have not fully tested.

One change you will need to make for your script is the display size - the current code draws to a 720x480 display.
In this version the CHDK UI buffer is 360x240 pixels so you will need to adjust any drawing you do to the screen.

Phil.

m3-120f-1.6.0-5869-full.zip

756.36KB

pigeonhill

@philmoz

Phil

Transformational!!!!!!! 😀

The overlays are rock solid and 100% compatible with the Canon GUI, ie as I cycle through, including the Canon histogram.

I won't try and tweak my code yet, as I'm sure you will be refining things.

As soon as you think you have a stable configuration, I'll tweak my Bracketing script to match the new screen sizes.

Fantastic work and thanks for your efforts: and my M3 builds 😉

Cheers

Garry

Ant

philmozIn case anyone is interested here is a patch of my current progress on using an RGBA Ximr layer for the CHDK UI.

Why don't you use own YUV+Opacity buffers(as Ximr input layer) for drawing CHDK UI?

I also suggest not to patch dry_memcpy or TakeSemaphore. I think these functions are being called more intensively.
It's better to hook disabled debug messages inside mzrm_sendmsg function:

void __attribute__((naked,noinline)) MZRM_hook() {
 asm volatile (
"ldr     r0, =0xfc2fd873\n"
"cmp     r0,lr\n"
"bne     mh_exit\n"

"push {lr}\n"
"mov r1, r11\n" // MZRM message
"bl mylogfunc2\n"
"pop  {pc}\n"

"mh_exit:\n"
"bx lr\n"
".ltorg\n"
);
}

EOS M3 v1.01a

But this require parsing the MZRM messages.

srsa_4c

Ant
Why don't you use own YUV+Opacity buffers(as Ximr input layer) for drawing CHDK UI?

I would consider using the area where the firmware's yuv layer is drawn. By hooking the ximrexe call that creates it, we could reconfigure the yuv layer to be half height or width (and later double its height (or width) when it is used as input layer). The remaining space in that area is more than enough for a 720x240 RGBA buffer for CHDK - we wouldn't have to allocate any memory.
I found more than one byte in the layer struct where certain bits cause the layer to be rendered with double width and/or height. I don't know what this effect actually is, it's not a smooth bilinear operation, it just doubles pixels.

Ant

srsa_4cBy hooking the ximrexe call that creates it, we could reconfigure the yuv layer to be half height or width

This will decrease the quality of Canon graphics

we wouldn't have to allocate any memory.

Is this a big problem to allocate a couple of megabytes from hundreds?

Currently I'm experimenting with my own YUV buffer created using exmem_ualloc. I use it as third layer to draw CHDK UI over Canon GUI.

reyalp

Ant
Is this a big problem to allocate a couple of megabytes from hundreds?

It's virtually certain there's megabytes to spare *somewhere*, but that's not the same as knowing where they are or being able to allocate them without impacting the original firmware. For example, we've observed on earlier cams that using exmem can impact the maximum jpeg quality, as well as the conflicts with video buffers.

I also suggest not to patch dry_memcpy or TakeSemaphore. I think these functions are being called more intensively.

This would be my initial gut reaction too, but the hook is essentially just comparing LR with a constant and jumping back to the firmware if it doesn't match, which is negligible compared to the rest of the function.

Phil's version does slightly more, but could be optimized for the non-matching case. R12 and R3 are free to use. (edit: To be clear, I'm not against using the log function instead, I generally favor using the approach with lowest impact to other parts of the firmware. I just wouldn't worry much about hooking dry_memcpy)

philmoz

Thanks @Ant - i was able to figure out patching the mzrm_sendmsg logging call and access the Ximr context data.
I am still able to then add the new layer in the Ximr context so no need to parse MZRM stuff.

This is more efficient (called less often), especially in record mode.

Phil.

Ant

philmozso no need to parse MZRM stuff.

By parsing, I meant checking the message type number(the 1st word) and taking the address of the XIMR context (the 4th word). This can be done using assembler.
XIMR context in XimrExeGain message also can be modified.

How do you clear the CHDK layer when you enter canon menu?

P.S. Original canon buffers are aligned to 0x1000.

« Previous Page Next Page »