Display (bitmap overlay)

Ant

srsa_4c
How well is the M3 working in qemu? Does arm side mzrm messaging work, so stuff like this can be debugged?

Menu navigation works only in playback mode, without images on card(probably because of missing the Omar core).
Yes.
https://www.magiclantern.fm/forum/index.php?topic=2864.msg211107#msg211107

Other alternative (for me) is to port a1ex's mpu exception stuff as I see no easy way to intercept this drawing path.

Probably there is another way. But it's need to check if canon code updates the pointer at 0xBFF00420 or not.

I wonder what happens if you're outside the menu

srsa_4c

Ant
Menu navigation works only in playback mode, without images on card(probably because of missing the Omar core).

Have you published your qemu changes somewhere?

Other alternative (for me) is to port a1ex's mpu exception stuff as I see no easy way to intercept this drawing path.

Probably there is another way. But it's need to check if canon code updates the pointer at 0xBFF00420 or not.

I meant the higher level ARM side functions, many of those don't call any RAM functions and can only be intercepted with the MPU exception method.
I'm still trying to research what jedidraw is capable of (the yuv bitmap method). I already found that one of the 0x64 (100 decimal) bytes is image brightness. Would be great if it supported different image formats or transparency.

srsa_4c

On the m10, one of the bytes in the JediDraw struct appears to control mixing the old drawings and the new. It is initially 0x64 (this 0x64 is separate from those words that are set to 0x640064). From Ant's example, it looks like this struct member is either missing or located elsewhere on the m3. On the m10, it's in the second word of the big struct.
I could not find this until I stopped experimenting with changing the fw's JediDraw messages on-the-fly: Canon uses multiple layers for almost everything, so many of the changes have no visible effects because lower and upper layers are concealing them.
Also found a possible hook, called right after GraphicSystemCoreFinish is sent. Did not test it yet.

loc_FC20DA5A: 6 refs â-¼
 push      {r4, lr}
 movs      r4, #0
 bl        0xfc3eb656 ; <GraphicSystemCoreFinish>
 ldr       r0, =0xC500 @ FC20DA74 ; <ptr_hook_after_gscf>
 ldr       r0, [r0, #0]
 cbz       r0, 0xfc20da6c
 blx       r0
 mov       r4, r0
loc_FC20DA6C: 1 refs â-²
 mov       r0, r4
 pop       {r4, pc}

Ant

srsa_4cFrom Ant's example, it looks like this struct member is either missing or located elsewhere on the m3. On the m10, it's in the second word of the big struct.

Do you mean this word 006a64a0[+128]: 00640100 ?
There need to see your logs. Which GUI elemets use this mixing?

many of the changes have no visible effects

Just in case: did you call Clean_data_cache_line?

P.S. attached my log...

debug-SHOOT5-SETUP1.zip

94.4KB

srsa_4c

Ant
Do you mean this word 006a64a0[+128]: 00640100 ?

I don't know. Here's mine (m10, all words are hex):

17810: [CtrlSrv] JD, abb 0;
*R0: 460560
5652414d,
41e7eb00,
0,
5000004,
3c0,
1e0,
0,
*R1: 1abeb40
2000103,    -> drawing type and direction
640202,     -> byte2: percentage of new drawing vs old content at the same place (defaults to 100)
4c,         -> never changes
1abec20,    -> points after this struct
1abebdc,    -> points to bitmap descriptor, inside this struct
0,
1abebf8,    -> points to some coordinates, inside this struct
4,
0,
0,
e0,         -> length of this struct in bytes
1,
0,
0,
ff,
0,
0,
0,          -> pointer in some other JD messages
0,
43eb0000,   -> destination coordinate in float
438c8000,   -> destination coordinate in float
9600c8,     -> destination height, width
0,
640064,     -> influences pixel data interpretation
640064,     -> influences pixel data interpretation
0,          -> influences pixel data interpretation
0,
0,          -> pointer in some other JD messages
0,
0,
20001,      -> border width
3e3e3eff,   -> border color
640064,     -> 080008 makes grey border dark blue
640064,     -> 080008 makes grey border vanish
0,
0,
200,        -> crash when byte1 changed
0,
0,
2,          -> crash when altered
10107,      -> junk word
140000,     -> y, x offset in src
7800a0,     -> height, width to copy
a000a0,     -> full height, width
5823cb00,   -> bitmap (yuv) buffer
0,          -> junk word
0,          -> ? some coords
9,          -> ? some coords
2d0,        -> ? some coords
1d4,        -> ? some coords
c0,         -> junk
1,          -> junk
0,          -> junk
0,          -> junk
ff,         -> junk
0,          -> junk

Which GUI elemets use this mixing?

I only hand-picked some JD structs, none of them has a value different from 0x64. Historically, Canon has been avoiding overlapping graphics as far as I can tell.

Just in case: did you call Clean_data_cache_line?

Yes, I based my function on your example.

A logged session (startup in playback mode and entering thumbnail-view) of JD structs is attached.

edit:
Added some empirical notes to some struct members. byte0: lsb in a word, byte3: msb in a word

m10_jd_log.txt

322.7KB

srsa_4c

Just in case: did you call Clean_data_cache_line?

Oops. I was not doing this during that part of research, so that may be it. 😳
But the multiple layer stuff I mentioned still stands (drop shadow, borders, etc).

Ant

srsa_4c(drop shadow, borders, etc).

If you mean shadow around OSD icons - it's a border that can be modified in JD structure.

Your JD structure is slightly different. Can you specify your mixing word?

Historically, Canon has been avoiding overlapping graphics as far as I can tell.

Found this situation: see histogram and focus frame.

But this is vector graphic. Not sure this possible for bitmaps...

srsa_4c

Ant

srsa_4c(drop shadow, borders, etc).

If you mean shadow around OSD icons - it's a border that can be modified in JD structure.

Your JD structure is slightly different. Can you specify your mixing word?

Not sure what mixing word is, but I added notes to the example struct above. JD can draw borders by itself, but I think the drop shadow is not specified in the same message as the yuv bitmap.

Found this situation: see histogram and focus frame.

Yeah, and seems like there's more of that. It's very likely though that the histogram and/or grid and/or focus rect is drawn onto the other rgba surface, so only the end result is mixed.

Ant

srsa_4cIt's very likely though that the histogram and/or grid and/or focus rect is drawn onto the other rgba surface, so only the end result is mixed.

Yes. Focus frame is drawn at 42703000 and histogram is drawn at 42541000.
I also found the third type of JD structure used for histogram:

21170:22189900: CtrlSrv:0x00000000: 
Message: 0108: JediDraw, size:0044,addr:bff00500 [11-10]
bff00500: 0000006c 00000000 0000002c bff0051c 02341680 00000000 000004a0 5652414d 
bff00520: 42541000 00000000 05000004 000003c0 000001e0 00000000 
JediDraw content at 0x02341680:
02341680: 00400105 02341b20 00000000 00000000 02341b0c 00000004 00000000 00000000 
023416a0: 000004a0 00000001 00000100 00000080 f0f0f0ff 0234170c 00247864 00000000 
023416c0: 43b98000 422c0000 00aa0100 00640064 00640064 00000000 00000000 00000000 
023416e0: 00000000 00000000 000000ff 00640064 00640064 00000000 00000000 00000000 
02341700: 00640000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 
02341720: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 
02341740: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 
02341760: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 
02341780: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 
023417a0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 
023417c0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 
023417e0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 
02341800: 00000000 00000001 00000001 00000001 00000001 00000001 00000001 00000000 
02341820: 00000000 00000000 00000000 00000001 00000000 00000001 00000000 00000001 
02341840: 00000000 00000001 00000001 00000001 00000001 00000001 00000001 00000001 
02341860: 00000000 00000001 00000001 00000001 00000001 00000001 00000001 00000000 
02341880: 00000001 00000001 00000001 00000001 00000001 00000001 00000002 00000001 
023418a0: 00000001 00000001 00000001 00000001 00000001 00000002 00000002 00000002 
023418c0: 00000002 00000002 00000002 00000002 00000002 00000001 00000001 00000002 
023418e0: 00000001 00000002 00000002 00000002 00000002 00000002 00000002 00000002 
02341900: 00000002 00000003 00000003 00000003 00000003 00000002 00000003 00000003 
02341920: 00000002 00000003 00000005 00000006 00000005 00000005 00000005 00000006 
02341940: 00000006 00000008 00000008 0000000b 0000000b 00000009 00000007 00000007 
02341960: 00000008 00000008 00000007 00000007 00000007 00000009 00000008 00000009 
02341980: 00000007 00000007 0000000b 00000009 00000009 0000000b 0000000a 00000009 
023419a0: 0000000a 0000000a 0000000c 0000000c 0000000d 0000000d 0000000d 0000000f 
023419c0: 0000000d 00000010 00000014 00000014 0000000f 00000010 0000000f 00000014 
023419e0: 00000016 00000017 00000015 00000013 00000015 00000017 00000017 00000018 
02341a00: 00000018 00000017 00000019 00000016 00000017 00000017 00000018 0000001a 
02341a20: 00000019 0000001e 00000020 00000021 00000021 00000022 00000026 00000027 
02341a40: 0000002b 00000031 0000002c 0000002b 0000002d 0000002f 00000030 00000032 
02341a60: 00000032 00000037 00000037 0000003f 0000003f 00000037 00000034 00000039 
02341a80: 00000039 00000043 0000004a 0000004f 0000004b 00000048 0000004c 00000048 
02341aa0: 00000050 00000051 0000004e 0000004f 0000004e 00000055 00000053 00000052 
02341ac0: 00000056 0000005d 0000005f 00000061 00000068 00000065 0000006b 00000071 
02341ae0: 0000007c 0000008e 000000ab 000000b5 0000009a 00000099 000000b6 00000064 
02341b00: 0000005f 00000098 00000333 00000173 0000002b 00000100 000000aa 00000000 
21170:JediDraw [256,170] at [000,000-000,000] col:00000000 [0371:0043]

srsa_4c

srsa_4cfound a possible hook, called right after GraphicSystemCoreFinish is sent. Did not test it yet.

Calling my jedidraw (drawing yuv bitmap, same size as the one in the playback information display menu) from that hook worked. This method obviously only updates that bitmap when the Canon overlay is being drawn.

Ant

srsa_4cFrom Ant's example, it looks like this struct member is either missing or located elsewhere on the m3. On the m10, it's in the second word of the big struct.

Do you mean this word 006a64a0[+128]: 00640100 ?
There need to see your logs. Which GUI elemets use this mixing?

In playback mode when multiple thumbnails are shown at once, scrolling through them at max speed (by holding down 'up' or 'down') fires jedidraw messages where that byte is less than 100.

I started collecting bigger logs by using one of the RAM areas that appear to be unused for a longer time. (I filled a large part of RAM with a magic word and used a function to find untouched areas).
For logging format, I currently output each jedidraw (and other) message in one line, csv format. This log can then be loaded in a spreadsheet (LibreOffice Calc in my case) which can help evaluating some of the words in those messages.

Ant

srsa_4cI started collecting bigger logs by using one of the RAM areas that appear to be unused for a longer time.

Why don't you use _exmem_alloc ?
It gives me at least 16MB.

srsa_4c

Ant
Why don't you use _exmem_alloc ?
It gives me at least 16MB.

That could work too, of course. But knowing those seldom used areas can be useful - logging stuff before the OS starts, for example.
I'm unable to fill more than 0x1a000000 bytes of uncached RAM (done in the very first lines of boot.c) using memset32 - a watchdog shuts down the cam if I attempt more.

Ant

In playback mode when multiple thumbnails are shown at once, scrolling through them at max speed (by holding down 'up' or 'down') fires jedidraw messages where that byte is less than 100.

These thumbnails don't look transparent.
I assume that MZRM can't render transparent bitmaps.

srsa_4c

Ant
This thumbnails don't look transparent.

No, and it happens too fast to observe. It's also possible that the M3 version is different. Log a session in which you do the thumbnail scroll and look for any changes in those 0x64 bytes.

Ant

srsa_4cIt's also possible that the M3 version is different.

What about SX280?

No, and it happens too fast to observe.

The thumbnails are drawn on a gray background. So I don't see it even on CHDK-PTP screenshots.

Log a session in which you do the thumbnail scroll and look for any changes in those 0x64 bytes.

Have you managed to render transparent bitmaps from your code?

srsa_4c

Ant
What about SX280?

Not yet. I'll probably port the "aborttrick" version to this cam. But it is 1st generation so I don't have high hopes.

Have you managed to render transparent bitmaps from your code?

Yes, but as I said, the byte I found is about mixing new overlay with the old. When used repeatedly, the transparent bitmap will turn opaque sooner or later.

Ant

But new overlay still contain a black area that should be fully transparent to be used as CHDK layer.
Maybe it's better to replace one of rgba buffers in XimrExe message or add the third buffer(if it possible)?

srsa_4c

Ant
But new overlay still contain a black area that should be fully transparent to be used as CHDK layer.

The source bitmap has no transparency information, so whatever its background is, it will remain visible. I wonder if rgba bitmaps are supported by one of the methods (JD or not). Ximrexe does support rgba as source, but its destination is yuv+opacity, and I can't recall whether it can enlarge images.

Maybe it's better to replace one of rgba buffers in XimrExe message or add the third buffer(if it possible)?

We can't control which of the rgba buffers is used for which kind of content, plus we can't control the active yuv+opacity buffer. The latter can fluctuate quite often in some modes.
Plus, if ximrexe can't upscale images, we'd need a full size rgba buffer. With jedidraw, we could use a 720x240 yuv buffer and draw on it like on classic powershots (360x240).

srsa_4c

No real progress, just observations.
The sx280 constructs a large (taller than overlay's height) yuv bitmap in playback mode's thumbnail view, rather than drawing the thumbnails one-by-one. I found the larger jedidraw struct to be seemingly always 128 bytes long. Unfortunately, its yuv bitmap mode refers to a (probably previously prepared) external MARV struct that also refers to other structs...
I collected some statistics about the mzrm messages actually used in playback and rec mode (it's possible that some of the other messages are also used under different circumstances). The attachment (.ods file in 7z) lists all messages of sx280 and m10 together with the number of uses in a few minute session. The usage counts are in hex.

mzrm_msg_use.7z

15.01KB

srsa_4c

I accidentally discovered that there is a way to disable most of the Canon overlay drawings on newer cameras. From approx. DryOS r52 on, there is an event procedure named SetWindowSystemDrawSetting. It just sets a global variable which I named canon_drawing_enabled. It is 1 by default, setting it to 0 makes the Canon overlay drawing code skip its main drawing path. Not affected are: mostly animated icons and buffer switches. Still, it makes CHDK menu browsing in rec mode a bit more comfortable. Example implementation for two of my cameras follows.

Index: platform/ixus150_elph140/lib.c
===================================================================
--- platform/ixus150_elph140/lib.c	(revision 5233)
+++ platform/ixus150_elph140/lib.c	(working copy)
@@ -113,6 +113,18 @@
     _ScreenUnlock();
 }
 
+extern int canon_drawing_enabled;
+
+void vid_turn_off_updates()
+{
+    canon_drawing_enabled = 0;
+}
+
+void vid_turn_on_updates()
+{
+    canon_drawing_enabled = 1;
+}
+
 int vid_get_palette_type()   { return 5; }
 int vid_get_palette_size()   { return 256 * 4 ; }
 
Index: platform/ixus150_elph140/sub/100c/stubs_min.S
===================================================================
--- platform/ixus150_elph140/sub/100c/stubs_min.S	(revision 5233)
+++ platform/ixus150_elph140/sub/100c/stubs_min.S	(working copy)
@@ -29,3 +29,5 @@
 DEF(video_target_framesize2, 0xBDB0) // approx. 2*video_target_framesize (for intraframes?), found in sub_FFAB938C
 //DEF(video_framesize, 0x70f0) // most recent frame's compressed size, found at 0xFF993D74
 //DEF(video_frame_data, 0x70f4) // most recent frame's RAM address
+
+DEF(canon_drawing_enabled, 0x7b60) // 0x7b54 + 0xc, from SetWindowSystemDrawSetting_FW
Index: platform/sx280hs/lib.c
===================================================================
--- platform/sx280hs/lib.c	(revision 5233)
+++ platform/sx280hs/lib.c	(working copy)
@@ -8,6 +8,18 @@
     _transfer_src_overlay(1);
 }
 
+extern int canon_drawing_enabled;
+
+void vid_turn_off_updates()
+{
+    canon_drawing_enabled = 0;
+}
+
+void vid_turn_on_updates()
+{
+    canon_drawing_enabled = 1;
+}
+
 void shutdown() {
 //***TODO***
 /*
Index: platform/sx280hs/sub/102b/stubs_min.S
===================================================================
--- platform/sx280hs/sub/102b/stubs_min.S	(revision 5233)
+++ platform/sx280hs/sub/102b/stubs_min.S	(working copy)
@@ -51,3 +51,5 @@
 //FAKEDEF(enabled_refresh_physical_screen,    1)
 
 DEF(displaytype,                            0x8eb😎 //0x8ea4 + 0x14, found  @ fc135f2a (GetDisplayType)
+
+DEF(canon_drawing_enabled, 0x12ad0) // 0x12ac4 + 0xc, from SetWindowSystemDrawSetting_FW
Index: platform/sx280hs/sub/102c/stubs_min.S
===================================================================
--- platform/sx280hs/sub/102c/stubs_min.S	(revision 5233)
+++ platform/sx280hs/sub/102c/stubs_min.S	(working copy)
@@ -51,3 +51,5 @@
 //FAKEDEF(enabled_refresh_physical_screen,    1)
 
 DEF(displaytype,                            0x8eb😎 //0x8ea4 + 0x14, found  @ fc135f2a (GetDisplayType)
+
+DEF(canon_drawing_enabled, 0x12ad0) // 0x12ac4 + 0xc, from SetWindowSystemDrawSetting_FW

This global variable and its support code is completely missing from earlier cameras, so this method only helps those more recent models.

edit:
There is not a lot of talk about vid_turn_off_updates() and vid_turn_on_updates(). Turns out, using ScreenLock() and ScreenUnlock() in those functions seems to get rid of all Canon overlay updates, but the black screen side effect is still there when trying on sx280.

« Previous Page Next Page »