Clean Canon overlays (M10, M3, also M6, M100 via script)

srsa_4c

4243

« on: 15 / July / 2018, 07:27:25 »

Advertisements

update:

CHDK for G5X and EOS M3, M10 now supports clean overlay out of the box. A script is not needed.
For information on how to install, read https://chdk.fandom.com/wiki/Prepare_your_SD_card starting with the blue information box at the top.
In CHDK menu, apply these settings:
Video Parameters -> Clear Canon overlays [Rec] (or MviRec) Video Parameters -> Show remaining videotime [Don't] CHDK Settings -> Disable LCD Off [Always] CHDK Settings -> OSD Settings -> Show OSD [ ]
It appears that on EOS M3, some lenses can make the camera ignore the Disable LCD Off CHDK setting. To help us fix that, please post in the linked thread.

Cleaning the overlay on EOS M6 and M100 can be done with a script.
Script for M100
Script for M6
Save the script as extend.m in the root directory of the card. The card should be prepared (https://chdk.fandom.com/wiki/Canon_Basic/Card_Setup) to run scripts.
Switch on the camera in playback mode and press SET. If the script does what is intended, the overlay should be mostly clean when you switch the cam to any shooting mode. There might be some parts of the overlay that remain unaffected.
If it doesn't work, the card isn't prepared correctly. MS Windows may hide the real file extension which can be a cause of failure.

Support can be added for other DIGIC 6 and DIGIC 7 models, on request.

Original post follows.

In case anybody asks for it.

Initial implementation of clearing Canon overlays. I only tested this using the LCD, but I believe it will work for other output devices.

Current features:
- Overlays can be removed in
- rec mode, or just
- during movie recording
- CHDK overlays are not affected

CHDK menu: Video parameters -> Clean Canon Overlays
[Never] - do not hide overlays
[Rec] - hide Canon overlays in rec mode
[MviRec] - only hide Canon overlays when recording video

Current known limitations:
- When using the "during recording" setting, some Canon overlays may remain on screen. One that I know is the Canon grid (it needs to be switched off in Canon menu).
- Some extra graphical elements may remain visible, regardless of setting: On the M10, the (blue) background of the mode icon can't be removed. This icon only exists in some modes like Hybrid Auto. I also found that when accessing picture controls (Av, Exp. comp, Tv), background shadow of these controls may be visible.
- The M3 port doesn't currently support alternative output devices (that only affects the CHDK overlay though).
- The M3 code is done blindly, it's possible that a palette other than '0' is used in rec mode (someone could verify this by peeking 'active_palette_buffer' - address is in stubs_min.S).

Tech details

Current implementation is DIGIC 6 specific. Implementing on previous DIGIC generations makes little sense as none of them supports higher resolution output in rec mode. Same goes for the other D6 cameras - HDMI is disabled on them in rec mode. However, it should be possible to make an implementation on D4 and D5 ports that support palette manipulation.

The M3 and M10 seem to have 4 palettes (4 bytes per entry, 256 entries). All 4 are in use while using the camera with LCD, which makes me think that the output device does not influence them.
I believe the palettes are for the ARM side firmware, the Xtensa side only receives the actual 32bit colors when it is instructed to draw something.
The palettes are copied from ROM to RAM on first access and (I think) they remain unchanged afterwards.
As written in the limitations section, not all screen elements respect the palette.

D6 implementation

Conditions are continuously evaluated. When the overlay needs to be removed, the palette is copied to a malloc'd buffer (this is only done once), then the Canon palette is zeroed out (all entries set to 0, which means "invisible"). To make (most of) the overlay disappear, a full-screen empty dialog is displayed and removed quickly. Remaining elements disappear when they are redrawn by the Canon firmware (exception: see "current limitations" above).
When the overlay needs to become visible, the palette is copied back and the dialog-trick is used again.

It's probably possible to improve the current implementation (such as: change the Canon palette pointer instead of copying the whole palette), and add other features (make the overlays visible when user interacts with buttons/wheels/touchscreen).

Test builds are provided on request.

cleancanonoverlays_m3_m10_v1.diff (14.81 kB - downloaded 59 times.)

« Last Edit: 18 / July / 2020, 08:51:38 by srsa_4c »

Logged

Menu jumping/flickering on M10?

Ant

Re: Clean Canon overlays (M10, M3)

« Reply #1 on: 15 / July / 2018, 14:30:31 »

Quote from: srsa_4c on 15 / July / 2018, 07:27:25

- The M3 code is done blindly, it's possible that a palette other than '0' is used in rec mode (someone could verify this by peeking 'active_palette_buffer' - address is in stubs_min.S).

I didn't try HDMI or EVF, but with LCD active_palette_buffer is '0'.

chdkptp_20180715_212450.png (225.1 kB, 720x480 - viewed 230 times.)

Logged

srsa_4c

4243

Re: Clean Canon overlays (M10, M3)

« Reply #2 on: 15 / July / 2018, 14:52:36 »

Quote from: Ant on 15 / July / 2018, 14:30:31

I didn't try HDMI or EVF, but with LCD active_palette_buffer is '0'.

That's OK so far, I guess it also remains 0 when recording?
About your screenshot: if you get rid of the grid and the level, does it look clean?
edit: You need to switch off CHDK overlay manually.

Logged

Menu jumping/flickering on M10?

Ant

Re: Clean Canon overlays (M10, M3)

« Reply #3 on: 15 / July / 2018, 15:16:37 »

Quote from: srsa_4c on 15 / July / 2018, 14:52:36

so far, I guess it also remains 0 when recording?
About your screenshot: if you get rid of the grid and the level, does it look clean?

yes. yes.

Quote

edit: You need to switch off CHDK overlay manually.

I know it. I did this screenshot to display the value of active_palette_buffer

« Last Edit: 15 / July / 2018, 15:18:13 by Ant »

Logged

srsa_4c

4243

Re: Clean Canon overlays (M10, M3)

« Reply #4 on: 15 / July / 2018, 16:44:06 »

Quote from: Ant on 15 / July / 2018, 15:16:37

Quote from: srsa_4c on 15 / July / 2018, 14:52:36
so far, I guess it also remains 0 when recording?
About your screenshot: if you get rid of the grid and the level, does it look clean?
yes. yes.

Thanks.

Quote

I did this screenshot to display the value of active_palette_buffer

Now I get it.

So, it seems the patch works for both cameras. I'll probably add this to trunk sometime later, but some research (and possibly code improvement) is to be done first.

Logged

Menu jumping/flickering on M10?

reyalp

12651

Re: Clean Canon overlays (M10, M3)

« Reply #5 on: 22 / July / 2018, 23:05:02 »

Nice work

Another case where it would be nice to disable Canon drawing:

Long zoom cameras like sx710 have a "frame assist" feature, where holding the frame assist button zooms out quickly to help you find the subject, and zooms back in when you release the button.

The area of the full zoom FOV is outlined with a white rectangle, and everything outside it has a semi-transparent overlay (see screenshot). Having the zoom area clearly indicated is nice, but the overlay can make it hard to see small or low contrast subjects.

tl; dr:
It would be nice to be able to disable the overlay. Disabling all Canon UI while the "FramingAssist" controller is active would probably be good enough. If just the rectangle could be kept (replace only the semi-transparent color with fully transparent) that would be better.

chdkptp_20180722_194925.png (263.2 kB, 640x480 - viewed 109 times.)

Logged

Don't forget what the H stands for.

srsa_4c

4243

Re: Clean Canon overlays (M10, M3)

« Reply #6 on: 23 / July / 2018, 17:56:18 »

Quote from: reyalp on 22 / July / 2018, 23:05:02

Another case where it would be nice to disable Canon drawing:

Long zoom cameras like sx710 have a "frame assist" feature, where holding the frame assist button zooms out quickly to help you find the subject, and zooms back in when you release the button.

The area of the full zoom FOV is outlined with a white rectangle, and everything outside it has a semi-transparent overlay (see screenshot). Having the zoom area clearly indicated is nice, but the overlay can make it hard to see small or low contrast subjects.

tl; dr:
It would be nice to be able to disable the overlay. Disabling all Canon UI while the "FramingAssist" controller is active would probably be good enough. If just the rectangle could be kept (replace only the semi-transparent color with fully transparent) that would be better.

I'd expect the palette related parts to be approximately the same on other D6 models (fw variables, number of palettes), and you'd need to find out which palette entry is that background.
This feature would require replacement of 1 palette entry, so the platform code would be different.
Perhaps the clean overlay patch should be turned into something that covers both cases, somehow...
Also, there is at least one model with both rec-mode-HDMI and framing assist (g3x, unported).

Logged

Menu jumping/flickering on M10?

effectzero

Re: Clean Canon overlays (M10, M3)

« Reply #7 on: 03 / December / 2018, 00:02:41 »

Many thanks for your help srsa_4c!!

I ran through all your links/info and I totally get how your hack works along with the idea of trying it in CanonBasic. By looking at the various scripts and CanonBasic info I get the majority of how that works and how I can eventually make this execute in a slick way to bounce between the original palette and the zero'd one... But, I'm a bit stuck trying to translate the actual bzero'ing of the palette into Canon Basic. (my dev skills cap out around c# & python)
Is palette_buffer_ptr a palette property that I can just set with set_prop? How do i reference the firmware addresses you so kindly gave me?

I'm digging through the various script examples but having a hard time finding the 'ah ha' moment.

Logged

reyalp

12651

Re: Clean Canon overlays (M10, M3)

« Reply #8 on: 03 / December / 2018, 01:14:36 »

Quote from: effectzero on 03 / December / 2018, 00:02:41

But, I'm a bit stuck trying to translate the actual bzero'ing of the palette into Canon Basic.

memset is available as an eventproc. Pointers are just numbers in Canon Basic.

Logged

Don't forget what the H stands for.

srsa_4c

4243

Re: Clean Canon overlays (M10, M3)

« Reply #9 on: 03 / December / 2018, 16:08:30 »

@effectzero
The following (very basic) script zeroes out palette 0 (it's the rec mode palette on M3 and M10, and it might be that on M6 too). I believe there are 4 palettes, not all of them are necessarily in use.

Code: [Select]

' EOS M6, 100b
 
DIM palette_buffer_ptr = 0x11d4c
DIM active_palette_buffer = 0x11d44
DIM palette_to_zero = 0
 
private sub RegisterProcs()
    System.Create()
    'UI.CreatePublic()
end sub
 
private sub Initialize()
    RegisterProcs()
    adr = *palette_buffer_ptr
    adr = adr + (palette_to_zero * 4)
    if *adr <> 0 then
        adr = *adr + 4
        memset(adr, 0, 256 * 4)
    end if
end sub

Save it as extend.m in the root directory of the card. The card should be prepared to run scripts.
Switch on the camera in playback mode and press SET. If the script does what is intended, the overlay should be mostly clean when you switch the cam to any shooting mode. There might be some parts of the overlay that remain unaffected.
If this works, you could either use it as is, or it could be turned into a more sophisticated script that attempts to clean/restore the palette on a certain keypress.

Logged

Menu jumping/flickering on M10?