Looking to implement R and B Channel Swap? - Creative Uses of CHDK - CHDK Forum

Looking to implement R and B Channel Swap?

  • 4 Replies
  • 395 Views
Looking to implement R and B Channel Swap?
« on: 22 / April / 2022, 03:56:21 »
Advertisements
So I'm not too familiar with hardware hacking such as CHDK. However, I'm looking to learn enough to implement a way to swap red and blue channels for IR photography on my A3300IS PowerShot camera.

I am able to build the source code successfully for my camera, and I have implemented a new boolean in the CHDK misc menu for future toggling of said channel swapping feature.

I have done a little research on this forum and have come across a few topics mentioning different ways to add this feature. Just to give an example, here are a few:

This post mentions modifying the cam_CFAPattern in the platform_camera.h file. However, my camera's setting is 0x02010100 for Red, Green, Green, Blue. The goal would be to swap the Red and Blue Channels but I don't know how to translate the value 0x02010100 to Blue, Green, Green, Red. I also have no clue if this will work as desired, but it's worth a shot.

My ultimate end goal would be to have an option to toggle the live view on and off with this color swap feature, so I can have an idea of what my future image will look like in post processing. The camera doesn't have to save the picture with swapped red and blue colors but that would be an added bonus.

Any ideas for where I can start would be appreciated!  :D

*

Offline Caefix

  • *****
  • 888
  • Sorry, busy deleting test shots...
Re: Looking to implement R and B Channel Swap?
« Reply #1 on: 22 / April / 2022, 08:57:58 »
 ;) ... Maybe some hints also in zebra / zones approach from there...  ????
https://chdk.setepontos.com/index.php?topic=14469.msg147494#msg147494
« Last Edit: 22 / April / 2022, 09:01:19 by Caefix »
All lifetime is a loan from eternity.

*

Offline reyalp

  • ******
  • 13792
Re: Looking to implement R and B Channel Swap?
« Reply #2 on: 23 / April / 2022, 00:13:10 »
So I'm not too familiar with hardware hacking such as CHDK. However, I'm looking to learn enough to implement a way to swap red and blue channels for IR photography on my A3300IS PowerShot camera.
Welcome :)
Quote
My ultimate end goal would be to have an option to toggle the live view on and off with this color swap feature, so I can have an idea of what my future image will look like in post processing.
Most of the posts you found are about modifying raw data, or modifying interpretation of CHDK DNG files by changing DNG tags.

Neither of these will help you change the live view. The live view seen by CHDK 8 bit YUV (411 for pre-digic 6 cams like your a3300)

While the exact details of readout are not known (or at least, I don't know them and haven't seen them discussed here) it's clear that the live view readout process does not expose a full RBG bayer image. The downscaling has to happen in readout, because the system does not have anywhere close to sufficient bandwidth to read out full resolution raw at live view frame rates. Presumably, YUV conversion happens at the same time, since RGB versions of the live view are not observed in memory dumps.

You can modify the YUV live view data. It is generally in the frame buffers provided by vid_get_viewport_live_fb in your platform lib.c or platform/sub lib.c.

However, for writing vid_get_viewport_live_fb may not return the exact buffer you want. The live view is usually triple or quadruple buffered (or more on some cams). vid_get_viewport_live_fb is normally implemented to get the current or most recent buffer for motion detection.

The other thing to keep in mind is that the CPU is quite slow, as is accessing the frame buffers. Re-writing the whole buffer from C code every frame would likely be a challenge.

A possible alternative approach would be reverse engineer the Canon "my colors" functionality, which presumably is done by digic hardware. I'm not aware of any significant work in this area, so it would likely be a significant reverse engineering effort, and it may or may not be programmable enough to do what you want.
Don't forget what the H stands for.

Re: Looking to implement R and B Channel Swap?
« Reply #3 on: 23 / April / 2022, 13:28:18 »
Quote
A possible alternative approach would be reverse engineer the Canon "my colors" functionality, which presumably is done by digic hardware. I'm not aware of any significant work in this area, so it would likely be a significant reverse engineering effort, and it may or may not be programmable enough to do what you want.

Where is the "my colors" functionality located? Is this a feature built into the camera that happens in the background or something that any user can experiment with?

Instead of modifying the live view data, lets say I take an approach where I manipulate the DNG data. Is it possible to view the color swapped image in the camera preview menu then?

My main goal is to get an idea of what the final image might look like with swapped Red and Blue channels. I don't want to modify the actual image, unless it is easy to undo later.

I also have no idea where to begin reverse engineering such things. My knowledge of assembly and low level languages is very limited.


*

Offline reyalp

  • ******
  • 13792
Re: Looking to implement R and B Channel Swap?
« Reply #4 on: 23 / April / 2022, 20:55:11 »
Where is the "my colors" functionality located? Is this a feature built into the camera that happens in the background or something that any user can experiment with?
It's a feature of the Canon firmware available in the Func menu (icon looks something like a bell, described on page 90 of the A3300 PDF manual I have), which allows various color effects like B&W, Sepia and various color manipulations. It applies to both live view and Canon jpeg, but does not affect CHDK raw. There isn't an option to swap R and B channels in the Canon UI, but it's possible the underlying functionality can be programmed to it. It's also possible that it only supports the options exposed in the Canon UI.

Some models do have a "color swap" option, but that does single colors, not channels.

Quote
Instead of modifying the live view data, lets say I take an approach where I manipulate the DNG data. Is it possible to view the color swapped image in the camera preview menu then?
Manipulating the CFA pattern in CHDK would only affect programs reading a CHDK DNG. Canon jpeg would be be unaffected.

If you modify the raw data in the raw hook, that is reflected in the Canon jpeg. You can do this using CHDK script: https://chdk.fandom.com/wiki/Lua/Raw_Hook_Operations

Reading / writing individual pixels from Lua is very slow, but may be useful for prototyping.

Writing dedicated C code would be somewhat faster, but processing the entire raw buffer will still take significant time. I'd guess on the order of a few seconds.

Quote
My main goal is to get an idea of what the final image might look like with swapped Red and Blue channels. I don't want to modify the actual image, unless it is easy to undo later.
If you swap it in the raw data, the saved jpeg will be changed. You could presumably reverse it with image processing software, but there would likely be some loss of quality. If you implement it in C code, you could decide whether you wanted to swap before or after CHDK raw/DNG is saved. In any case, you could reverse the process on raw data later.


If you only want to change what is seen in playback, there's another option: Like the live view, the playback frame buffer is YUV 411, and is available from the vid_get_viewport_fb_d function. Unlike the live view, it's only updated when needed, like when you switch images or do other things that erase the screen so it would be much less performance sensitive than the live view. It should be quite manageable to implement a hotkey or something the transforms the current playback frame in some way.

Quote
I also have no idea where to begin reverse engineering such things. My knowledge of assembly and low level languages is very limited.
If you have an interest in exploring the firmware, Ghidra is very helpful, see https://chdk.fandom.com/wiki/Firmware_analysis_with_Ghidra but it's certainly something that takes significant time and technical knowledge to pick up.
Don't forget what the H stands for.

 

Related Topics