Advice Requested - CHDK Security Project - Object Detection and Steganography - General Discussion and Assistance

Vegetothe1st

19

Advice Requested - CHDK Security Project - Object Detection and Steganography

« on: 10 / February / 2014, 14:04:50 »

Advertisements

Hi everyone,

I'm looking to do a security-related project using CHDK and wanted to get some tips and pointers before beginning. I haven't dug into the firmware at all yet, so pointers on which files would be important to look at would be appreciated.

I'd like to modify the firmware to do the following (more detail below):

1. Prevent a user from taking a picture when a certain object is in the frame (company logo, etc...)

2. Embed additional information inside the photo itself in a non-obvious way (i.e. encode it into the image)

3. Detect a particular user's face in an image

For the first point, the idea is to prevent a user from taking an image if they're taking a photo of some particular object (a person with a specific badge on, documents with watermarks, etc...). Ideally this detection could be performed before the user pressed the capture button and prevent the photo from ever being taken. I'm open to suggestions on object detection algorithms to use, I plan on starting by detecting a simple mono-colored shape and advancing from there.

The second point is a bit simpler and can be done after image capture. The idea is to use steganography to encode additional information into the photo, such as the time and date it was taken. There should be some simple methods to do this, but I'll need to modify the image before it's saved to the SD card.

Finally, the third point is easily the most difficult (and just a stretch goal). I'm wondering if it's possible to detect a particular user's face in an image and mask that person's face. I would probably start with masking all faces, but ideally it would only mask the face of a specific person. I know Canon cameras with DIGIC 3 processors and above have face detection built into them, can this feature be accessed and leveraged for this goal? Also, is the goal possible on DIGIC 2 cameras that don't have built-in face detection?

Any thoughts, tips, etc... on directions to approach these goals from would be greatly appreciated!

Logged

Vegetothe1st

19

Re: Advice Requested - CHDK Security Project - Object Detection and Steganography

« Reply #1 on: 10 / February / 2014, 14:05:56 »

Also, if this isn't the right forum to post this in then please let me know! I looked around and thought this was the best place.

Logged

philmoz

3450

Re: Advice Requested - CHDK Security Project - Object Detection and Steganography

« Reply #2 on: 10 / February / 2014, 16:15:46 »

Interesting idea; but I think you will face some challenges trying to do this with a Canon P&S camera.
Here are some of my initial thoughts.

CHDK does not modify the camera firmware, it loads every time the camera is started from special files on the SD card.

Unlocking the SD card, or formatting it in the camera will remove CHDK and all your security options will be gone.

Your second goal, to embed data in the image, would be the easiest - CHDK has access to the raw captured data before the camera converts it to a JPEG. You could fairly easily modify the captured data to embed a watermark. Remember you are working with raw sensor data (in a Bayer format) not RGB pixels.

For the other two you would probably need to work with the live view data shown in the LCD. This has much lower resolution than the sensor is capable of (720x240 for most cameras) and is in YUV format. Being able to detect a logo or particular person, at different angles, positions and lighting at such a low resolution could be a challenge.

There is also not a lot of CPU power available. I don't know what the state of the art in recognition algorithms is like these days; but I imagine it still needs a fair bit of horsepower.

Phil.

Logged

CHDK ports:
sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
g12 (1.00c, 1.00e, 1.00f & 1.00g)
sx130is (1.01d & 1.01f)
ixus310hs (1.00a & 1.01a)
sx40hs (1.00d, 1.00g & 1.00i)
g1x (1.00e, 1.00f & 1.00g)
g5x (1.00c, 1.01a, 1.01b)
g7x2 (1.01a, 1.01b, 1.10b)

nafraf

1308

Re: Advice Requested - CHDK Security Project - Object Detection and Steganography

« Reply #3 on: 10 / February / 2014, 16:42:33 »

Quote from: Vegetothe1st on 10 / February / 2014, 14:04:50

3. Detect a particular user's face in an image

How many faces in your library? Some cameras has built in function to recognize particular user's face, camera shows their names on LCD.

Logged

Vegetothe1st

19

Re: Advice Requested - CHDK Security Project - Object Detection and Steganography

« Reply #4 on: 10 / February / 2014, 19:52:47 »

Quote from: philmoz on 10 / February / 2014, 16:15:46

Interesting idea; but I think you will face some challenges trying to do this with a Canon P&S camera.
Here are some of my initial thoughts.

CHDK does not modify the camera firmware, it loads every time the camera is started from special files on the SD card.

Unlocking the SD card, or formatting it in the camera will remove CHDK and all your security options will be gone.

Your second goal, to embed data in the image, would be the easiest - CHDK has access to the raw captured data before the camera converts it to a JPEG. You could fairly easily modify the captured data to embed a watermark. Remember you are working with raw sensor data (in a Bayer format) not RGB pixels.

For the other two you would probably need to work with the live view data shown in the LCD. This has much lower resolution than the sensor is capable of (720x240 for most cameras) and is in YUV format. Being able to detect a logo or particular person, at different angles, positions and lighting at such a low resolution could be a challenge.

There is also not a lot of CPU power available. I don't know what the state of the art in recognition algorithms is like these days; but I imagine it still needs a fair bit of horsepower.

Phil.

Hey Phil,

Thanks for your quick response! You're of course right that replacing the SD card would remove the security features, this project is more just to see what's possible with CHDK. I'll keep using the live view data in mind, I'm not sure there is another way to prevent image capture for the first point without using this data. The third point could be done with the higher resolution image (the image should just be saved with the face obscured rather than obscure it in real-time), but I'll need to figure out how face detection is done in these cameras.

Logged

Vegetothe1st

19

Re: Advice Requested - CHDK Security Project - Object Detection and Steganography

« Reply #5 on: 10 / February / 2014, 19:56:09 »

Quote from: nafraf on 10 / February / 2014, 16:42:33

Quote from: Vegetothe1st on 10 / February / 2014, 14:04:50
3. Detect a particular user's face in an image
How many faces in your library? Some cameras has built in function to recognize particular user's face, camera shows their names on LCD.

I'd just start with my own for now, it would be great if I had access to the functionality you mentioned or a way to code it into a camera that didn't have it. I'm not sure if it's a new feature that was included in the later processors because of an increase in processing power or just a new feature that happened to be developed at the time.

« Last Edit: 15 / February / 2014, 05:56:26 by Vegetothe1st »

Logged

reyalp

14114

Re: Advice Requested - CHDK Security Project - Object Detection and Steganography

« Reply #6 on: 10 / February / 2014, 23:37:40 »

Quote from: Vegetothe1st on 10 / February / 2014, 19:56:09

I'd just start with my own for now, it would be great if I had access to the functionality you mentioned or a way to code it into a camera that didn't have it. I'm not sure if it's a new feature that was included in the later processors because of an increase in processing power or just a new feature that happened to be developed at the time.

I suspect it depends on hardware (Likely the Digic DSP rather than just raw CPU speed) but it's kind of irrelevant, because it would be impractical to try to port Canon firmware features like this anyway.

I did some very preliminary investigation of the face detect code in http://chdk.setepontos.com/index.php?topic=8243.msg86889#msg86889 - This isn't directly applicable to your project, but might give you some starting places to look in the firmware code.

Regarding #2, as phil says modifying the raw data is relatively easy, but keep in mind that unless you are saving raw or DNG the resulting data will be encoded with lossy jpeg compression.

Logged

Don't forget what the H stands for.

Vegetothe1st

19

Re: Advice Requested - CHDK Security Project - Object Detection and Steganography

« Reply #7 on: 15 / February / 2014, 06:00:21 »

It seems that the camera I'll be using is the Canon PowerShot A2500. As far as I can tell it has a CHDK port that supposedly works (firmware is available on the downloads page and the porting thread has a decent amount of activity). This camera has a DIGIC 4 processor, so face detection should be a go.

A2500 Porting Thread: http://chdk.setepontos.com/index.php?topic=10626.0

Logged

Vegetothe1st

19

Re: Advice Requested - CHDK Security Project - Object Detection and Steganography

« Reply #8 on: 04 / March / 2014, 02:47:45 »

Started digging into this a bit after finally getting all the hardware, I'm having trouble starting however. I planned on doing the Stego portion first as it seems the simplest, but I'm having trouble understanding the order in which code is run. It seems that main.c loops forever in its core_spytask() function waiting for the raw_data_available variable to be set so it can process the data. This variable is set in /platform/generic/capt_seq.c, apparently when a capture is in the process of occurring; raw.c then processes and saves the data.

However, this doesn't seem to reference writing JPEG images at all. The code related to JPEGs seems to be contained in remotecap.c and /platform/generic/filewrite.c. raw.c does make one reference to remotecap_get_target(), but it's unclear to me how remotecap.c is ever aware the camera wants to save JPEG images.

I believe that modifying the raw image in raw.c to embed time/date information should be straightforward enough, but this will likely be lost (I believe) when the image is transformed into a JPEG via the functionality in filewrite.c. Ideally I could wait until this process had been done and then embed time/date information (so the information remains decipherable), but I can't figure out where this finalized JPEG would be available to modify. I'd prefer to embed the image in the JPEG rather than the RAW image because JPEGs are the more common image type.

Any pointers are appreciated as always.

Logged

srsa_4c

4451

Re: Advice Requested - CHDK Security Project - Object Detection and Steganography

« Reply #9 on: 04 / March / 2014, 13:56:12 »

Quote from: Vegetothe1st on 04 / March / 2014, 02:47:45

Started digging into this a bit after finally getting all the hardware, I'm having trouble starting however. I planned on doing the Stego portion first as it seems the simplest, but I'm having trouble understanding the order in which code is run. It seems that main.c loops forever in its core_spytask() function waiting for the raw_data_available variable to be set so it can process the data. This variable is set in /platform/generic/capt_seq.c, apparently when a capture is in the process of occurring; raw.c then processes and saves the data.

However, this doesn't seem to reference writing JPEG images at all.

JPEGs are made and saved by Canon firmware tasks, not by CHDK. (Note that I haven't tried to spot mistakes - if any - in the above)
If you need to access the JPEG data before it's saved, you'll need one of the filewritetask hooks. filewrite_main_hook() in platform/generic/filewrite.c is the first CHDK function that gets called in the JPEG saving process. This hook is only called once for each JPEG file on DryOS < r50 cameras. Later DryOS cameras save large JPEG files in multiple passes, you often won't see all parts of the file at once. The JPEG file data itself is broken into one or more chunks. On DryOS >= r50 the EXIF part of big JPEGs is the last chunk, it gets written to the beginning of the file. For smaller files and for earlier cameras, the EXIF is the first chunk.

Quote

I believe that modifying the raw image in raw.c to embed time/date information should be straightforward enough, but this will likely be lost (I believe) when the image is transformed into a JPEG via the functionality in filewrite.c.

If the CHDK raw hook is placed correctly, changes to the raw data will make it into the JPEG (you can try this easily, modify smaller parts of the raw buffer and check the resulting JPEGs). There's one thing we know about which carries information about the original image: "i-contrast" - it stores some image data elsewhere and applies it after our raw hook.

Logged