Pixel level image processing - LUA Scripting - CHDK Forum
« previous next »
Pages: [1] 2   Go Down

Pixel level image processing

  • 10 Replies
  • 10466 Views
*

Offline travis4

  • *
  • 3
Pixel level image processing
« on: 21 / February / 2013, 11:37:43 »
Advertisements
I'm trying to write a script which takes an action when the camera takes a photo containing a certain image (say a big red box).

My plan is to do this by examining individual pixels of the photo, but I've been unable to find an easy way to do this. I haven't been able to find any non-c based Lua libraries to read DNG or JPEG files.

Do you guys have any ideas? Are there any helpful libraries I haven't been able to find? Is there functionality already in place to make reading pixel values (reading off the display screen should work too) possible without rewriting a JPEG encoder? How hard would it be to read pixel values based purely off the binary DNG or JPEG files?

Thanks in advance for your help.
Logged
*

Offline waterwingz

  • ******
  • 12139
    • Waterwingz
Re: Pixel level image processing
« Reply #1 on: 21 / February / 2013, 12:44:06 »
Quote from: travis4 on 21 / February / 2013, 11:37:43
I'm trying to write a script which takes an action when the camera takes a photo containing a certain image (say a big red box).
That's almost certainly a complex task to do properly and I would suggest probably beyond the processing capability of CHDK's  scripting.   It would even be a tough assignment to do in "C" as an embedded program.

Quote
Reading off the display screen should work too
That might just make the problem manageable. The display screen buffer(s) are available and fairly small so you don't need to deal with JPG compression or the size/complexity of a RAW/DNG image. Might need to add a hook so that you can get the buffer address in Lua but that would not be hard.

Your other option might be to use the Motion Detection script functions.  You can divide the LCD screen up into small areas and watch for shifts in YUV or RGB individual levels.   So if the object you are looking for is sufficiently different than the normal background,  detecting it based on a color shift is theoretically possible.  It will take some experimenting to see how well that will work for you however.


« Last Edit: 21 / February / 2013, 18:14:15 by waterwingz »
Logged
Ported :   A1200    SD940   G10    Powershot N    G16
*

Offline Davo

  • ***
  • 188
Re: Pixel level image processing
« Reply #2 on: 22 / February / 2013, 09:41:16 »
Check out the Aforge.NET framework. It has all sorts of pattern recognition routines and is extremely well documented. It includes the source code which you would need to translate to Lua. I use it for guiding telescopes on craters and sunspots.

You will need to write a function like color = get_lcd_pixel(x,y) which would be a very useful addition to chdk.

Dave
Logged
*

Offline waterwingz

  • ******
  • 12139
    • Waterwingz
Re: Pixel level image processing
« Reply #3 on: 22 / February / 2013, 19:24:05 »
Quote from: Davo on 22 / February / 2013, 09:41:16
You will need to write a function like color = get_lcd_pixel(x,y) which would be a very useful addition to chdk.
@Dave :  pretty easy to add - probably needs to be more like  color = get_lcd_pixel(type,x,y) where type indicates Y,U,V,R,G, or B  - what kind of things were you thinking it might be used for?
Logged
Ported :   A1200    SD940   G10    Powershot N    G16
*

Offline Davo

  • ***
  • 188
Re: Pixel level image processing
« Reply #4 on: 23 / February / 2013, 08:04:30 »
Quote
what kind of things were you thinking it might be used for?


I don't have a real need right now - I am too busy using your md_get_cell_val() in my video scripts. Thanks again.

I can imagine adding AForge.NET type functions to chdk would open up the whole machine vision field. Adding external control capability via usb or leds or sound would also allow robotics control. Your camera could take your model car for a drive using obstacle avoidance routines. :)

Image processing like AForge.NET uses is a real cpu hog. I am not sure how lua script calls compare speed wise to the compiled builds so it might be better to include some high level calls in chdk instead.

Dave
Logged
*

Offline travis4

  • *
  • 3
Re: Pixel level image processing
« Reply #5 on: 23 / February / 2013, 15:24:43 »
Thank you guys both for the great suggestions! I wasn't aware the display buffers were so easy to access - my plan is to implement the image detection in the compiled build (using the display buffer). If things go well I'll look into adding a "color = get_lcd_pixel(type,x,y)" hook for lua scripting.

Also, I will probably use a simpler detection algorithm than what AForge.NET provides, but thanks for pointing me to it. It looks great, and will be a great upgrade to my naive algorithm if I get the time!
Logged
*

Offline reyalp

  • ******
  • 14082
Re: Pixel level image processing
« Reply #6 on: 23 / February / 2013, 17:32:24 »
Quote from: Davo on 23 / February / 2013, 08:04:30
Image processing like AForge.NET uses is a real cpu hog. I am not sure how lua script calls compare speed wise to the compiled builds so it might be better to include some high level calls in chdk instead.
Since we have loadable modules, an image processing module could be written in C or assembler without unnecessarily bloating CHDK for people who don't need it.

To make something like get_lcd_pixel useful, you would probably want to expose much of the information that is currently used by the PTP live view code for the dimensions an positioning of the viewport.

Somewhat OT:
Lua normally has the capability for modules to be in shared libraries (e.g. require('foo') will search for both Lua modules and shared libraries in some defined search path.) This capability doesn't currently exist for flt modules in CHDK, but implementing it could be quite useful.
Logged
Don't forget what the H stands for.
*

Offline Davo

  • ***
  • 188
Re: Pixel level image processing
« Reply #7 on: 24 / February / 2013, 06:17:19 »
Quote
Since we have loadable modules, an image processing module could be written in C or assembler without unnecessarily bloating CHDK for people who don't need it.

That is good to know. I have a Windows focus meter (C#) which is very handy for astro work but it obviously needs a PC to run it. Maybe I will try to port it to chdk where it will run on the LCD. It would be a much neater solution.

Is there a Lua module for dummies page?
Logged
*

Offline waterwingz

  • ******
  • 12139
    • Waterwingz
Re: Pixel level image processing
« Reply #8 on: 24 / February / 2013, 12:43:56 »
I took a few minutes and created a simple md_get_pixel_val() function  (patch file attached - PM me if you'd like a version compiled for a particular camera / firmware version).

Here's a little test script I used to draw a small target box on the screen and report the six pixel values :
Code: [Select]
--[[
@title Pixel Value Test
]]
screenwidth=360
screenheight=240
if ( get_mode() == false ) then
    set_record(1)
    while ( get_mode() == false ) do
        sleep(100)
    end
end
sleep(2000)

repeat
      row = screenheight/3
      col = screenwidth/2
           press("shoot_half")
           repeat sleep(100) until get_shooting() == true
           y = md_get_pixel_val(1,col,row)
           u = md_get_pixel_val(0,col,row)
           v = md_get_pixel_val(2,col,row)
           r = md_get_pixel_val(3,col,row)
           g = md_get_pixel_val(4,col,row)
           b = md_get_pixel_val(5,col,row)
           for i=1, 50, 1 do
               draw_string( 10, ((screenheight/3)*2) , string.format("Y:%d U:%d V:%d R:%d G:%d B:%d",y,u,v,r,g,b) , 258, 263)
               draw_rect( col-6, row-6, col+6,row+6, 263, 3)
               sleep(100)
           end
           draw_clear()
           release("shoot_half")
           repeat sleep(100) until get_shooting() == false
until (false)
It seems to respond to color changes although its hard to get a clean R, G, B separation.

@philmoz :  if you can spare a couple of minutes,  would you check for me to see if I correctly interpreted the pixel code you put into motion_detect.c ?  TIA ..

Next step is to see if I can extend the script to locate a colored object against a neutral background.
« Last Edit: 24 / February / 2013, 13:08:38 by waterwingz »
Logged
Ported :   A1200    SD940   G10    Powershot N    G16
*

Offline philmoz

  • *****
  • 3450
    • Photos
Re: Pixel level image processing
« Reply #9 on: 24 / February / 2013, 16:30:12 »
Quote from: waterwingz on 24 / February / 2013, 12:43:56
@philmoz :  if you can spare a couple of minutes,  would you check for me to see if I correctly interpreted the pixel code you put into motion_detect.c ?  TIA ..

Haven't tested it; but I think the following change is needed:
change
    int y = y_pixel * vid_get_viewport_width();
to
    int y = y_pixel * vid_get_viewport_byte_width() * vid_get_viewport_yscale();

Phil
Logged
CHDK ports:
  sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
  g12 (1.00c, 1.00e, 1.00f & 1.00g)
  sx130is (1.01d & 1.01f)
  ixus310hs (1.00a & 1.01a)
  sx40hs (1.00d, 1.00g & 1.00i)
  g1x (1.00e, 1.00f & 1.00g)
  g5x (1.00c, 1.01a, 1.01b)
  g7x2 (1.01a, 1.01b, 1.10b)
Pages: [1] 2   Go Up
« previous next »
 

Related Topics