edge overlay

[hiker_jon]

Hi,

I have written some code to create an edge image of the previous shot and overlay it on the screen.  The code is just something quick to test out the feasibility.  Hopefully something like this will be useful when creating a stereo pair of images.
I have used the Zebra mode configuration to scale the threshold (you should not have zebra mode enabled when running this.)  Different Zebra modes give different edge thresholds.
I have also added a delay so that the screen is updated more slowly.

It appears to work great. 
I am using allbest build 50.  Paste the code into main.c after the line #include "motion_detector.h".

Then add the line
edge_overlay();

after the
histogram_process();
line.

CODE REMOVED>  New version is here:edge overlay

[Microfunguy]

Hi,

I have written some code to create an edge image of the previous shot and overlay it on the screen.  The code is just something quick to test out the feasibility.  Hopefully something like this will be useful when creating a stereo pair of images.

Great !

I guess it will have to be in SDM 1.6x (or even 1.70) as I have just released 1.60.

This looks very promising, will need a bit of development.

1. Try to reduce the amount of noise so we just get clean outlines in most situations.

2. Works fine on half-press and overlays live screen with edge-processed viewport image.
    On full-press, does not work, image overlaid with wide horizontal bands.

Edge-detection and noise-removal is quite an extensive and well-documented topic.
Presumably, there are ImageJ examples of such processing.





David

[hiker_jon]

1. Try to reduce the amount of noise so we just get clean outlines in most situations.

2. Works fine on half-press and overlays live screen with edge-processed viewport image.
    On full-press, does not work, image overlaid with wide horizontal bands.

1: Try increasing thresh. May need different thresh for different cameras or situations.  We should make it into a configurable parameter.
2: What camera do you have?  On my a720is it works fine even when shooting. 

Do you have Zebra mode enabled?  Should be off.

May be timing considerations.  Digic 2 vs 3? 

The calculation could be split up like the histogram calculation.  To test on your camera change the number of lines in the edge calc and overlay display loops.  Say start with something like for(y=100; y<150; y++).

[Microfunguy]

We should make it into a configurable parameter.

Yes, if there are any shortcuts left (in SDM).

What camera do you have?

A620 (DIGIC II)

start with something like for(y=100; y<150; y++).

0 to 150 certainly works.

Need to greatly reduce the refresh-rate for this overlay, cannot read other items on screen.


I will have to study the histogram code.


David

[Microfunguy]

Looking at the edge overlay indoors, the noise is random points that constantly change position.

Maybe the overlay could be AND'ed with itself (while shutter half-pressed) ten times so that the noise was excluded and only the static edges remained.



David

[hiker_jon]

Looking at the edge overlay indoors, the noise is random points that constantly change position.

This sounds like a threshold problem, it should be higher for noisy indoor images. 

I have updated the code with a delay for screen updating and the ability to use the Zebra mode as a way to change the threshold.  I think blink1 gives the smallest threshold, subsequent modes will give higher thresholds.  Zebra should be turned off, however.

[Microfunguy]

I have updated the code

I will try that now.

I have just been researching sobel and canny edge detectors, gaussian smoothing, median filters, etc.

In comparison, your code is remarkably brief and works well.

I do not really understand how it works.


David

[Microfunguy]

Yes, that is very interesting and useful.

Could you do one more thing ?

Split the calculation into two parts so that it works when I shoot an image with my A620 ?


Thanks.



David

[Microfunguy]

I have just made the following changes that work really well, you can see the noise dissolve-away as you keep the button pressed :-

static  float thresh = 10.0f;

......

      if (kbd_is_key_pressed(KEY_SHOOT_HALF)) {
                       thresh+= 0.5;
                       if(thresh>50.0) thresh = 10.0f;

......

(int)thresh

[PhyrePhoX]

Great stuff! I guess we also have an enhanced pano function,thanks!

[Microfunguy]

I guess we also have an enhanced pano function,thanks!

For panos, it will be interesting to see how different the height of the images is in the overlap regions due to distortion.

For stereo with a single camera, if you point the camera down at a sheet of paper on a desk-top and capture the outline, if you then move sideways very slightly for the second shot you can see how very different the perspectives are, especially at wide-angle.

Your brain would have great difficulty fusing such an image.

You only need two  distinct points in order to align two images.



David

[hiker_jon]

[I do not really understand how it works.

Hi,

The display of the c code cuts off the right side so that may make things a little difficult to understand.
Here is an explanation in words:

The edge calculation I do a simple absolute value of the horizontal difference:  abs(pix - pix[i-1]).  (Note: maybe pix[i+1] - pix[i-1] would be better, I'll have to try this)  The layout of the buffers is a little strange so that makes for some complications.  To explain:
The viewport has 6 bytes per 2 pixels in the format U,Y1,V,Y2,Y3,Y4.  This has 4 luminosity values (Y1,Y2,Y3,Y4) and one set of color values (U,V) for the two pixels.  I think the 4 luminosity values are used for antialiasing in the calculation of the RGB dots for the lcd.
The screen overlay has one byte per pixel.  That byte is a code that indexes into a limited pallet of transparent and opaque colors.
I make use of only two codes, transparent and opaque white.  (Maybe using more would improve the edge display... another thing to try out).
Edge Calculation:
Done in place on the stored viewport data.  Done once on a shutter half press.
I use the fact that the viewport has more bytes per pixel than the screen overlay to do the edge calculation in-place, rather than having to allocate a separate buffer for the edge image.  I calculate the edge value for the first pixel as abs(Y1 - previous Y1) and store it in Y2.  For the second pixel I calculate abs (Y3-previousY3) and store it in Y4.
Edge Display:
This is constantly repainted while in shoot mode.  I look at 2 pixels at a time.  That is 6 bytes of viewport and 2 bytes of the screen overlay.  I compare the edge values in Y2 to a threshold that determines whether I write a transparent or opaque pixel to the screen overlay.  Same with Y4.


Hope this helps.

[Microfunguy]

Hope this helps.

Thanks Jon.

I will have to read this a few times.

I have get absolutely nowhere trying to get this to work on my A620 AFTER I have pressed the shutter.

I tried splitting into two or three operations but could not get it to work.

For some reason, even limiting the number of rows to 0 to 150 no longer works, it did a few days ago.

The overlay after shooting is some of the edges combined with random patterns.

This is going to be extremely useful if it works on DIGIC II cameras.

I will continue tomorrow.


David

[hiker_jon]

I have get absolutely nowhere trying to get this to work on my A620 AFTER I have pressed the shutter.

Here is a new version that is split into stages, minimizes the amount of calculation when shooting, and has a vertical as well as horizontal edge calculation.  Now it also includes horizontal and vertical shifts using the up,down,left right keys.  Do this in alt mode to avoid changing the camera options.

Code: C++

1.  
2. // Code to test the idea of an edge overlay
3. // Put this after  the line #include "motion_detector.h" in main.c
4. #include "gui_draw.h"
5. // until the edge thresh is put into conf structure I have hardcoded the threshold  see "thresh" below
6. #define MARGIN 30
7. #define CALCYMARGIN 3
8. #define DELAY 10
9. #define MEMDELAY 10
10. #define EDGECOLOR 0x26
11. #define NSTAGES 8
12. #define XINC 6
13. #define YINC 2
14. void edge_overlay(){
15.         static char *imgbuf = 0;
16.         static int inmem=0;
17.         static int viewport_height;
18.         static int viewport_width;// screenwidth * 3
19.         static int viewport_size;
20.         static int imgmem = 0;
21.         static int ymin = 0;
22.         static  int thresh = 30;
23.         static int delay = 0;
24.         static int memdelay = 0;
25.         static int xoffset = 0;
26.         static int yoffset = 0;
27.         static unsigned char *img;
28.         int i, hi, c;
29.         int x, y, h, v, ymax, y1, x1;
30.         char * ptrh1;
31.         char * ptrh2;
32.         char * ptrv1;
33.         char * ptrv2;
34.         char * optr;
35.        
36.         //if(!conf.edge_thresh) return;
37.        
38.     img=((mode_get()&MODE_MASK) == MODE_PLAY)?vid_get_viewport_fb_d():((kbd_is_key_pressed(KEY_SHOOT_HALF))?
39. vid_get_viewport_fb():vid_get_viewport_live_fb());
40.     if (img==NULL){
41.            img = vid_get_viewport_fb();
42.         }
43.     viewport_height = vid_get_viewport_height();
44.         viewport_width = screen_width * 3;
45.     viewport_size = viewport_height * screen_width;
46.         if(imgbuf == 0) imgbuf = malloc(viewport_size * 3);
47.                
48.         if((mode_get()&MODE_MASK) != MODE_PLAY) {
49.                 if (kbd_is_key_pressed(KEY_RIGHT)) {
50.                         xoffset -=XINC;
51.                         delay = DELAY;
52.                 }
53.                 if (kbd_is_key_pressed(KEY_LEFT)) {
54.                         xoffset +=XINC;
55.                         delay = DELAY;
56.                 }
57.                 if (kbd_is_key_pressed(KEY_DOWN)) {
58.                         yoffset -=YINC;
59.                         delay = DELAY;
60.                 }
61.                 if (kbd_is_key_pressed(KEY_UP)) {
62.                         yoffset +=YINC;
63.                         delay = DELAY;
64.                 }
65.  
66.                 if (kbd_is_key_pressed(KEY_SHOOT_HALF)) {
67.                         if(!(memdelay++)) {
68.                                 memcpy(imgbuf,img,viewport_size * 3);
69.                                 ymin = CALCYMARGIN;
70.                                 inmem = 1;
71.                                 xoffset = 0;
72.                                 yoffset = 0;
73.                         }
74.                         return;
75.                 }
76.                 else {
77.                         if(memdelay) memdelay++;
78.                         if(memdelay > MEMDELAY) memdelay = 0;
79.  
80.                         if (inmem) {
81.                                 ymax = ymin + (screen_height - 2 * CALCYMARGIN) / NSTAGES;
82.                                 if(ymax > screen_height - CALCYMARGIN) ymax = screen_height - CALCYMARGIN;
83.                                 for (y=ymin; y<ymax; y++) {
84.                                         ptrh1 = imgbuf + y * viewport_width + 7;
85.                                         ptrh2 = imgbuf + y * viewport_width - 5;
86.                                         ptrv1 = imgbuf + (y + 1) * viewport_width + 1;
87.                                         ptrv2 = imgbuf + (y - 1) * viewport_width + 1;
88.                                         optr = imgbuf + y * viewport_width + 3;
89.                                         for (x=12; x<(screen_width- 4) * 3; x+=6) {
90.                                                 h = ptrh1[x] - ptrh2[x];
91.                                                 if(h  < 0) h = -h;
92.                                                 v = ptrv1[x] - ptrv2[x];
93.                                                 if(v  < 0) v = -v;
94.                                                 optr[x] = h + v;
95.                                                 h = ptrh1[x + 3] - ptrh2[x + 3];
96.                                                 if(h  < 0) h = -h;
97.                                                 v = ptrv1[x + 3] - ptrv2[x + 3];
98.                                                 if(v  < 0) v = -v;
99.                                                 optr[x + 2] = h + v;
100.                                         }
101.                                 }
102.                                 if(ymin < screen_height-CALCYMARGIN) delay = DELAY;
103.                                 ymin += (screen_height - 2 * CALCYMARGIN) / NSTAGES;
104.                         }
105.                         if((ymin >= screen_height-CALCYMARGIN)&& ((gui_get_mode() == GUI_MODE_NONE) || (gui_get_mode() == GUI_MODE_ALT)))
106.                         {
107.                                 if(++delay > DELAY) {
108. //                                      thresh = (conf.edge_thresh - 1) * 10;
109.                                         for (y=MARGIN; y<screen_height-MARGIN; y++) {
110.                                                 y1 = y + yoffset;
111.                                                 if((y1 < CALCYMARGIN) || (y1 >= screen_height - CALCYMARGIN)) {
112.                                                         for (x=MARGIN; x < screen_width - MARGIN; x+=2) {
113.                                                         draw_pixel(x, y, 0);
114.                                                         draw_pixel(x+1, y, 0);
115.                                                         }
116.                                                 }
117.                                                 else {
118.                                                         for (x=MARGIN; x < screen_width - MARGIN; x+=2) {
119.                                                                 x1 = x + xoffset;
120.                                                                 if((x1 < 12) || (x1 >= screen_width-13)) {
121.                                                             draw_pixel(x, y, 0);
122.                                                             draw_pixel(x+1, y, 0);
123.                                                                 }
124.                                                                 else {
125.                                                             c = 0;
126.                                                                         if(imgbuf[y1 * viewport_width + x1 * 3 + 3]  > thresh)
127.                                                                                 c = EDGECOLOR;
128. //                                              else if(imgbuf[y * viewport_width + x * 3 + 3]  > thresh/2)
129. //                                                      c = 0x01;
130.                                                             draw_pixel(x, y, c);
131.                                                             c = 0;
132.                                                                         if(imgbuf[y1 * viewport_width + x1 * 3 + 5]  > thresh)
133.                                                                                 c = EDGECOLOR;
134. //                                              else if(imgbuf[y * viewport_width + x * 3 + 5]  > thresh/2)
135. //                                                      c = 0x01;
136.                                                             draw_pixel(x+1, y, c);
137.                                                                 }
138.                                                         }
139.                                                 }
140.                                         }
141.                                         delay=0;
142.                                 }
143.                         }
144.                 }
145.         }
146.        
147. }
148.  
149. //End of edge detector code.  next put                  edge_overlay();                 after the line histogram_process();
150.  

[Microfunguy]

Here is a new version that is split into stages, minimizes the amount of calculation when shooting, and has a vertical as well as horizontal edge calculation.

thanks Jon, I will check that next.

I have been testing this all day,

Sometimes just rows 30 to 70 does not playbacl properly, other times 30 to full height does playback properly.

May be connected to autofocus.

After focusing, pressing full-shutter quickly seems to work more often.


David