Shot Histogram Request - page 12 - CHDK Releases - CHDK Forum
« previous next »
Pages: 1 ... 7 8 9 10 11 [12] 13 14 15 16 17 ... 47   Go Down

Shot Histogram Request

  • 467 Replies
  • 150587 Views
*

Offline lapser

  • *****
  • 1093
Re: Shot Histogram Request
« Reply #110 on: 15 / February / 2013, 13:33:55 »
Advertisements
Quote from: reyalp on 14 / February / 2013, 21:16:31
It's very unlikely this is a deliberate error message
I'm pretty sure it's related to the script accessing variables as they're being changed by build_shot_histogram() in a different thread. I added a busy flag to build_shot_histogram(), so the script thread waits until it's finished. I don't have a busy flag for the script call to get_shot_meter(), so that could be the problem. I'll have to be absolutely certain both busy flags are never true at the same time, or that would hang both threads.

I'm thinking it would be best to make both flags "volatile" so I know the values are always current. Does that sound like a good idea?
===========

I made an interesting discovery from my test last night. As the light fades, changing the shutter speed to compensate seems to overshoot the desired amount. Then it corrects on the next shot, and oscillates 1 or 2 EV96 units. It only becomes visible with rapidly changing light after sundown as the sky darkens, and the compression artifacts bring it out.

Changing the ISO doesn't seem to have this problem, though. It shows no oscillations in the log, and looks smoother in the video. I explain it more in the description on YouTube.

There's also a hot or dead red pixel that shows up near the end of the video. It looked like a comet at first. I thought maybe I captured the meteorite that hit Russia a few hours later  :P
[EDIT]That's not a hot pixel. It's a small lens reflection from the street light at the bottom.

http://www.youtube.com/watch?v=Lx53E5AeXWU#ws
« Last Edit: 18 / February / 2013, 13:03:21 by lapser »
Logged
EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos
*

Offline reyalp

  • ******
  • 14128
Re: Shot Histogram Request
« Reply #111 on: 16 / February / 2013, 21:14:21 »
Quote from: lapser on 15 / February / 2013, 13:33:55
I'm pretty sure it's related to the script accessing variables as they're being changed by build_shot_histogram() in a different thread.
Having not looked at the code, I don't know. However, the only way I could see this causing the behavior you describe is if your code got the values and did something with them that corrupted the Lua state or other memory. Lua isn't going to care that C code in one task is reading variables that are "out of date" with respect to some other task.

Note that lua API calls should only ever be made from the kbd_task, with the possible exception of loading a script like ptp does (I'm not sure if that's 100% safe either, but it seems to work...)
Quote
I'm thinking it would be best to make both flags "volatile" so I know the values are always current. Does that sound like a good idea?
I prefer something like
Code: [Select]
static int some_busy_flag;
int get_some_busy_flag() {return some_busy_flag;}
Because
1) this also forces the variable to be loaded fresh from ram, just like volatile
2) if in the future the ready state gets more complicated (perhaps by using proper task synchronization structures) then the calling code doesn't need to change.
3) People tend to attribute magic to volatile which it does not possess.
Logged
Don't forget what the H stands for.
*

Offline lapser

  • *****
  • 1093
Re: Shot Histogram Request
« Reply #112 on: 17 / February / 2013, 00:08:53 »
Quote from: reyalp on 16 / February / 2013, 21:14:21
Code: [Select]
static int some_busy_flag;
int get_some_busy_flag() {return some_busy_flag;}

//this is what I have right now
static int hbusy; //change to volatile?
void build_shot_histogram()
{
  hbusy=1;
  //calculations that take about 30msec
  hbusy=0;
}

int get_shot_meter()
{
  while(hbusy)msleep(10);
  //calculations, with call to log10(x)
}
So the problem isn't reading the busy flag, it's making sure it is stored immediately in build_shot_histogram().
=====

I modified get_shot_meter() to try to reduce the oscillations visible in the video as the light fades. It seems to have worked pretty well.

Code: [Select]
result= //calculations
if(result>0)result--;
if(result<0)result++;
return result;
[EDIT]This doesn't work right with an ev correction, so it has to be done in the script, not in CHDK. My script makes the exposure darker as it gets darker (ev correction), so the shot meter isn't returning 0 for no change.

[EDIT 2]Changed my mind again and put it back in the C code. I test if set_shot_interval(..) is active, and if it is, compare the current result with the last result, and reduce the amount of change by 1 as above.
« Last Edit: 18 / February / 2013, 12:14:37 by lapser »
Logged
EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos
*

Offline lapser

  • *****
  • 1093
Re: Shot Histogram Request
« Reply #113 on: 18 / February / 2013, 13:01:30 »
I've done several tests with over 8,000 pictures and haven't had the script stop. I did change the flag variables to volatile for now, so I know they are stored immediately and they'll test correctly in the other thread. I think this is one of the accepted uses of volatile:
http://www.drdobbs.com/cpp/volatile-the-multithreaded-programmers-b/184403766

I moved everything to shot_histogram.c so it will be easier to make into a module in the future. All that's left in raw.c is the slightly modified call to build_shot_histogram(). There's also a call to the pre-shot delay routine in capt.seq.c. The rest of the new calls are all in luascript.c, and there are initialization and termination calls in script.c.

There's lots of cleanup to do, but I think the time lapse and shot meter parts are working now. I still need to figure out why the shot meters don't work on the SX50.

I also need to finish up the shot histogram part, and see if I can return a shot meter value based on the histogram, so you can get the correct exposure on the next shot. I should be able to add up a histogram for all 4 shot meters, and save a few exposure values (ETTR, ETTL ?) for each meter. Meter #1 will hold a valid histogram between shots for use with get_histo_range(..)

Here's the latest 2 test videos. The first is from Skinner's Butte again, but with a 650 msec shot interval, or 1.5 shots per second. You can go faster, but the shots tend to back up saving to the card occasionally. Of course, as it gets dark, the shutter open time increases and lowers the shot interval. I capped the shutter time at 8 seconds here, and set the exposure compensation to -192 ev at -500 Bv.

The second video is a 12 hour, overnight sequence using my external lithium battery with the G1X. It still didn't run out of power. The battery only weighs one pound, so I should be able to carry it backpacking. The shot interval was set to 750 msec, but the shutter time of 10 seconds makes the video go much faster overnight. The fog really rolls in at sunrise, so the last half of the video shows the fog getting lighter. It's a good test of the exposure change smoothness, but it's not very interesting to watch. There were some birds that flew by in the fog a few times, and appeared on single frames.

http://www.youtube.com/watch?v=l0KdUiCoQ8M#ws

http://www.youtube.com/watch?v=SLGAmLpTXxo#ws
Logged
EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos
*

Offline lapser

  • *****
  • 1093
Script interrupted
« Reply #114 on: 22 / February / 2013, 13:32:01 »
I'm still having occasional script interrupted errors even with all the thread safe coding I've been doing. I added some flags so I could tell where the script was interrupting. Of course, it's not in the place I thought it was.

Since installing the flags, the script interrupted on 2 occasions AFTER capt_seq_hook_set_nr() is called, and BEFORE raw_save_file() is called. This time period is around 250 msec plus the shutter open time. During this time period, core_spytask() and the script are both active in different threads.

The flags also show that the script error happens when the script is waiting for the shot to be ready. The last flagged routine was get_shot_ready(). However, during the loop, I also am calling other functions, including:

is_pressed("display")
is_pressed("menu")
is_pressed("set")
sleep(10)
get_shot_ready()

I've added a flag in each of these routines to try to determine where the script error happens, but I haven't been able to trigger the error with the new flags yet. I think core_spytask() is calling is_pressed() too, so it might be possible that is_pressed() is being called from different threads at the same time.

Anyway, I'm getting a little closer to an answer, I hope. If anyone has an idea how a multi-threading problem could cause a script interrupted error at this point in the shooting cycle, please let me know. The error message is "set" the last 3 times it happened. Here's the end of the last log file:

1524 3519 3470 30 50 411 88 -3 1/5.78
1525 3520 3450 40 30 411 89 -2 1/5.69
1526 3521 3460 40 30 411 89 -2 1/5.61
set
*** TERMINATED ***

This is what I'm printing in the log: (G1X)
shot#,  exposure#,  pre-shot delay time (4 sec interval), shot meter time, script time, sv96, bv correction, ev96 change, shutter time

=========
The exposure time, tv96, isn't as accurate as sv96. So when you set tv96 based on get_shot_meter(), it sometimes over-corrects and causes small oscillations in brightness in the time lapse video. These are only visible as it gets dark. Anyway, I modified get_shot_meter() to under_correct by 1 ev96, which appears to have eliminated or greatly reduced the oscillations.

My test time lapses lately have been deadly dull, since it's just watching fog get darker or lighter. But at least they change exposure smoothly. Here's the one from this morning (no script interrupt, unfortunately):

http://www.youtube.com/watch?v=fM0TV7bm4tA#ws
Logged
EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos
*

Offline reyalp

  • ******
  • 14128
Re: Shot Histogram Request
« Reply #115 on: 23 / February / 2013, 18:13:28 »
Quote from: lapser on 22 / February / 2013, 13:32:01
Anyway, I'm getting a little closer to an answer, I hope. If anyone has an idea how a multi-threading problem could cause a script interrupted error at this point in the shooting cycle, please let me know.
I don't really see how you concluded this is a "multi-threading" problem. The script stuff all runs in one task. Other threads doing stuff won't cause it to randomly die, unless they are corrupting memory or something like that.
Quote
The error message is "set" the last 3 times it happened. Here's the end of the last log file:
As I said before, this isn't an error message in any normal sense. It's garbage, presumably left around due to the is_pressed("set") in your code.

Quote
I think core_spytask() is calling is_pressed() too, so it might be possible that is_pressed() is being called from different threads at the same time.
This would not be a problem.
Logged
Don't forget what the H stands for.
*

Offline lapser

  • *****
  • 1093
Re: Shot Histogram Request
« Reply #116 on: 23 / February / 2013, 18:24:13 »
Quote from: reyalp on 23 / February / 2013, 18:13:28
As I said before, this isn't an error message in any normal sense. It's garbage, presumably left around due to the is_pressed("set") in your code.
That's a help, thanks.
is_pressed("set") is the last function I called before sleep(10) and then get_shot_ready(). I switched it so is_pressed("display") is last. We'll see if that ends up as the next error message, if it ever happens again.
Logged
EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos
*

Offline lapser

  • *****
  • 1093
Tonight's Time Lapse Test
« Reply #117 on: 24 / February / 2013, 01:27:26 »
http://chdk.setepontos.com/index.php?topic=9496.0

I'm trying wait_click instead of is_pressed in my script wait loop in case the script error is coming from is_pressed. I've had to modify wait_click a little for short timeouts, as described in the above link.

I tried a 4 second interval tonight, which generated the "set" error before. No error this time. I ended up with an interesting video and a few shots of the International Space Station flying by (at 57 seconds):

http://www.youtube.com/watch?v=Jel8iw99ikM#ws
Logged
EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos
*

Offline lapser

  • *****
  • 1093
Re: Shot Histogram Request
« Reply #118 on: 26 / February / 2013, 09:16:30 »
Phil helped me fix an error in the log10 function address in the SX50 port, so I was able to get my first time lapses with the new camera. The 50X optical zoom is fantastic! Here's a short time lapse of the full moon last night:

Moon Time Lapse 130225 Canon SX50 CHDK

And here's my first sunset time lapse with the SX50:

Skinner Butte 130225 Canon SX50 CHDK

And here's the same sunset with the G1X:

Skinner Butte 130225 Canon G1X CHDK
Logged
EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos
*

Offline lapser

  • *****
  • 1093
Re: Shot Histogram Request
« Reply #119 on: 04 / March / 2013, 16:05:59 »
I switched to the new wait_click(10) in my loop waiting for get_shot_ready() instead of using 3 is_press() calls with "set", "menu", and "display". I was getting script interrupted errors once in awhile, but did not have one with the new wait_click until last night. Instead of a "set" error, I got a "NULL" error with the sx260.

I'll try calling wait_click(10) only between shots, and not repeatedly in a tight loop. I think the error is coming from the call to get_clicked_key() in wait_click(), possibly from a conflict with calls from a different task accessing the same function and keyboard variables. Calling it only once per shot is a small script change, and we can see if it still gives interrupted errors.

I hiked Spencer Butte yesterday to get an extra sunset test using all 3 cameras. The SX260 was on my worst mini-tripod, so it blew over in the wind. I switched to my even worse imitation gorillapod, with a lower center of gravity, and it stayed upright long enough to throw the error. The video turned out pretty interesting.

Here's the last 6 time lapse tests with all 3 cameras. Descriptions are in the YouTube info sections.

http://www.youtube.com/watch?v=lWSAgGiC0Fc#ws

http://www.youtube.com/watch?v=FR6rWXkivqM#ws

http://www.youtube.com/watch?v=DqoWVgAZRZ8#ws

http://www.youtube.com/watch?v=ALWIlL9l0YI#ws

http://www.youtube.com/watch?v=OuZdgg1owGc#ws

http://www.youtube.com/watch?v=8W9z6BnbOx4#ws
Logged
EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos
Pages: 1 ... 7 8 9 10 11 [12] 13 14 15 16 17 ... 47   Go Up
« previous next »
 

Related Topics


SimplePortal © 2008-2014, SimplePortal