Shot Histogram Request

lapser

1093

set_shot_interval(time)

« Reply #50 on: 13 / December / 2012, 11:19:35 »

Advertisements

I think I figured out the best way to implement a delay for continuous mode so it will be useful for time lapses, or other uses. I'll add a function called
set_shot_interval(time)

The interval would be the same as the interval in all time lapse scripts. A 2 second interval means delay the next shot until 2 seconds after the last shot. Writing a time lapse script would be simpler, since you wouldn't need to worry about a timer. It would go something like this:

Code: (lua) [Select]

interval=2000 -- for 2 seconds
set_shot_interval(interval)
press("shoot_full") -- in continuous mode
repeat
  nshot=get_shot_number()
  repeat sleep(10) until get_shot_number~=nshot
  -- set new exposure levels here using get_shot_meter()
until false

When the script is finished, CHDK would call set_shot_interval(0), like it does for shot_histogram. This would prevent a script from disabling the entire camera, and reset everything back to normal if you interrupt a script. The above script would work this way.

It would be nice to be able to put the camera in continuous mode with a script. get_drive_mode()==1 tells me I'm in continuous mode. But there doesn't appear to be a set_drive_mode(1). Does anyone know how to do that?

The interval delay should happen right before triggering a new shot (not in raw_save_file(). It would be good if the timing accuracy could be better than the 10 or 20 msec you get from get_tick_count(). I'll see if I can figure out where to put the new code, and how to do the timing from looking through the source files. If someone wants to help point me in the right direction, that would be nice. Anyone? . . . Anyone? . . . Philmoz?

Logged

EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos

philmoz

3450

Re: Shot Histogram Request

« Reply #51 on: 13 / December / 2012, 14:12:55 »

Quote from: lapser on 13 / December / 2012, 11:19:35

The interval delay should happen right before triggering a new shot (not in raw_save_file(). It would be good if the timing accuracy could be better than the 10 or 20 msec you get from get_tick_count(). I'll see if I can figure out where to put the new code, and how to do the timing from looking through the source files. If someone wants to help point me in the right direction, that would be nice. Anyone? . . . Anyone? . . . Philmoz?

The only place I can think of that might work is in wait_until_remote_button_is_released.
However this isn't guaranteed to be called correctly (or at all) by all cameras - depends on the port. Most seem ok though.

10 msec is the best accuracy you can get - there aren't any higher resolution timers that I know and that's the time interval for task switching.

I need to think about this more; but a couple of quick suggestions.
- the relevant code should only be active when a script is running (and probably in continuous mode), alternatively you could add config items and menu UI items to allow it to be used outside a script. Not sure how useful it would be though.
- if it's only used by scripts then the value should be reset by the script engine when the script ends. This will prevent one script from affecting another if the first script forgets to clear the value.

Phil.

Logged

CHDK ports:
sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
g12 (1.00c, 1.00e, 1.00f & 1.00g)
sx130is (1.01d & 1.01f)
ixus310hs (1.00a & 1.01a)
sx40hs (1.00d, 1.00g & 1.00i)
g1x (1.00e, 1.00f & 1.00g)
g5x (1.00c, 1.01a, 1.01b)
g7x2 (1.01a, 1.01b, 1.10b)

lapser

1093

Re: Shot Histogram Request

« Reply #52 on: 13 / December / 2012, 15:03:44 »

Quote from: philmoz on 13 / December / 2012, 14:12:55

10 msec is the best accuracy you can get - there aren't any higher resolution timers that I know and that's the time interval for task switching.

Thanks Phil. I found something that looks interesting:
extern volatile long shutter_open_tick_count; // defined in platform/generic/capt_seq.c

I just tested it, and it does indeed contain a value that appears to be the time the shutter opened.

However, I can probably get away with simply delaying in raw_save_file based on the the time of the last call to raw_save_file, and not bother with anything else. That's probably the best solution, and plenty accurate. The per shot time should remain accurate, which is the main thing. Correcting for the shutter open time probably won't be necessary, although I'll have to think about that a little more.

As for the accuracy issue, take a look at line #321 in main.c. It calculates a value for:
camera_info.tick_count_offset
Can you explain what that is? It's used in dng.c at lines #567 and #576

And on a related issue, I thought I read that "long" was just a 32 bit "int" in CHDK. Is this correct? I hope not.

Thanks again.

Logged

EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos

philmoz

3450

Re: Shot Histogram Request

« Reply #53 on: 13 / December / 2012, 16:30:08 »

Quote from: lapser on 13 / December / 2012, 15:03:44

Quote from: philmoz on 13 / December / 2012, 14:12:55
10 msec is the best accuracy you can get - there aren't any higher resolution timers that I know and that's the time interval for task switching.
Thanks Phil. I found something that looks interesting:
extern volatile long shutter_open_tick_count; // defined in platform/generic/capt_seq.c

I just tested it, and it does indeed contain a value that appears to be the time the shutter opened.

However, I can probably get away with simply delaying in raw_save_file based on the the time of the last call to raw_save_file, and not bother with anything else. That's probably the best solution, and plenty accurate. The per shot time should remain accurate, which is the main thing. Correcting for the shutter open time probably won't be necessary, although I'll have to think about that a little more.

As for the accuracy issue, take a look at line #321 in main.c. It calculates a value for:
camera_info.tick_count_offset
Can you explain what that is? It's used in dng.c at lines #567 and #576

And on a related issue, I thought I read that "long" was just a 32 bit "int" in CHDK. Is this correct? I hope not.

Thanks again.

The camera_info.tick_count_offset value is used to set the msec part of the image capture time in the DNG header. It's only accurate to the same 10msec range as everything else.

The size of long and int are both 32 bits.

Phil.

Logged

CHDK ports:
sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
g12 (1.00c, 1.00e, 1.00f & 1.00g)
sx130is (1.01d & 1.01f)
ixus310hs (1.00a & 1.01a)
sx40hs (1.00d, 1.00g & 1.00i)
g1x (1.00e, 1.00f & 1.00g)
g5x (1.00c, 1.01a, 1.01b)
g7x2 (1.01a, 1.01b, 1.10b)

lapser

1093

Re: Shot Histogram Request

« Reply #54 on: 13 / December / 2012, 17:23:29 »

Quote from: philmoz on 13 / December / 2012, 16:30:08

The camera_info.tick_count_offset value is used to set the msec part of the image capture time in the DNG header. It's only accurate to the same 10msec range as everything else.

The size of long and int are both 32 bits.

Thanks! I like to explore all the options, and pick the simplest one. I appreciate your patience in answering so many questions.

OK, one last question. What function should I use for the "sleep" delay in raw.c?

I think that msleep(long t) is also used by the keyboard task, and thus by Lua. Is it safe to use in a different thread at the same time?

I noticed that capt_seq.c uses _SleepTask(10). Should I use this one?

So long IS 32 bits! That's important to know, thanks.

[EDIT]Here are the changes to raw.c . . does this look right?
[EDIT2]OK, it's working in continuous mode shooting at 2.5 seconds per shot (normally it's 0.5 seconds per shot). There's a picture showing on the screen at all times, which can be turned off with set_backlight.

Code: (c) [Select]

...in raw_save_file()
if (state_kbd_script_run)
{
  if(shot_histogram_isenabled())build_shot_histogram();
  while((shot_interval>0)&&(get_tick_count()<shot_next))msleep(10); //set_shot_interval(0) aborts
  shot_next=get_tick_count()+shot_interval;
}
shot_number++; //polled by Lua get_shot_count()
...
int raw_get_shot_number() // called from Lua - increments after shot histo is ready
{
  return shot_number; //incremented in raw_save_file() ... different thread
}

void raw_set_shot_interval(int interval) //called from Lua ... sets minimum interval
{
  shot_interval=interval-5; //subtract 5 for rounding to 10msec sleep time
}
...
static int shot_number=0; //incremented after build_shot_histogram()
static int shot_interval=0; //msec .. minimum time between shots .. works in continuous mode
static int shot_next=0; //minimum tick time for next shot . . delay until this time

« Last Edit: 13 / December / 2012, 19:24:20 by lapser »

Logged

EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos

lapser

1093

First Time Lapse Video with new continuous mode delay function

« Reply #55 on: 13 / December / 2012, 23:14:15 »

The delay function above worked well. It held the time between shots in continuous mode to 2500 msec, until the shutter time exceeded that. Even then, there is a minimum delay between shots compared to single shot mode and repeated shoot_full_only presses. It looks like msleep works fine in more than one thread, so it must be thread safe.

The screen display of the pictures during shooting worked too. It was nice to see exactly what the pictures looked like as they were being taken. My back light toggle also worked, since the keyboard thread is active while the delay takes place in the camera thread. This appears to be a superior method for doing time lapses.

http://www.youtube.com/watch?v=X5Zkw_G7KHs#ws

First shot with metering areas drawn on image.
Log file attached

House_1212131617.LOG (140.92 kB - downloaded 64 times.)

Logged

EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos

lapser

1093

Re: Shot Histogram Request

« Reply #56 on: 14 / December / 2012, 13:37:59 »

I noticed a few things from looking at the pictures and the log file above. First, the exposure didn't change on the second shot. The reason is that the 2nd shot happened immediately, with no delay. The Lua script didn't calculate and store the adjusted tv value until the 2nd shot was already in progress. This is probably also happening on the shots where the shutter time exceeded the interval time, and the delay goes to 0. The exposure changes were so small, it's not possible to tell, but I'm pretty sure it was happening all the time. That means I would be 2 shots behind in setting exposure instead of one.

One reason is that the shot_number was being incremented AFTER the delay, instead of between the call to build_histogram and the start of the delay loop. shot_number++ signals the Lua script that the shot_meters are valid, so it can set the new TV values. But at this high shooting speed, it's a race between the camera and the script, and the script was losing. With the shot_number++ set up this way, the problem was probably happening after every shot.

I'll start by adding a minimum 20 msec delay AFTER shot_number++, and before returning and letting the camera shoot the next picture. If that's not enough time, I can add a longer delay that can be aborted by the Lua script after it sets the new TV values.
=========
I also figured out what's happening when going from light to dark areas rapidly, like taking a picture of a bright monitor at night, and then at a dark area of the room. Going from bright picture to dark picture is no problem. The shot_meter correctly estimates the exposure change needed. The first shot is dark, and the next one is exposed properly. With a really dark area, there might be a 3rd shot needed to zero in on the correct exposure, but usually not.

Going from dark to bright is a different story. The first shot of the computer monitor taken with the dark exposure is totally white, all pixels blown, as expected. To the shot meter, totally white is only 1 f/stop above the 50% exposure level. I return 2 f/stops below that, so the maximum correction the meter will make downward from a totally blown white shot is 3 f/stops. It works, but it takes 3 or 4 pictures, 3 f/stops less each picture, to get the right exposure. It's interesting to watch the details appear out of white picture a little more each shot, but it's not the desired result.

I need to add an extra correction based on how far above 50% the linear pixel average is. A completely blown picture might need 7 or 8 f/stops less exposure, instead of the 3 maximum. I can probably design an experimental lua script that collect this data systematically, and computes an adjustment factor for the various over exposure levels. I guess I'll have to take a few thousand more pictures of my blank wall! Fun.

Logged

EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos

jmac698

138

Re: Shot Histogram Request

« Reply #57 on: 15 / December / 2012, 23:06:48 »

Hi,
I'm interested in this for doing auto HDR, but at extreme levels. Taking shots at night for example, I have the problem that an auto exposure sits at the maximum which returns complete black. So if you have black or white, just think that a normal picture has say 8 stops exposure and you need to vastly change the settings. My night exposure needed 6 or 8 fstops more, depending on camera. The most efficient way is to do a binary search pattern. Also I think a blank wall is an unrealistically low dynamic range to test with. I've also graphed a sunrise with a light meter, and it obviously goes over a huge range very quickly.

So ultimately, I'd like the camera to do what I do manually, take test shots until I find a good range to start shooting, and to know when to stop for extreme HDR.

Also to guess where you're at, you can guess at a normal histogram and compare, to see how much you should change.. would have to work out those ideas, it could be simple or complex, and in fact is probably what autoexposure does anyhow....

« Last Edit: 15 / December / 2012, 23:08:52 by jmac698 »

Logged

waterwingz

12139

Re: Shot Histogram Request

« Reply #58 on: 15 / December / 2012, 23:39:35 »

Quote from: lapser on 14 / December / 2012, 13:37:59

I guess I'll have to take a few thousand more pictures of my blank wall!

Quote from: jmac698 on 15 / December / 2012, 23:06:48

Also I think a blank wall is an unrealistically low dynamic range to test with.

You probably need to spend several hundred hours of your time working on CHDK development to understand what lapser meant. Reyalp's keyboard, the oak tree outside my window and a lone koala bear might be the most photographed things in history.

« Last Edit: 15 / December / 2012, 23:42:09 by waterwingz »

Logged

Ported : A1200 SD940 G10 Powershot N G16

lapser

1093

Re: Shot Histogram Request

« Reply #59 on: 16 / December / 2012, 00:31:51 »

Quote from: jmac698 on 15 / December / 2012, 23:06:48

Also I think a blank wall is an unrealistically low dynamic range to test with. I've also graphed a sunrise with a light meter, and it obviously goes over a huge range very quickly.

Actually, you're correct that a blank wall wouldn't work for the problem I stated, which is, too many pixels blown out to get an accurate exposure correction for the next shot. The solution isn't all that complicated, though, as you suggested. When you have a lot of blown pixels, give up on getting the right exposure in the next shot. Just decrease by 6 or 8 EV instead of 3. Once you find a spot where the pixels are in the linear range, not too many totally dark or blown pixels, you can hit the right exposure on the next shot. I got to it eventually (in 3 to 5 shots) with 3 EV steps.

I've got the shot meter working perfectly (thanks to the blank wall). The histogram analysis shouldn't be that hard, either. Instead of using the average of all the pixels like in the shot meter, I'll add up the number pixels from the top of the histogram down, until I get to 1% of the total pixels (or some other percentage). The exposure correction is the desired level minus the 1% level (in log units). There's no binary search involved. If the pixels are in range, you hit the exposure you want on the next shot. It works for the shot meter, and hopefully it will work for the histogram.

The blank wall was ideal for calibrating the shot meter part. The histogram part will require a wide dynamic range so there will be dark and bright pixels represented. I was kind of kidding about the blank wall thing, but I'm almost serious now when I say that I may need a few thousand pictures of a black and white wall now.

Logged

EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos