Shot Histogram Request

philmoz

3450

Re: Shot Histogram Request

« Reply #10 on: 23 / November / 2012, 23:05:44 »

Advertisements

Quote from: lapser on 23 / November / 2012, 22:58:33

Quote from: waterwingz on 23 / November / 2012, 22:21:07
Quote from: lapser on 23 / November / 2012, 22:09:09
OK, I've been avoiding getting into the C programming side of CHDK, but I guess it's time to cross that bridge.
Its a pretty small group of developers from CHDK beginning to current state when you think about how many Canon P&S cameras are out there in people's purses, pockets and desk drawers and even about how many people have found some pleasure / amusement from using CHDK.

I think philmoz was suggesting all help is appreciated .. although debate about coding style, quality, value of features and functions, testing and documentation come with the territory.
OK, I figured out how to compile with the windows shell. Can you explain the patch and diff files? If I just change the source files, how do I create the patch file to submit for inclusion in CHDK? If you have a link, that would be great, too.

If you're using Windows then TortoiseSVN is the easiest thing to use.

Create a local repository from the CHDK SVN server.
Install TortoiseSVN then right click somewhere in Windows Explorer and select 'Repo borwser'.
Enter https://tools.assembla.com/svn/chdk as the URL.
When it finishes loading right click on the 'trunk' folder and select 'Checkout' - enter a local directory to store your copy.

After you've made your changes, then you can create the patch file using TortoiseSVN.

I also find WinMerge a very useful tool for comparing source trees.

Phil.

Logged

CHDK ports:
sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
g12 (1.00c, 1.00e, 1.00f & 1.00g)
sx130is (1.01d & 1.01f)
ixus310hs (1.00a & 1.01a)
sx40hs (1.00d, 1.00g & 1.00i)
g1x (1.00e, 1.00f & 1.00g)
g5x (1.00c, 1.01a, 1.01b)
g7x2 (1.01a, 1.01b, 1.10b)

waterwingz

12139

Re: Shot Histogram Request

« Reply #11 on: 24 / November / 2012, 00:38:45 »

Quote from: philmoz on 23 / November / 2012, 23:05:44

If you're using Windows then TortoiseSVN is the easiest thing to use.

Create a local repository from the CHDK SVN server.
Install TortoiseSVN then right click somewhere in Windows Explorer and select 'Repo borwser'.
Enter https://tools.assembla.com/svn/chdk as the URL.
When it finishes loading right click on the 'trunk' folder and select 'Checkout' - enter a local directory to store your copy.

You can also get the 1.1.0 branch using a similiar process.

It took me a while but I figured out how to make a batch file to build the directories downloaded from the svn using the CHDK-Shell tools. I never did figure out a way to use CHDK-Shell and Tortoise SVN together - CHDK-Shell likes to do things its own way. CHDK-Shell is a great tool for building the latest version of CHDK - don't get me wrong - but for development that you want to submit patch files back to the svn for - Tortoise SVN and a batch build file will cover that better.

Logged

Ported : A1200 SD940 G10 Powershot N G16

lapser

1093

Re: Shot Histogram Request

« Reply #12 on: 24 / November / 2012, 01:36:55 »

Thanks for the info. I noticed that CHDK shell has a button for "source code tools" which opens a window with diff and patch options. Do you have any information on what that does?

Quote from: philmoz on 23 / November / 2012, 18:47:13

The shot histogram is calculated from the RAW sensor data (scaled to 10 bits), and ignores the RGB BAYER pattern of the sensor.

The average value of this data on its own is not a measure of brightness or exposure. The sensor data is linear, you would first need to demosaic the BAYER sensor data then apply a tone curve, and then you would have to factor in the shutter speed, aperture and ISO used to take the image.

I'm trying to use the shot_histogram data to adjust the exposure when taking multiple shots with "shoot_full_only." I know the eV of the first picture, and want to know the change in ev compared to the first image. I'm hoping to be able to compute that from the histogram.

Now that I can get accurate values for all 1024 histogram points, I can experiment in Lua to see how that relates to eV. I'm thinking I can choose an area, say the upper left, that contains only sky. As the sun goes down, the sky gets darker and the average of the histogram data decreases. I'll try to relate that to eV. I'm not trying to set camera exposure from scratch, just sense slow lighting changes and slowly change exposure to compensate.

I noticed that the histogram ignores RGB and steps by 31. I assume it's sampling R, G, and B pixels alternately that way? Wouldn't averaging a large enough area be related to luminosity, especially if it was clear sky?
==
Again, thanks for all the help.

Logged

EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos

philmoz

3450

Re: Shot Histogram Request

« Reply #13 on: 24 / November / 2012, 01:51:23 »

Quote from: lapser on 24 / November / 2012, 01:36:55

Thanks for the info. I noticed that CHDK shell has a button for "source code tools" which opens a window with diff and patch options. Do you have any information on what that does?

Probably creates and applies patches - haven't used it so I can't say for sure.
If you make some changes then run the diff tool see if it generates a .patch or a .diff file with the changes.

Quote

Quote from: philmoz on 23 / November / 2012, 18:47:13
The shot histogram is calculated from the RAW sensor data (scaled to 10 bits), and ignores the RGB BAYER pattern of the sensor.

The average value of this data on its own is not a measure of brightness or exposure. The sensor data is linear, you would first need to demosaic the BAYER sensor data then apply a tone curve, and then you would have to factor in the shutter speed, aperture and ISO used to take the image.
I'm trying to use the shot_histogram data to adjust the exposure when taking multiple shots with "shoot_full_only." I know the eV of the first picture, and want to know the change in ev compared to the first image. I'm hoping to be able to compute that from the histogram.

Now that I can get accurate values for all 1024 histogram points, I can experiment in Lua to see how that relates to eV. I'm thinking I can choose an area, say the upper left, that contains only sky. As the sun goes down, the sky gets darker and the average of the histogram data decreases. I'll try to relate that to eV. I'm not trying to set camera exposure from scratch, just sense slow lighting changes and slowly change exposure to compensate.

I noticed that the histogram ignores RGB and steps by 31. I assume it's sampling R, G, and B pixels alternately that way? Wouldn't averaging a large enough area be related to luminosity, especially if it was clear sky?

Have a look at the 'get_bv96' Lua function - it may give you what you need.

Phil.

Logged

CHDK ports:
sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
g12 (1.00c, 1.00e, 1.00f & 1.00g)
sx130is (1.01d & 1.01f)
ixus310hs (1.00a & 1.01a)
sx40hs (1.00d, 1.00g & 1.00i)
g1x (1.00e, 1.00f & 1.00g)
g5x (1.00c, 1.01a, 1.01b)
g7x2 (1.01a, 1.01b, 1.10b)

msl

1280
A720 IS, SX220 HS 1.01a

Re: Shot Histogram Request

« Reply #14 on: 24 / November / 2012, 05:02:38 »

Quote from: waterwingz on 24 / November / 2012, 00:38:45

I never did figure out a way to use CHDK-Shell and Tortoise SVN together - CHDK-Shell likes to do things its own way.

It's not a big deal. You need only observe the subdir structures of the CHDK shell.

The shell create 2 subdirs for the CHDK source code: 'branches' & 'trunk'. In 'trunk' dir you find dirs like 'trunk2315'. This is the source code folder of trunk with revision 2315. The same in 'branches'. There begins the dir name with the branch name, e.g. 'philmoz-uitest2283'.

You can create new folders in 'trunk' & 'branches' for tortoise, e.g. 'trunk_work' in trunk dir or 'philmoz-uitest_work' in branches dir. In this folder you can check out a working copy of svn source code.

In the CHDK shell you choose the working copy with the button 'Change'.

Note: The CHDK shell modified some files of the source code, e.g. version.inc. You can reset this changes with an option of the CHDK shell context menu 'Reset Build Options ...'. This is important for the creation of a clean patch file.

Quote from: lapser on 24 / November / 2012, 01:36:55

I noticed that CHDK shell has a button for "source code tools" which opens a window with diff and patch options. Do you have any information on what that does?

In the shell you can see a big number of the revision. Click left beside this number and you can create a copy of the current source code. This is your working copy. In 'source code tools' you can make a patch file with the option 'diff'. You can choose a reference. With the option 'patch' you can insert patch files in the current source code.

I think, using tortoise is the better and more comfortable way.

msl

Logged

CHDK-DE: CHDK-DE links

waterwingz

12139

Re: Shot Histogram Request

« Reply #15 on: 24 / November / 2012, 08:56:17 »

Quote from: msl on 24 / November / 2012, 05:02:38

Quote from: waterwingz on 24 / November / 2012, 00:38:45
I never did figure out a way to use CHDK-Shell and Tortoise SVN together - CHDK-Shell likes to do things its own way.
It's not a big deal. You need only observe the subdir structures of the CHDK shell.

If you use the "DOS box" icon in CHDK-Shell, the banner text tells you what to type in to build a particular camera & firmware version. Using that, you can change to wherever your SVN trunk is locate and compile.

Quote from: msl on 24 / November / 2012, 05:02:38

Quote from: lapser on 24 / November / 2012, 01:36:55
I noticed that CHDK shell has a button for "source code tools" which opens a window with diff and patch options. Do you have any information on what that does?
In the shell you can see a big number of the revision. Click left beside this number and you can create a copy of the current source code. This is your working copy. In 'source code tools' you can make a patch file with the option 'diff'. You can choose a reference. With the option 'patch' you can insert patch files in the current source code.

The diff function works but using Tortoise SVN allows you to create patch files that are 100% usable by the devs. I only played with diff for a bit but was never able to really get the same compatibility.

Logged

Ported : A1200 SD940 G10 Powershot N G16

msl

1280
A720 IS, SX220 HS 1.01a

Re: Shot Histogram Request

« Reply #16 on: 24 / November / 2012, 09:21:07 »

Quote from: waterwingz on 24 / November / 2012, 08:56:17

If you use the "DOS box" icon in CHDK-Shell, the banner text tells you what to type in to build a particular camera & firmware version. Using that, you can change to wherever your SVN trunk is locate and compile

That's another way but not my way. I'm not the command line junkie.

The CHDK shell is really a great tool. But you're right the diff tool is not 100% useable.

msl

Logged

CHDK-DE: CHDK-DE links

lapser

1093

Re: Shot Histogram Request

« Reply #17 on: 24 / November / 2012, 12:41:21 »

static int luaCB_shot_histo_enable( lua_State* L )
{
shot_histogram_set(luaL_checknumber( L, 1 ));
return 0;
}

I want to change shot_histo_enable(1) to shot_histo_enable(1,x1,y1,x2,y2) where the new parameters are optional. How do I tell if the number of parameters is 1 or 5?

It looks like the total number of pixel samples can be returned by shot_histogram_set, and this value returned by Lua without breaking any old scripts. Is this correct?

lua_pushnumber( L, total )

Logged

EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos

lapser

1093

Re: Shot Histogram Request

« Reply #18 on: 24 / November / 2012, 13:11:07 »

Quote from: philmoz on 24 / November / 2012, 01:51:23

Have a look at the 'get_bv96' Lua function - it may give you what you need.

get_bv96() will give me a valid value for the first shot, but the rest of the shots are done with "shoot_full_only" without releasing "half_shoot". If I also compute the average of the histogram data for the first shot and save it, I have an eV reference point. If the average pixel value of subsequent shots goes above this reference, I know I'm over exposed, and can reduce exposure in the next shot.

My test script saves tv0 from get_tv96() after a half_shoot. It then takes a series of exposure bracketed shots from -2 ev to +2 ev in 1/3 ev increments, and prints the shot histogram for each shot (both truncated and full resolution), so you can see how the shot_histogram changes with each 1/3 ev step. I can use these test shots to calibrate how the histogram brightness average translates to eV changes. I'm not sure how well it will work, but it's a stab at adjusting exposure without the overhead of measuring it before each shot in a time lapse.

The log file from my test script is here:
http://chdk.setepontos.com/index.php?topic=8997.msg93658#msg93658

« Last Edit: 24 / November / 2012, 14:04:55 by lapser »

Logged

EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos

lapser

1093

Re: Shot Histogram Request

« Reply #19 on: 24 / November / 2012, 14:03:29 »

int newshot=0;

void build_shot_histogram()
{
newshot=1;
if(!shot_histogram_isenabled())
return;
. . .
}

int shot_ready()
{
ns=newshot;
newshot=0;
return ns;
}

This is my idea to tell when a new picture is ready when using "shoot_full_only". Currently, I'm using this method:

nxp=get_exp_count()
press("shoot_full_only")
while(nxp==get_exp_count)do sleep(10) end

With the shot_ready() function above, I could change the last line to:

repeat sleep(10) until shot_ready()

Do you know if build_shot_histogram() is called right after each picture, regardless of whether or not shot_histo is enabled? Thanks.

« Last Edit: 24 / November / 2012, 15:06:23 by lapser »

Logged

EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos