KAP & UAV Exposure Control Intervalometer Script

waterwingz

12139

Re: KAP & UAV Exposure Control Intervalometer Script

« Reply #1030 on: 27 / January / 2019, 18:49:50 »

Advertisements

Quote from: PaulOski on 27 / January / 2019, 17:32:22

Those previous logs were all tests in the garage primarily making sure the pixhawk would trigger the camera.

I thought that might be the case - the Bv (brightness values) are quite low as might be expected indoors. Important clue there - read on.

Quote

I used your suggestion and tested the intervalometer. The intervalometer scripted running looks good.

Actually, I meant for you to run the kap_uav.lua script in "intervalometer mode" (i.e. with the Shot Interval set for shooting every 10 seconds or so - no pixhawk used).

Quote

I ran KAP again and made numerous adjustments to TV, ISO, Exposure and I'm getting closer.

Looking at your log file, the final script run (with the overexposed images) was done in very bright daylight. And that seems to be causing problems between the camera and script over the use of the ND filter. Strange to stumble on this now with a script that's been downloaded over 5000 times and used mostly outdoors.

Here's what I think I see, using the image you posted (IMG_0889.JPG) for reference.

The camera wants to expose for 1/500 of a second at ISO80. Its fixed aperture lens opening is at f4.0 (determined by the zoom position you are using). From what I can see, the camera also used its ND filter - resulting in an effective f-stop of f11 instead of f4.0
The script takes those values and tried to increase the exposure time towards your 1/1000 second target time. As it does so, it increases the ISO setting to compensate. When the ISO setting reaches the maximum value of 800, it stops increasing the shutter speed - in this case stopping at 1/640 seconds. It then shoot but without using the ND filter.

Something is very wrong here.

the ISO setting goes up by a factor of 10x
the shutter speed only increases by about 10%
the effective aperture opens by 3x due to the script not using the ND filter

No wonder the image is massively over exposed. I need to spend a little more time studying how the ND filter is interacting here.

Maybe the reason we've not seen this before is that AFAIK most Powershot don't use the ND filter very often - in some cases only in scene modes like "fireworks".

Interestingly, your camera reports that it's running in something called HYBRID_AUTO mode, described in the manual for your camera as taking a 2 - 4 second video before each shot?? Do you really have it set to HYBRID_AUTO? Or is this a bug in the camera port resulting in the wrong shooting mode being reported?

So, I'd suggest setting the camera into "P" mode if you have not done so, as very strongly recommended here : KAP_UAV Exposure Control Script : Camera Settings . It won't fix whatever is wrong (if anything) in the script's ND filter logic but P mode might stop the camera from trying to use the ND filter in the first place.

More to come ....

« Last Edit: 27 / January / 2019, 20:30:04 by waterwingz »

Logged

Ported : A1200 SD940 G10 Powershot N G16

reyalp

14082

Re: KAP & UAV Exposure Control Intervalometer Script

« Reply #1031 on: 27 / January / 2019, 19:02:30 »

[quote author=waterwingz link=topic=10822.msg139243#msg139243
Looking at your log file, the final script run (with the overexposed images) was done in very bright daylight. And that seems to be causing problems between the camera and script over the use of the ND filter. Strange to stumble on this now with a script that's been downloaded over 5000 times and used mostly outdoors.
[/quote]
It seems likely that sx610 could suffer the same ND control problem that elph180, since they are both relatively new cameras. Background: https://chdk.setepontos.com/index.php?topic=13228.msg137603#msg137603

Try a build with #define CAM_ND_SET_AV_VALUE 1

Logged

Don't forget what the H stands for.

waterwingz

12139

Re: KAP & UAV Exposure Control Intervalometer Script

« Reply #1032 on: 27 / January / 2019, 19:09:50 »

Quote from: reyalp on 27 / January / 2019, 19:02:30

It seems likely that sx610 could suffer the same ND control problem that elph180, since they are both relatively new cameras.

Thanks - I did consider that the sx610 camera port is new. AFAIK, the bug you mention is the ND filter not doing what the script asks. But in the log file, I'm not sure the script is actually making the request in the first place. It's more like it is failing to realize that the exposure settings (Tv,Sv, NDF position) are being made with the ND filter engaged.

I also scanned this complete thread in print mode and there are way too many issues with the ND filter handling for me to be happy. Many of them resulted in underexposure but there are also overexposure issues reported. It's been "fixed" several times and still here we are.

Not having looked at the script's ND filter logic from exposure measurement to shot settings in a couple of years now, I'll take an hour or two and try to remap what it's doing.

« Last Edit: 27 / January / 2019, 19:17:56 by waterwingz »

Logged

Ported : A1200 SD940 G10 Powershot N G16

PaulOski

12

Re: KAP & UAV Exposure Control Intervalometer Script

« Reply #1033 on: 29 / January / 2019, 11:04:23 »

I've made some progress on the over exposure. I reset both the camera settings and went back the KAP defaults. It's overcast today so I will not be able to test brighter conditions until the sun comes out. My biggest take away was I needed to lock the ISO down to 100 in the script and adjust the TV Min and Target to 1/640 or 1/800. I just took two pictures, P mode normal camera operation and KAP script. The Normal P mode picture auto ISO to 80 and 1/500. The KAP picture came out as ISO 100 and 1/800. A side by side comparison shows the Normal operation still looks better. Any addition thoughts? log and sample pictures can be download at the link below. Thanks again for any feedback.

https://inet.idealab.com/dbd/21CE7446/KAP_log_and_sample_pics_1_29_2018.zip

KAP vs Normal.JPG (180.55 kB, 1911x1009 - viewed 43 times.)

Logged

waterwingz

12139

« Reply #1034 on: 29 / January / 2019, 12:40:29 »

Quote from: PaulOski on 29 / January / 2019, 11:04:23

Any addition thoughts?

As I mentioned previously, the script seems to struggle when the Canon firmware tries to use the ND filter. Your tests are not using brightly lit subjects so the script will work fine as is. I'm looking at whether the script has a bug when the camera uses the ND filler. If you lock the exposures settings to a small fixed range you are essentially defeating that purpose of the script and basically taking manual mode exposures.

You did not answer my question about what mode you have the camera set to? It should be P mode but your log file says that's not what you are using? (It says hybrid auto)

Logged

Ported : A1200 SD940 G10 Powershot N G16

PaulOski

12

Re: KAP & UAV Exposure Control Intervalometer Script

« Reply #1035 on: 29 / January / 2019, 12:58:54 »

Some of my early tests in the garage where in a "Hybrid" mode. The latest items uploaded have the phone in Camera only mode and in "P". 2019Jan29 11:42:08 Mode switched to P

I agree I'm defeating the script purpose by restriction the settings. I welcome any recommendations.

Logged

waterwingz

12139

Re: KAP & UAV Exposure Control Intervalometer Script

« Reply #1036 on: 29 / January / 2019, 15:30:46 »

Quote from: PaulOski on 29 / January / 2019, 12:58:54

Some of my early tests in the garage where in a "Hybrid" mode. The latest items uploaded have the phone in Camera only mode and in "P". 2019Jan29 11:42:08 Mode switched to P

I agree I'm defeating the script purpose by restriction the settings. I welcome any recommendations.

Sorry to keep going back to this but unless I missed a post, your over-exposed shots taken from your UAV were all done using etiher HYBRID_AUTO mode in the first log you posted and LOWLIGHT in the second log.

Your testing since then has been in much less bright conditions. So you have not really tested in bright conditions with the camera in P mode?

The reason I ask is that while I might be able to figure out and correct the ND filter bug (if there is one), I'm guessing that running the script on your UAV in brightly lit conditions might work okay in P mode as most of the shooting I have seen done in that mode does not used the ND filter.

So if you can test and let me know if actual shooting (using the script default exposure settings) actually works well in P mode, that would be interesting to know.

Meanwhile, I'll look at the ND issues some more and see if i can understand what is not working there.

Logged

Ported : A1200 SD940 G10 Powershot N G16

waterwingz

12139

Re: KAP & UAV Exposure Control Intervalometer Script

« Reply #1037 on: 29 / January / 2019, 15:38:27 »

Quote from: PaulOski on 29 / January / 2019, 11:04:23

A side by side comparison shows the Normal operation still looks better. Any addition thoughts? log and sample pictures can be download at the link below.

The biggest difference between the two images seems to be the focus. The log shows that the camera is AFL (auto focus lock) mode so it's possible that it is not focusing at infinity well. CHDK based manual focus & focus at infinity on some Powershots just does not work right, But that is the subject of a different very long thread on this forum.

Try the shot again but disable the Focus @ Infinity infinity setting?

Logged

Ported : A1200 SD940 G10 Powershot N G16

PaulOski

12

Re: KAP & UAV Exposure Control Intervalometer Script

« Reply #1038 on: 30 / January / 2019, 10:40:15 »

Sun was out today so I put the SX610 on an old quad and took a couple pictures. Every picture seemed to be extremely washed out. Focus seemed to be a problem as well. I had better luck setting the Focus to infinity in the script. New log and sample pictures are in the link.

https://inet.idealab.com/dbd/195F5B27/KAP_Log_and_sample_pics_1_30_2019.zip

KAP is in default settings.
Camera is in P mode
ISO Auto
Continuous AF Off
IS Mode Off
Lens retract 0 sec.

Logged

waterwingz

12139

Re: KAP & UAV Exposure Control Intervalometer Script

« Reply #1039 on: 30 / January / 2019, 10:53:26 »

Quote from: PaulOski on 30 / January / 2019, 10:40:15

Sun was out today so I put the SX610 on an old quad and took a couple pictures. Every picture seemed to be extremely washed out. Focus seemed to be a problem as well.

Thanks Paul. Those pictures and the log file help a lot.

Meanwhile, I've very carefully mapped out what I think we are seeing here. I'll post a technical description shortly in a new thread as it's not really script specific and will get buried here. I'm really hoping to get this solved properly this time.

What does not make sense is why you are the first person to stumble across this after thousands of downloads from my file share and many more from the UAV vendors. (Note : that's a very popular comment when debugging any software). But it could be something to do with your camera's port as that is quite new. TBD.

Testing now is tough as I only have one of my camera's with me at the moment (S100) and it has a real aperture so does not handle the ND filter the same way in P mode. In fact I'm pretty sure it never uses it.

If I PM you a link to a test version of the script with some additional log messages and a quick fix, are you able to test for me? I've had a version 4.0 of the script ready for some time but not released it due to lack of test time so we could get that out of the way too. Mostly just logging cleanups and the use of the newer script header style in the new release FWIW.

Quote

I had better luck setting the Focus to infinity in the script. New log and sample pictures are in the link.

You mention using an "old quad". Could this be a vibration issue? We've been down that road here many times but it will be hard to tell from the photos until we get the exposure fixed. The usual first test is to shoot when stable on the ground at a brightly lit scene.

« Last Edit: 30 / January / 2019, 11:04:28 by waterwingz »

Logged

Ported : A1200 SD940 G10 Powershot N G16