Script sleep oddity

lapser

1093

Re: Script sleep oddity

« Reply #50 on: 24 / January / 2013, 19:02:44 »

Advertisements

Quote from: philmoz on 24 / January / 2013, 14:10:54

That tech note is for HQ burst mode.
The user manual has the same information for both the SX260 and G1X.

Yes, this problem only occurs in HQ burst mode (AKA continuous).

Quote

On all my cameras lens correction only appears to be applied at the wide end not when zoomed in.

That's true for the sx260 too. I think I may have an answer though. Zoomed in, the jpg includes the entire raw buffer. Zoomed out, the raw buffer is cropped to cut off the distorted corners. You have to process more raw pixels when zoomed in, which may take more time. The sx260 has more lens distortion, hence more cropping, so the effect would be greater. Also, the burst shot rate for the sx260 is faster than the G1X, which may be because of having less raw data to process in wide angle, and the faster processor. They got their fast shot rate spec for the sx260, but at the expense of messing up when it's zoomed in.

Quote

It could also be a memory problem - how much free memory is there when the script is running?
There's a block of code near the end of core_spytask in an #ifdef DEBUG_PRINT_TO_LCD you can edit to display stuff all the time on screen.

Thanks for the tip. I used:

core_get_free_memory()

Is that what you were thinking of? It's pretty interesting to watch. It changes when scripts load and unload, and when my histogram routine is enabled. But the number always stays well above 2 megabytes (sx260), and doesn't change while shooting and logging in a script when it reaches the slow down point.

By the way, it doesn't slow down when using my set_shot_interval(1000) function to keep the shots 1 second apart, even when zoomed. It seems like the camera chokes if a new shot finishes before the old shots have finished saving, i.e. the save buffer is full. I can experiment around with set_shot_interval to see what the minimum interval is that works without the delays. Next problem...

Even with a 1 second interval, and no delays, the script halted on both cameras with an error after about 3,000 shots. I was also writing a lot of log data with file:write(..). I remember you describing memory use before, and recommending taking a picture before starting the script to prevent fragmentation. Do you have any insight into the cause of this problem. What are file handles? I know I keep asking this. If you have a link to a discussion of this problem, that would be great. Thanks.

Logged

EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos

lapser

1093

Re: Script sleep oddity

« Reply #51 on: 25 / January / 2013, 18:43:09 »

I've been doing more testing and found out some interesting things. But first:

I think I have a different perspective on this shoot() bug thing. My feeling is that we need to find the cause of it, and a temporary work around that hangs re-shooting would be best to help us find the cause, if it happens. Re-shooting is not the answer, nor is an error return. We don't know the answer.

I thought of one more possibility. This bug only seems to be triggered by AS_SHOOT, and not by the equivalent press("shoot_half") etc. done in a script. So what's the difference between the two? Well, one thing is that there's no delay in AS_SHOOT between get_shooting() becoming true and press("shoot_full"). It's possible that get_shooting() could switch to true, but the camera isn't quite ready to shoot instantly. If the AS_SHOOT presses shoot_full during this time, it just might cause the problem. It wouldn't happen very often, because the timing between threads would have to sync up just right, but that's what we're seeing.

Anyway, it's a theory, and an easy one to test. Just add a short delay before pressing shoot_full in AS_SHOOT. I'll give it a try.
====
While looking into the set_sv96(sv) function, I discovered something strange:
while ((shooting_is_flash_ready()!=1) || (focus_busy));
I'm inclined to just delete this line. Does it have any purpose? Should it at least be changed to this?
while ((shooting_is_flash_ready()!=1) || (focus_busy))msleep(10);

The only other place I found focus_busy is in wrappers.c
_MoveFocusLensToDistance((short*)&newpos);
while ((shooting_is_flash_ready()!=1) || (focus_busy)) msleep(10);

While experimenting with focus_busy, I discovered another interesting thing. Setting the focus override on my sx260 to infinity does nothing in auto-focus mode. In manual mode, it crashes the camera on the next half_shoot.

However, in manual focus mode, but with safety manual focus ON, it focuses at infinity! It sounds like manual focus mode turns off the focus motor completely, so if you try to focus... crash. Safety manual focus leaves it on. I need to test it with set_focus() in half_shoot, which crashes the camera in manual focus without safety MF. Focus bracketing in continuous works, but it looks like the "safety" part is messing up the bracketing.

Logged

EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos

philmoz

3450

Re: Script sleep oddity

« Reply #52 on: 26 / January / 2013, 17:13:00 »

Quote from: lapser link=topic=7611.msg96298#msg96298

My feeling is that we need to find the cause of it, and a temporary work around that hangs re-shooting would be best to help us find the cause, if it happens. Re-shooting is not the answer, nor is an error return. We don't know the answer.

What makes you believe there is only one possible cause across all cameras and firmware versions and the almost infinite ways people may find to use CHDK?

Quote

This bug only seems to be triggered by AS_SHOOT, and not by the equivalent press("shoot_half") etc. done in a script. So what's the difference between the two? Well, one thing is that there's no delay in AS_SHOOT between get_shooting() becoming true and press("shoot_full"). It's possible that get_shooting() could switch to true, but the camera isn't quite ready to shoot instantly. If the AS_SHOOT presses shoot_full during this time, it just might cause the problem. It wouldn't happen very often, because the timing between threads would have to sync up just right, but that's what we're seeing.

In the current release and trunk versions there is at least 20-30msec between them due to the action stack only processing one operation per cycle. You are also missing the other obvious difference - the call to wait for the flash to be ready.

Quote

====
While looking into the set_sv96(sv) function, I discovered something strange:
while ((shooting_is_flash_ready()!=1) || (focus_busy));
I'm inclined to just delete this line. Does it have any purpose? Should it at least be changed to this?
while ((shooting_is_flash_ready()!=1) || (focus_busy))msleep(10);

The only other place I found focus_busy is in wrappers.c
_MoveFocusLensToDistance((short*)&newpos);
while ((shooting_is_flash_ready()!=1) || (focus_busy)) msleep(10);

While the line may not be necessary and should probably have a sleep it's purpose is lost in undocumented history. It's not safe to delete things like this without extensive testing especially on old cameras where it probably originated.

Quote

While experimenting with focus_busy, I discovered another interesting thing. Setting the focus override on my sx260 to infinity does nothing in auto-focus mode. In manual mode, it crashes the camera on the next half_shoot.

However, in manual focus mode, but with safety manual focus ON, it focuses at infinity! It sounds like manual focus mode turns off the focus motor completely, so if you try to focus... crash. Safety manual focus leaves it on. I need to test it with set_focus() in half_shoot, which crashes the camera in manual focus without safety MF. Focus bracketing in continuous works, but it looks like the "safety" part is messing up the bracketing.

Sounds like a bug in the port.

Phil.

Logged

CHDK ports:
sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
g12 (1.00c, 1.00e, 1.00f & 1.00g)
sx130is (1.01d & 1.01f)
ixus310hs (1.00a & 1.01a)
sx40hs (1.00d, 1.00g & 1.00i)
g1x (1.00e, 1.00f & 1.00g)
g5x (1.00c, 1.01a, 1.01b)
g7x2 (1.01a, 1.01b, 1.10b)

lapser

1093

Re: Script sleep oddity

« Reply #53 on: 26 / January / 2013, 18:26:11 »

Quote from: philmoz on 26 / January / 2013, 17:13:00

What makes you believe there is only one possible cause across all cameras and firmware versions and the almost infinite ways people may find to use CHDK?

Bugs bug me. I'll see if skrytlen will test it for me, and I won't bug you unless I find something that works.

I see you have the error return in the trunk. If you get the time to add the single re-shoot that would be a further improvement. I agree this should be left in as back up, even if we discover a specific cause for a shoot failure on a single camera. Let's agree to agree, and move on.

Quote

While the line may not be necessary and should probably have a sleep it's purpose is lost in undocumented history. It's not safe to delete things like this without extensive testing especially on old cameras where it probably originated.

I agree. I'll leave to you to decide if you want to add the msleep(10). I thought I should point it out.

I'll try to explore the focusing problems when I get the time. Thanks for your help

Logged

EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos

philmoz

3450

Re: Script sleep oddity

« Reply #54 on: 30 / January / 2013, 04:08:06 »

I've updated the philmoz-action-stack branch with some minor changes to the action stack logic and a re-work of the script 'shoot' code.

I noticed while testing the retry logic for 'shoot' that it was possible to hang the shoot action stack code on the AS_WAIT_SHOOTING_IN_PROGRESS step if shooting_in_progress never goes true.

This can happen for a variety of reasons such as:
- on old cameras with 'Play' on the mode dial, a shutter half press won't change to record mode so running a script with a 'shoot' in play mode hangs.
- if the SD card is full it hangs.

So I've added a timeout of 5 seconds to AS_WAIT_SHOOTING_IN_PROGRESS. If it hasn't started shooting in this time the shoot action terminates and returns an error code to the script.

In order to have multiple error values returned I've changed the logic for the shoot return value:
0 = shoot succeeded
1 = shutter half press timed out
2 = shoot failed (even after a retry)

Phil.

Logged

CHDK ports:
sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
g12 (1.00c, 1.00e, 1.00f & 1.00g)
sx130is (1.01d & 1.01f)
ixus310hs (1.00a & 1.01a)
sx40hs (1.00d, 1.00g & 1.00i)
g1x (1.00e, 1.00f & 1.00g)
g5x (1.00c, 1.01a, 1.01b)
g7x2 (1.01a, 1.01b, 1.10b)

lapser

1093

Re: Script sleep oddity

« Reply #55 on: 30 / January / 2013, 12:30:17 »

Wow, it looks great Phil. You're a great programmer, if you don't know that already. Good job.

I didn't realize get_shooting() never goes true when the card fills up. I've only done that in continuous mode, and it kept on shooting pictures without saving them. It took me awhile to figure out what happened. At least there will be a way to know what happened with this code. It should make shoot() useful, finally.

Also, if you want continuous re-shooting, you don't have to check the return value from shoot() if it's in a loop, which it usually is in old scripts that hang.

The 5 second timeout for shooting_in_progress seemed excessive at first, but I think it's about right. When that happens, something is seriously wrong, like a full card, and it's best to delay long enough that it's obvious.

I compiled it for my new SX50, and it worked in a shoot loop (without failing). I'll compile a version for skrylten to test with his focus bracketing script that fails in shoot(). I added a "shoot failed" message so he can see that it happened and that it did a re-shoot.

Logged

EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos

philmoz

3450

Re: Script sleep oddity

« Reply #56 on: 31 / January / 2013, 16:36:48 »

If there are no objections I will move the philmoz-action-stack changes to the main trunk over the weekend.

Phil.

Logged

CHDK ports:
sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
g12 (1.00c, 1.00e, 1.00f & 1.00g)
sx130is (1.01d & 1.01f)
ixus310hs (1.00a & 1.01a)
sx40hs (1.00d, 1.00g & 1.00i)
g1x (1.00e, 1.00f & 1.00g)
g5x (1.00c, 1.01a, 1.01b)
g7x2 (1.01a, 1.01b, 1.10b)

lapser

1093

Re: Script sleep oddity

« Reply #57 on: 31 / January / 2013, 19:10:45 »

Quote from: philmoz on 31 / January / 2013, 16:36:48

If there are no objections I will move the philmoz-action-stack changes to the main trunk over the weekend.

Sounds good. I did another sx50 shoot() loop test with 520 pictures at 2 seconds per shot in manual focus and exposure mode. Everything worked.

Logged

EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos