suba threading bug (was Re: 3D scanner with multiple IXUS160 cameras) - page 3 - General Discussion and Assistance - CHDK Forum supplierdeeply

suba threading bug (was Re: 3D scanner with multiple IXUS160 cameras)

  • 77 Replies
  • 19052 Views
Re: suba threading bug (was Re: 3D scanner with multiple IXUS160 cameras)
« Reply #20 on: 03 / October / 2017, 05:53:40 »
Advertisements
Did you check "Miscellaneous -> Console -> Display last console" on the failed camera?
I did not.

New test and screenshot of that attached

I wonder if that camera is still running the old build?

I added cameaside function:
Code: [Select]
function cmds.build_stats()
    --e.g. to run !return mc:cmdwait('build_stats')
    local build_info ={}
    build_info = get_buildinfo()
    write_status(serialize(build_info), 'build_details')
end

cli trace of executing that (attached) shows cam14 as build 4900 (rather than 4917) - not sure how I managed that one ... slight hangover?  :-[

*

Offline reyalp

  • ******
  • 13792
Re: suba threading bug (was Re: 3D scanner with multiple IXUS160 cameras)
« Reply #21 on: 03 / October / 2017, 13:00:02 »
New test and screenshot of that attached
Thanks. Error isn't really informative but failing to open a file is likely due to low memory.

Unfortunately, this is going to make testing with philmoz build difficult. The only way I can think of to get back to a usable amount of RAM is the UI memory hack: https://chdk.setepontos.com/index.php?topic=11246.0

I'll try to come up with a stress test for some of the other scenarios, but it may be a while as I'm pretty busy with other stuff at the moment.

I added cameaside function:
Just FWIW, you don't need to add a new camera side function every time you want to do something like this, you can use the "call" command to run any lua:

Quote
!return mc:cmdwait('call return get_buildinfo()')
Don't forget what the H stands for.

Re: suba threading bug (was Re: 3D scanner with multiple IXUS160 cameras)
« Reply #22 on: 03 / October / 2017, 13:53:05 »
Appreciate all of that - if I can report anything useful in the meantime i'll do that.



Fmi: placeholder
https://www.lua.org/pil/8.html
Quote
Previously, we introduced dofile as a kind of primitive operation to run chunks of Lua code. The dofile function is actually an auxiliary function; loadfile does the hard work. Like dofile, loadfile also loads a Lua chunk from a file, but it does not run the chunk. Instead, it only compiles the chunk and returns the compiled chunk as a function
...[]...
The loadstring function is similar to loadfile, except that it reads its chunk from a string, not from a file. For instance, after the code
    f = loadstring("i = i + 1") 
f will be a function that, when invoked, executes i = i + 1:
    i = 0 f(); print(i)   --> 1 f(); print(i)   --> 2 

Code: [Select]
function cmds.call()
        local f,err=loadstring(mc.args)
        if f then
                write_status({f()})
        else
                write_status(false,err)
        end
end

=>

invoking f -via f() - executes  arbitrary code.

If the arbitrary code is a cmds function then a subsequent mc:cmd("functionname paramrers") is required to execute the function
« Last Edit: 03 / October / 2017, 13:54:48 by andrew.stephens.754365 »

*

Offline reyalp

  • ******
  • 13792
Re: suba threading bug (was Re: 3D scanner with multiple IXUS160 cameras)
« Reply #23 on: 03 / October / 2017, 16:44:27 »
If the arbitrary code is a cmds function then a subsequent mc:cmd("functionname paramrers") is required to execute the function
Just to clarify:
Multicam commands are executed with mc:cmd or cmdwait, like
mc:cmdwait('cmd parameters')

'call' is one such command, along with 'rec', 'shoot' and everything else defined in the camera side mc.cmds array. The meaning and format of any parameters is specific to each command.

The 'call' command takes a single parameter, which is a string of arbitrary Lua code. The code is compiled with loadstring(), and if compilation succeeds, executed as a function call, returning any return values in status.

Note 'call' catches compile time errors generated by loadstring (things like unclosed quotes or mismatched parentheses) and returns them gracefully as failed status, but runtime errors (e.g. a typo in a function name like pint instead of print) will cause the entire camera side muticam script to exit.

The 'pcall' command works like 'call' and catches runtime errors, but will itself generate an uncaught runtime error if the code calls CHDK functions that yield (sleep, key press functions, and generally anything that waits)

It's a hack  ;)
Don't forget what the H stands for.


*

Offline philmoz

  • *****
  • 3441
    • Photos
Re: suba threading bug (was Re: 3D scanner with multiple IXUS160 cameras)
« Reply #24 on: 03 / October / 2017, 17:18:36 »
Unfortunately, this is going to make testing with philmoz build difficult. The only way I can think of to get back to a usable amount of RAM is the UI memory hack: https://chdk.setepontos.com/index.php?topic=11246.0


Is it possible to test for the original issue just on the cameras that did load ok, to see if either the EXMEM_TESTING or 'suba' checks detect anything? Or is the free memory so low that even this won't work?


Another option would be to change the EXMEM code to allow some of the memory to be used by CHDK and only run the EMMEM_TESTING at the end of the EXMEM block (which is most likely to be corrupted).


Phil.
CHDK ports:
  sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
  g12 (1.00c, 1.00e, 1.00f & 1.00g)
  sx130is (1.01d & 1.01f)
  ixus310hs (1.00a & 1.01a)
  sx40hs (1.00d, 1.00g & 1.00i)
  g1x (1.00e, 1.00f & 1.00g)
  g5x (1.00c, 1.01a, 1.01b)
  g7x2 (1.01a, 1.01b, 1.10b)

*

Offline reyalp

  • ******
  • 13792
Re: suba threading bug (was Re: 3D scanner with multiple IXUS160 cameras)
« Reply #25 on: 03 / October / 2017, 17:47:01 »
Is it possible to test for the original issue just on the cameras that did load ok, to see if either the EXMEM_TESTING or 'suba' checks detect anything? Or is the free memory so low that even this won't work?
I'd expect a lot of random failures over time with so little memory, but it might be worth a try.

Quote
Another option would be to change the EXMEM code to allow some of the memory to be used by CHDK and only run the EMMEM_TESTING at the end of the EXMEM block (which is most likely to be corrupted).
That would probably make it a lot easier.
Don't forget what the H stands for.

*

Offline philmoz

  • *****
  • 3441
    • Photos
Re: suba threading bug (was Re: 3D scanner with multiple IXUS160 cameras)
« Reply #26 on: 03 / October / 2017, 21:13:27 »
Quote
Another option would be to change the EXMEM code to allow some of the memory to be used by CHDK and only run the EMMEM_TESTING at the end of the EXMEM block (which is most likely to be corrupted).
That would probably make it a lot easier.


Ok, I'll update the test build, hopefully in the next few days.


Phil.
CHDK ports:
  sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
  g12 (1.00c, 1.00e, 1.00f & 1.00g)
  sx130is (1.01d & 1.01f)
  ixus310hs (1.00a & 1.01a)
  sx40hs (1.00d, 1.00g & 1.00i)
  g1x (1.00e, 1.00f & 1.00g)
  g5x (1.00c, 1.01a, 1.01b)
  g7x2 (1.01a, 1.01b, 1.10b)

*

Offline philmoz

  • *****
  • 3441
    • Photos
Re: suba threading bug (was Re: 3D scanner with multiple IXUS160 cameras)
« Reply #27 on: 04 / October / 2017, 04:05:37 »
New test build.


Most of the EXMEM memory is now available to CHDK.
The last 8K is set up for EXMEM_TESTING.


Phil.

CHDK ports:
  sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
  g12 (1.00c, 1.00e, 1.00f & 1.00g)
  sx130is (1.01d & 1.01f)
  ixus310hs (1.00a & 1.01a)
  sx40hs (1.00d, 1.00g & 1.00i)
  g1x (1.00e, 1.00f & 1.00g)
  g5x (1.00c, 1.01a, 1.01b)
  g7x2 (1.01a, 1.01b, 1.10b)


Re: suba threading bug (was Re: 3D scanner with multiple IXUS160 cameras)
« Reply #28 on: 04 / October / 2017, 13:24:29 »
Of some interest:

The possibility that some previously reported crashes (prior to Phil's first test build) could, even then, be related to low memory availability, spurred me to load David Sykes (Microfunguy) 4755 based CHDK_LIGHT on all cameras.

Some comment about CHDK_LIGHT can be found here:
https://chdk.setepontos.com/index.php?topic=13059.msg131578#msg131578

CHDK_LIGHT can be downloaded from here:
http://sdm.camera/sdm/builds/CHDK_LIGHT.zip

Having failed to use chdkptp for the upload (crash on every camera) I done that manually.

After that, the first two attempts at switching to record & Cam12 crashed twice. I replaced it's power lead and battery emulator. On a 3rd attempt, Camera 9 crashed at switch to record. I then split Cams 13-16 into their own (3rd) power supply box. On a 4th attempt, Camera 12 crashed at switch to record. Oh dear!

I had noticed the first pole of cameras (Cams1-8) had been largely "stable" during recent testing (cf pole 2, Cams 9-16).

Cams 1-8 had been connected via a combination of a Manhattan 13 port hub and a 10 port Amazon Basic hub while Cams9-16 had been connected via an unbranded 8 port hub.
Having reconfigured to connect Cams1-8 to the Amazon Hub & Cams9-16 to the Manhattan hub (making the unbranded hub redundant):

First test, thereafter, of

lock_ptp_comms

repeat x 10

switch to record -> preshoot- > usb_sync_shoot -> open 5V switch -> play -> download to PC -> delete camera image

was successful

continuing the above but staying in record mode continuously, was successful for a further 29 repetitions before Cam 10 crashed at image download.   

It, tentatively, seems, to me, that identification / correction of the crash in chdk code could lead to a "fairly" stable system.

However, it is currently unclear, to me, whether the usbhub reorganisation is solely responsible for the improved results or whether that and the smaller footprint of CHDK_LIGHT was responsible.

New test build.

Will try that out asap, thanks. Can hopefully use chdkptp to upload from where I stand. 

It's a hack  ;)

A sophisticated one - I largely understand but may need to get back to you about something on that.

Thanks.


In case of interest, attached are the pre/post (edit: record) memory stats using CHDK_LIGHT via the cli trace (after hub reconfiguration described above).
   
« Last Edit: 04 / October / 2017, 15:47:57 by andrew.stephens.754365 »

*

Offline reyalp

  • ******
  • 13792
Re: suba threading bug (was Re: 3D scanner with multiple IXUS160 cameras)
« Reply #29 on: 04 / October / 2017, 16:27:45 »
The possibility that some previously reported crashes (prior to Phil's first test build) could, even then, be related to low memory availability,
IMO this is unlikely for several reasons:
1) This port was configured with 500K of EXMEM (in addition to ARAM and Canon heap), which should be plenty for normal operation. The memory issues using Philmoz first build were specifically due to EXMEM not being available.
2) The crash itself is not something that would be expected to result from an out of memory type situation
3) Low memory generally causes a variety of errors, where your original crash seemed to be consistently at one point.

Quote
In case of interest, attached are the pre/post (edit: record) memory stats using CHDK_LIGHT via the cli trace (after hub reconfiguration described above).
FWIW, I'm very unlikely to spend any of my time debugging CHDK_LIGHT or supporting it in any other way.
Don't forget what the H stands for.

 

Related Topics