Shot Histogram Request

reyalp

14082

Re: Shot Histogram Request

« Reply #260 on: 11 / April / 2013, 23:31:51 »

Advertisements

Here's the weirdness. In lib/lua/ldo.c

Code: [Select]

  status = luaD_rawrunprotected(L, resume, L->top - nargs);
  if (status != 0) {  /* error? */
    dbg_dump_write("A/LUARES1.DMP",0,sizeof(lua_State),(void *)L);
    L->status = cast_byte(status);  /* mark thread as `dead' */
    luaD_seterrorobj(L, status, L->top);
    L->ci->top = L->top;
  }

This is the only place LUARES1.DMP can be created. It will be hit if luaD_rawrunprotected returns non-zero.

Here's luaD_rawrunprotected

Code: [Select]

  int r = setjmp(lj.b);
  if(r == 0) {
    (*f)(L, ud);
  } else {
    dbg_dump_write("LUALJ1.DMP",0,r,NULL);
  }
  if(lj.status) {
    dbg_dump_write("A/LUALJ2.DMP",0,sizeof(lj),(char *)&lj);
  }

  L->errorJmp = lj.previous;  /* restore old error handler */
  return lj.status;

If lj.status is non-zero, A/LUALJ2.DMP should be written.

What am I missing

If you try a regular runtime error (return 1+nil), you get

LUALJ1.DMP - because setjmp returned via longjump
LUALJ2.DMP - because lj.status was non non-zero
LUARES1.DMP - because we hit the error case in resume (same as the real bug)
LUATHROW.DMP - because luaD_throw threw (longjmp) the error

My gut want's blame setjmp/longjmp, especially since it is a custom implementation lifted from web, but I can't really see how.

Logged

Don't forget what the H stands for.

philmoz

3450

Re: Shot Histogram Request

« Reply #261 on: 12 / April / 2013, 00:02:13 »

Quote from: reyalp on 11 / April / 2013, 23:31:51

My gut want's blame setjmp/longjmp, especially since it is a custom implementation lifted from web, but I can't really see how.

Weird indeed.
The setjmp.h / setjmp.s code is difficult to follow with all the #ifdef clutter.
Is it worth stripping these files back to just what is needed for CHDK?

Phil.

Logged

CHDK ports:
sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
g12 (1.00c, 1.00e, 1.00f & 1.00g)
sx130is (1.01d & 1.01f)
ixus310hs (1.00a & 1.01a)
sx40hs (1.00d, 1.00g & 1.00i)
g1x (1.00e, 1.00f & 1.00g)
g5x (1.00c, 1.01a, 1.01b)
g7x2 (1.01a, 1.01b, 1.10b)

reyalp

14082

Re: Shot Histogram Request

« Reply #262 on: 12 / April / 2013, 00:09:19 »

Quote from: philmoz on 12 / April / 2013, 00:02:13

The setjmp.h / setjmp.s code is difficult to follow with all the #ifdef clutter.
Is it worth stripping these files back to just what is needed for CHDK?

It wouldn't hurt, but you can get a de-cluttered version from the disassembler

Code: [Select]

00010c20 <setjmp>:
   10c20:	4778      	bx	pc
   10c22:	46c0      	nop			; (mov r8, r8)

00010c24 <.arm_start_of.setjmp>:
   10c24:	e8a07ff0 	stmia	r0!, {r4, r5, r6, r7, r8, r9, sl, fp, ip, sp, lr}
   10c28:	e3a00000 	mov	r0, #0
   10c2c:	e31e0001 	tst	lr, #1
   10c30:	01a0f00e 	moveq	pc, lr
   10c34:	e12fff1e 	.word	0xe12fff1e

00010c38 <longjmp>:
   10c38:	4778      	bx	pc
   10c3a:	46c0      	nop			; (mov r8, r8)

00010c3c <.arm_start_of.longjmp>:
   10c3c:	e8b07ff0 	ldm	r0!, {r4, r5, r6, r7, r8, r9, sl, fp, ip, sp, lr}
   10c40:	e1b00001 	movs	r0, r1
   10c44:	03a00001 	moveq	r0, #1
   10c48:	e31e0001 	tst	lr, #1
   10c4c:	01a0f00e 	moveq	pc, lr
   10c50:	e12fff1e 	.word	0xe12fff1e

Note the last .word is a bx lr

Logged

Don't forget what the H stands for.

philmoz

3450

Re: Shot Histogram Request

« Reply #263 on: 12 / April / 2013, 00:53:04 »

Quote from: reyalp on 12 / April / 2013, 00:09:19

Quote from: philmoz on 12 / April / 2013, 00:02:13
The setjmp.h / setjmp.s code is difficult to follow with all the #ifdef clutter.
Is it worth stripping these files back to just what is needed for CHDK?
It wouldn't hurt, but you can get a de-cluttered version from the disassembler
Code: [Select]
00010c20 <setjmp>: 10c20: 4778 bx pc 10c22: 46c0 nop ; (mov r8, r8) 00010c24 <.arm_start_of.setjmp>: 10c24: e8a07ff0 stmia r0!, {r4, r5, r6, r7, r8, r9, sl, fp, ip, sp, lr} 10c28: e3a00000 mov r0, #0 10c2c: e31e0001 tst lr, #1 10c30: 01a0f00e moveq pc, lr 10c34: e12fff1e .word 0xe12fff1e 00010c38 <longjmp>: 10c38: 4778 bx pc 10c3a: 46c0 nop ; (mov r8, r8) 00010c3c <.arm_start_of.longjmp>: 10c3c: e8b07ff0 ldm r0!, {r4, r5, r6, r7, r8, r9, sl, fp, ip, sp, lr} 10c40: e1b00001 movs r0, r1 10c44: 03a00001 moveq r0, #1 10c48: e31e0001 tst lr, #1 10c4c: 01a0f00e moveq pc, lr 10c50: e12fff1e .word 0xe12fff1eNote the last .word is a bx lr

How is the first 'bx pc' instruction in each function switching to ARM mode?
The PC value should still be odd at this point so why would this cause a mode switch?

Also I thought using the PC register in any BX instructions was deprecated.

Phil.

« Last Edit: 12 / April / 2013, 00:57:14 by philmoz »

Logged

CHDK ports:
sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
g12 (1.00c, 1.00e, 1.00f & 1.00g)
sx130is (1.01d & 1.01f)
ixus310hs (1.00a & 1.01a)
sx40hs (1.00d, 1.00g & 1.00i)
g1x (1.00e, 1.00f & 1.00g)
g5x (1.00c, 1.01a, 1.01b)
g7x2 (1.01a, 1.01b, 1.10b)

reyalp

14082

Re: Shot Histogram Request

« Reply #264 on: 12 / April / 2013, 01:15:45 »

Quote from: philmoz on 12 / April / 2013, 00:53:04

How is the first 'bx pc' instruction in each function switching to ARM mode?
The PC value should still be odd at this point so why would this cause a mode switch?

From: "ARM Architecture Reference Manual" A6.1 About the Thumb instruction set

Quote

When R15 is read, bit[0] is zero and bits[31:1] contain the PC.

Quote

Also I thought using the PC register in any BX instructions was deprecated.

gcc interworking does it the same way:

Code: [Select]

00175db8 <__mkdir_from_thumb>:
  175db8:	4778      	bx	pc
  175dba:	46c0      	nop			; (mov r8, r8)
  175dbc:	eaff847a 	b	156fac <mkdir>

The kicker is that for any defect in setjmp to be causing this, it would have to succeed 99.999% or more of the time, every call to lua resume goes through the setjmp code, and every lua error goes through longjmp. More likely would be the jumpbuf on the stack getting corrupted, but the check on lj.status is done after it has returned.

I think my next approach is to keep some kind of circular log to see where it actually comes from when that code is hit.

« Last Edit: 12 / April / 2013, 01:17:28 by reyalp »

Logged

Don't forget what the H stands for.

lapser

1093

No error with one Lua thread!

« Reply #265 on: 12 / April / 2013, 02:06:26 »

I decided to see what would happen if I changed lua_script_start() so it only used one Lua thread, by changing 1 line:

Code: [Select]

    //Lt = lua_newthread( L );
    Lt=L;

I left the SX50 with the new build running at the house, and took the G1X and SX260 to a spot with a better view of more sky so I could catch a ISS flyby. I managed to see the ISS in both cameras. The SX260 was using the old build (with 2 Lua threads), and it triggered the bug before the ISS appeared. So I restarted the script, and the bug happened again after a few seconds. I started it again, and it ran for about an hour with no bug, and caught the ISS.

So I got home expecting the SX50 to have stopped again, but it was still running. So mabe there's a bug in Lua related to multiple Lua threads, possibly a garbage collection problem as Phil suggested.

I'll put the new, 1 thread build on all the cameras, and see if the bug recurs over the next few nights.

Whe I started the script, the meters weren't quite right, so I stopped it with the menu key (normal exit) after about 10 seconds, and restarted. I discovered that there was a new .dmp file created, time stamped when I stopped the script. I'll attach it for you to look at, along with the modules.log.

LUAASRT.DMP (2.25 kB - downloaded 22 times.)

MODULES.LOG (0.3 kB - downloaded 22 times.)

Logged

EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos

reyalp

14082

Re: Shot Histogram Request

« Reply #266 on: 12 / April / 2013, 02:20:54 »

Here's quick and dirty patch to log some calls to a buffer, maybe we can see how it gets to that "impossible" return value.

I very much doubt the L vs Lt thing is the cause, but then again, I don't have any good explanation so why not

lua-debug-trunk-r2694.patch (4.79 kB - downloaded 25 times.)

Logged

Don't forget what the H stands for.

lapser

1093

Re: Shot Histogram Request

« Reply #267 on: 12 / April / 2013, 02:41:15 »

Quote from: reyalp on 12 / April / 2013, 02:20:54

Here's quick and dirty patch to log some calls to a buffer, maybe we can see how it gets to that "impossible" return value.
so why not

I'll install the new patch and give it a try tomorrow when I test all 3 cameras with the single thread. Or I can go back to 2 threads and try to trigger the bug again so you'll have the new log?

I wonder what the second thread is doing? And why does ptpCamGui fail to init when there's only one thread. Do you think you could test this on your cameras with the Lt=L change and see if you can get ptpCam to init? I would really like to solve that problem because 2 of my SD cards don't have write protect tabs now and ptpCamGui is the only way to load new builds (other than Scotch tape). Thanks.

Logged

EOS-M3_120f / SX50_100b / SX260_101a / G1X_100g / D20_100b
https://www.youtube.com/user/DrLapser/videos

reyalp

14082

Re: Shot Histogram Request

« Reply #268 on: 12 / April / 2013, 02:50:53 »

Quote from: lapser on 12 / April / 2013, 02:41:15

Or I can go back to 2 threads and try to trigger the bug again so you'll have the new log?

It's up to you, but for me the closer it is to stock CHDK the less chance there is of confounding factors.

Quote

I wonder what the second thread is doing?

As I said in the other thread (

), nothing. Everything is running the Lt thread, nothing is executed in the "main" thread at all. Lua "threads" aren't really like OS threads, a better name would be "coroutine state" or something like that. They don't execute unless they are explicitly executed.

Quote

And why does ptpCamGui fail to init when there's only one thread.

I have no idea, and I don't have time to dig into that code right now.

Logged

Don't forget what the H stands for.

philmoz

3450

Re: Shot Histogram Request

« Reply #269 on: 12 / April / 2013, 08:12:39 »

Quote from: lapser on 12 / April / 2013, 02:41:15

And why does ptpCamGui fail to init when there's only one thread. Do you think you could test this on your cameras with the Lt=L change and see if you can get ptpCam to init? I would really like to solve that problem because 2 of my SD cards don't have write protect tabs now and ptpCamGui is the only way to load new builds (other than Scotch tape). Thanks.

I don't use ptpCamGui; but chdkptp works fine with this change.
You can also use chdkptp in console mode (with batch files) to simplify downloading updates to the camera.

Phil.

Logged

CHDK ports:
sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
g12 (1.00c, 1.00e, 1.00f & 1.00g)
sx130is (1.01d & 1.01f)
ixus310hs (1.00a & 1.01a)
sx40hs (1.00d, 1.00g & 1.00i)
g1x (1.00e, 1.00f & 1.00g)
g5x (1.00c, 1.01a, 1.01b)
g7x2 (1.01a, 1.01b, 1.10b)