Camera crash on startup - investigation. - page 2 - General Discussion and Assistance - CHDK Forum
« previous next »
Pages: 1 [2] 3 4 5 6 7 ... 12   Go Down

Camera crash on startup - investigation.

  • 112 Replies
  • 34123 Views
*

Offline Bernd R

  • ****
  • 430
Re: Camera crash on startup - investigation.
« Reply #10 on: 09 / March / 2011, 09:50:22 »
Advertisements
I find this address in Ixus 1000 HS here.

sub_FF872800            ; CODE XREF: sub_FF872508+1Cp
               ; _sub_FF87253C__FsIoNotify.c__540+1Cp

I test a little and i get no crash and good speed on RAW series shoot.But maybe this fix is not rock solid

I get in mind, with the RAW series crash, the check for free slot report maybe a slot is free for chdk. chdk use the last slot free.now when call mkdir, there is no slot free, and camera crash.

this maybe happen not always, maybe it happen only somtimes afer longer use, depend on timing.i dont know.maybe i better let the old patch that do not call mkdir on 2. and more raw in
Logged
Ixus 1000 HS
*

Offline philmoz

  • *****
  • 3332
    • Photos
Re: Camera crash on startup - investigation.
« Reply #11 on: 15 / March / 2011, 19:28:27 »
Back to the drawing board :(

The previous fix (checking for a free FsIoNotify slot and calling 'open' instead of 'Open') does not work consistently.
I am still getting crashes on the G12, although far less often.

Reviewing the 'Open' logic in the firmware again, it also appears that the same assert is thrown if Open finds the new file handle is already in the FsIoNotify array. The Close code closes the file handle before removing it from the FsIoNotify array and neither function appears to have any locking around use of the array or file handles. Perhaps it's a timing problem with a file being closed in one task, and then before the entry is removed from the array, another task opens a file and gets the same file handle back.

I'm going to try implementing the logic of the firmware 'Open' code inside CHDK (for files opened by CHDK) and put in more diagnostics to see if I can work out why this is happening.

Phil.
Logged
CHDK ports:
  sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
  g12 (1.00c, 1.00e, 1.00f & 1.00g)
  sx130is (1.01d & 1.01f)
  ixus310hs (1.00a & 1.01a)
  sx40hs (1.00d, 1.00g & 1.00i)
  g1x (1.00e, 1.00f & 1.00g)
  g5x (1.00c, 1.01a, 1.01b)
  g7x2 (1.01a, 1.01b, 1.10b)
*

Offline wowsers51

  • **
  • 64
Re: Camera crash on startup - investigation.
« Reply #12 on: 15 / March / 2011, 20:09:57 »
I am following this thread with a lot of hope for my SX20. I cannot help as I have no skills in solving these crashes but all I know is that I am getting tired of those crashes happening randomly. :-[
Logged
SX20 1.02d
*

Offline philmoz

  • *****
  • 3332
    • Photos
Re: Camera crash on startup - investigation.
« Reply #13 on: 16 / March / 2011, 07:23:51 »
Some more progress on this problem (at least for the G12 & SX30).

After almost a week with no crashes the G12 started doing it again, today it was crashing on every 2nd or 3rd startup (with no changes to CHDK).

I've now copied the 'Open' (actually eventproc_export_Open) code from the firmware into the G12 CHDK code along with the problem code from the '_sub_FF8735FC__FsIoNotify.c__0' function (see first post).

This '_sub_FF8735FC__FsIoNotify.c__0' code has another path that can cause the crash that I missed earlier. In addition to crashing if no empty slots are found in the array, it will also crash if it finds the file handle (returned from 'open') already in the array.

So I modified the code so that instead of crashing (calling DebugAssert) if it finds the file handle, it just returns without adding the handle to the array. This makes the file open behave as though the 'open' function was called instead of 'Open'.

I also added code to increment counters on each successful and failed call, and displayed these on screen in gui_draw_debug_vals_osd.

On a normal startup it displays 0 for the failed count and 4 for the successful count. Occasionally it will now display 1 for the failed count and 3 for the successful count meaning that it is indeed hitting the 'found file handle in array' error condition. Since my code no longer crashes in this case, the camera continues to start and CHDK seems to work correctly.

What I now suspect is happening is that one task (probably a Canon task) is closing a file handle and before that call to 'Close' can remove the closed file from the FsIoNotify array our CHDK task calls 'Open'. Because the previous file was closed we get back the same file handle from 'open'; but this value is still in the array so the firmware code crashes.

The code changes are below.

boot.c (this is going to be camera specific so the equivalent code will need to be found, copied and modified from the firmware for each camera / firmware version):
Code: [Select]
int fsionotify_compfail = 0; // count of number of times the file handle was already in the array
int fsionotify_success  = 0; // count of number of times the code succeeded

int __attribute__((naked,noinline)) _Open(const char *name, int flags, int mode) {

    asm volatile (
"STMFD   SP!, {R4-R8,LR} \n"
"MOV     R6, R0 \n"
"LDRB    R0, [R0] \n"
"MOV     R7, R2 \n"
"MOV     R4, R1 \n"
"BL      sub_FF875690 \n"
"MOV     R8, R0 \n"
"MOV R0, #1 \n"
"BIC     R5, R0, R4,LSR#12 \n"
"BIC     R4, R4, #0x9000 \n"
"MOV R0, R8 \n"
"MOV     R1, #1 \n"
"BL      sub_FF876A20 \n"
"MOV     R2, R7 \n"
"MOV     R1, R4 \n"
"MOV     R0, R6 \n"
"BL      _open \n"
"CMP R5, #0 \n"
"MOV     R7, R0 \n"
"MOVNE R0, R7 \n"
"MOVNE   R2, R4 \n"
"MOVNE   R1, R6 \n"
"BLNE    sub_FF8735FC_my \n"
"MOV     R0, R8 \n"
"BL      sub_FF876A98 \n"
"MOV     R0, R7 \n"
"LDMFD   SP!, {R4-R8,PC} \n"
    );

return 0; // stop compiler warning
}

void __attribute__((naked,noinline)) sub_FF8735FC_my() {

    asm volatile (
"STMFD   SP!, {R4-R8,LR} \n"
"MOV     R5, R0 \n"
"LDR     R0, =0x2BA0 \n"
"MOV     R7, R1 \n"
"LDR     R0, [R0,#4] \n"
"MOV     R6, R2 \n"
"CMP     R0, #0 \n"
"LDMEQFD SP!, {R4-R8,PC} \n"
"CMP     R5, #0 \n"
"LDMLTFD SP!, {R4-R8,PC} \n"
"MOV     R4, #0 \n"
"LDR     R2, =0x39684 \n"
"MOV     R0, #0 \n"
"loc_FF873630: \n"
"ADD     R1, R0, R0,LSL#1 \n"
"LDR     R1, [R2,R1,LSL#5] \n"
"CMN     R1, #1 \n"
"ADDEQ   R0, R0, R0,LSL#1 \n"
"ADDEQ   R4, R2, R0,LSL#5 \n"
"BEQ     loc_FF873658 \n"
"CMP     R1, R5 \n"

"BEQ loc_2 \n" // + // branch if found entry matching new file handle

"ADDNE   R0, R0, #1 \n"
"CMPNE   R0, #0xA \n"
"BLT     loc_FF873630 \n"

"loc_FF873658: \n"
"CMP     R4, #0 \n"
"LDREQ   R1, =0x1C9 \n"
"LDREQ   R0, =0xFF873544 \n" //aFsionotify_c
"BLEQ    _DebugAssert \n"

"LDR R1, =fsionotify_success \n" // + // increment counter
"LDR R0, [R1] \n" // + // of successful calls
"ADD R0, R0, #1 \n" // +
"STR R0, [R1] \n" // +

"MOV     R0, #0 \n" // original code - save handle in array
"STR     R0, [R4,#0x58] \n"
"STR     R5, [R4] \n"
"MOV     R0, R4 \n"
"MOV     R1, R7 \n"
"STR     R6, [R4,#0x24] \n"
"BL      sub_FF8733C4 \n"
"ADD     R1, R4, #0x28 \n"
"MOV     R0, R7 \n"
"LDMFD   SP!, {R4-R8,LR} \n"
"B       sub_FF837244 \n"

"loc_2: \n" // + // Handle case when new file handle returned from _open is already in array
"LDR R1, =fsionotify_compfail \n" // + // increment counter then return rather than throw exception
"LDR R0, [R1] \n" // + // equivalent to calling _open rather than _Open
"ADD R0, R0, #1 \n" // +
"STR R0, [R1] \n" // +
"LDMFD   SP!, {R4-R8,PC} \n" // +
    );
}

You also need to comment out the entry for Open in stubs_entry_2.s

Phil.

Logged
CHDK ports:
  sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
  g12 (1.00c, 1.00e, 1.00f & 1.00g)
  sx130is (1.01d & 1.01f)
  ixus310hs (1.00a & 1.01a)
  sx40hs (1.00d, 1.00g & 1.00i)
  g1x (1.00e, 1.00f & 1.00g)
  g5x (1.00c, 1.01a, 1.01b)
  g7x2 (1.01a, 1.01b, 1.10b)
*

Offline philmoz

  • *****
  • 3332
    • Photos
Re: Camera crash on startup - investigation.
« Reply #14 on: 16 / March / 2011, 07:32:10 »
Quote from: wowsers51 on 15 / March / 2011, 20:09:57
I am following this thread with a lot of hope for my SX20. I cannot help as I have no skills in solving these crashes but all I know is that I am getting tired of those crashes happening randomly. :-[

If you are interested I can create a version for the SX20 for you to test with the current fix (see previous post). I don't have an SX20 so I can't test it.

Your signature says you have firmware 1.02d - is that correct?

Phil.
Logged
CHDK ports:
  sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
  g12 (1.00c, 1.00e, 1.00f & 1.00g)
  sx130is (1.01d & 1.01f)
  ixus310hs (1.00a & 1.01a)
  sx40hs (1.00d, 1.00g & 1.00i)
  g1x (1.00e, 1.00f & 1.00g)
  g5x (1.00c, 1.01a, 1.01b)
  g7x2 (1.01a, 1.01b, 1.10b)
*

Offline wowsers51

  • **
  • 64
Re: Camera crash on startup - investigation.
« Reply #15 on: 16 / March / 2011, 14:10:00 »
Quote from: philmoz on 16 / March / 2011, 07:32:10
Quote from: wowsers51 on 15 / March / 2011, 20:09:57
I am following this thread with a lot of hope for my SX20. I cannot help as I have no skills in solving these crashes but all I know is that I am getting tired of those crashes happening randomly. :-[

If you are interested I can create a version for the SX20 for you to test with the current fix (see previous post). I don't have an SX20 so I can't test it.

Your signature says you have firmware 1.02d - is that correct?

Phil.

Hi Phil and thanks for responding,

Yes I have firmware 1.02d and I would be very interested if you could develop something to help cure this pesky problem. I will wait until you give me an answer on this. Many thanks.
Logged
SX20 1.02d
*

Offline philmoz

  • *****
  • 3332
    • Photos
Re: Camera crash on startup - investigation.
« Reply #16 on: 16 / March / 2011, 18:26:16 »
Quote from: wowsers51 on 16 / March / 2011, 14:10:00
Yes I have firmware 1.02d and I would be very interested if you could develop something to help cure this pesky problem. I will wait until you give me an answer on this. Many thanks.

Try the attached version (extract the diskboot.bin from the zip, everything else should be unchanged).

If it works this will display two numbers at the bottom right of the LCD. The first is the count of failed calls to _Open (where the file handle is found to already exist in the array). The second is the count of successful calls.

Normally the first should always be 0. If it isn't 0 at startup then it means that the error condition was hit; but the new code has ignored it and continued on instead of crashing.

Phil.
Logged
CHDK ports:
  sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
  g12 (1.00c, 1.00e, 1.00f & 1.00g)
  sx130is (1.01d & 1.01f)
  ixus310hs (1.00a & 1.01a)
  sx40hs (1.00d, 1.00g & 1.00i)
  g1x (1.00e, 1.00f & 1.00g)
  g5x (1.00c, 1.01a, 1.01b)
  g7x2 (1.01a, 1.01b, 1.10b)
*

Offline wowsers51

  • **
  • 64
Re: Camera crash on startup - investigation.
« Reply #17 on: 16 / March / 2011, 22:17:17 »
Thanks you very much. I will try that starting tomorrow and keep you informed whether it work or not. :D
Logged
SX20 1.02d
*

Offline Bernd R

  • ****
  • 430
Re: Camera crash on startup - investigation.
« Reply #18 on: 17 / March / 2011, 08:52:51 »
@philmoz

this sound logic, that the problem is not of too much handles.
I get the idea to make a fix without special camera asm code, if its possible that only write commands use fsio in chdk, because only slowdown is in chdk the write raw.

Or if that not work exakt, can add a special openmode that is check in _Open, if its set it open the raw save only with fsio

what do you think ?
« Last Edit: 17 / March / 2011, 09:01:47 by Bernd R »
Logged
Ixus 1000 HS
*

Offline philmoz

  • *****
  • 3332
    • Photos
Re: Camera crash on startup - investigation.
« Reply #19 on: 17 / March / 2011, 20:10:12 »
Quote from: Bernd R on 17 / March / 2011, 08:52:51
@philmoz

this sound logic, that the problem is not of too much handles.
I get the idea to make a fix without special camera asm code, if its possible that only write commands use fsio in chdk, because only slowdown is in chdk the write raw.

Or if that not work exakt, can add a special openmode that is check in _Open, if its set it open the raw save only with fsio

what do you think ?

That's a good idea and might work out well - if the open flags specify a read-only file then we can use '_open' rather than '_Open'. This should not have any significant performance hit.

I'll do some testing and see what happens.

Phil.
Logged
CHDK ports:
  sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
  g12 (1.00c, 1.00e, 1.00f & 1.00g)
  sx130is (1.01d & 1.01f)
  ixus310hs (1.00a & 1.01a)
  sx40hs (1.00d, 1.00g & 1.00i)
  g1x (1.00e, 1.00f & 1.00g)
  g5x (1.00c, 1.01a, 1.01b)
  g7x2 (1.01a, 1.01b, 1.10b)
Pages: 1 [2] 3 4 5 6 7 ... 12   Go Up
« previous next »
 

Related Topics