A4000IS porting thread - page 10 - DryOS Development - CHDK Forum  

A4000IS porting thread

  • 205 Replies
  • 74477 Views
*

Offline srsa_4c

  • ******
  • 4451
Re: A4000IS porting thread
« Reply #90 on: 10 / April / 2013, 18:46:34 »
Advertisements
Seems to be an exmem-related issue, probably affects every image where the file size gets over that limit.

@ricardo28, @rkomar
Can you trigger this bug with regular L sized JPEG images at fine quality setting (without any CHDK overrides or scripts)? Shoot some very detailed scenes to test.

Re: A4000IS porting thread
« Reply #91 on: 10 / April / 2013, 19:37:33 »
Yes, that's the case. More complex images lead to the error. In my case, above 6200 KB aprox.
But the script interval.bas and the script waterwingz uploaded corrected my problems (tested just with a few photos in a row)
My camera has 101b firmware

So, just to be clear, did you check the file sizes in the latest tests to see if they were as big as the ones that were corrupted before?

Re: A4000IS porting thread
« Reply #92 on: 11 / April / 2013, 10:14:19 »
Seems to be an exmem-related issue, probably affects every image where the file size gets over that limit.

@ricardo28, @rkomar
Can you trigger this bug with regular L sized JPEG images at fine quality setting (without any CHDK overrides or scripts)? Shoot some very detailed scenes to test.

without the use of the script, I had no problems

Re: A4000IS porting thread
« Reply #93 on: 11 / April / 2013, 10:17:42 »
So, just to be clear, did you check the file sizes in the latest tests to see if they were as big as the ones that were corrupted before?

When repeating the corrupted photos, I couldn't take the photo with the same exact zoom, but very similar.
(if someone could tell me how to control the zoom in an exact manner through chdk it would be great, because manually it goes very fast from one point to another...)

They were not as big as the corrupted ones.


Re: A4000IS porting thread
« Reply #94 on: 12 / April / 2013, 13:40:17 »
When repeating the corrupted photos, I couldn't take the photo with the same exact zoom, but very similar.
(if someone could tell me how to control the zoom in an exact manner through chdk it would be great, because manually it goes very fast from one point to another...)

I don't know if you can do that through the CHDK menus, but if you have a setup where you are always zoomed the same amount, then you could add a call to the set_zoom() function in your script.  You have to know what value to set, though.  If you have chdkptp, then you can connect to the camera via USB and use get_zoom() to get the current value, and set_zoom() to change it.  Once you know the value you need, you can put it into your script.  If you don't want to bother with chdkptp, then you can probably guess the value you need.  For the a4000, you can set the zoom from 0 to 127.  If you look at the zoom slider on the display as you set the zoom, you can probably make a rough guess as to what the zoom level is at that point.

Re: A4000IS porting thread
« Reply #95 on: 14 / April / 2013, 14:46:47 »
I've been playing around with the ExMem settings to see if I can get some understanding of what's going on with the corrupted JPG files.  I don't have a deep understanding of how EXMEM works, so I've been somewhat slapdash about it.

The a4000 has 64MB of memory, and the default build sets EXMEM_BUFFER_SIZE=0x100000 and OPT_CHDK_IN_EXMEM=1.  I've gone over the arithmetic in the a4000/sub/101b/makefile.inc header, and it looks good to me.  I find that, with the default quality setting (Fine, I think), JPG files over something like 4300000 bytes get corrupted.  I don't intend on using the camera to take video, so I haven't been doing any ExMem testing with that (although the quick test I did to see if the "muting during zoom" feature worked produced good video with nafraf's build).

I've built the gcc-4.5.2 toolchain using the build-arm-elf-tools.sh script found on the wiki (very handy, by the way).  I have updated to the latest CHDK source code (r2698).

I started by disabling OPT_CHDK_IN_EXMEM and enabling OPT_EXMEM_TESTING.  The first thing I found was that I wasn't getting corrupted JPG files at the default settings like I did before.  Files over 5MB were still okay.  However, when I turned on the SuperFine JPG quality override, I started seeing the corruption again in the large (8-9MB) files.  The interesting thing is that the OPT_EXMEM_TESTING indicator in the bottom left of the screen always shows OK, even when the image gets corrupted.  I've uploaded an example image to http://www3.telus.net/rkomar/IMG_1370.JPG.  The top of the image is okay, but the bottom is moved up into the middle with the left and right sides switched (probably a wrong scanline start).  This is just a guess, but I wonder if we're seeing corruption due to a lack of memory rather than due to overwriting the wrong things.

I started playing with EXMEM_BUFFER_SIZE=0x100000 setting to see if it would make a difference.  Setting it to 0x400000 causes the camera to crash immediately after it takes the shot.  Setting it to 0x200000 and 0x80000 keeps it from crashing, but doesn't stop the corruption.  The OPT_EXMEM_TESTING indicator still shows OK in all cases.

That's about all I know for now.  If anyone has some suggestions for further debugging this, I would be glad to try them out.
« Last Edit: 14 / April / 2013, 15:09:19 by rkomar »

*

Offline philmoz

  • *****
  • 3450
    • Photos
Re: A4000IS porting thread
« Reply #96 on: 14 / April / 2013, 18:39:40 »
I've been playing around with the ExMem settings to see if I can get some understanding of what's going on with the corrupted JPG files.  I don't have a deep understanding of how EXMEM works, so I've been somewhat slapdash about it.

The a4000 has 64MB of memory, and the default build sets EXMEM_BUFFER_SIZE=0x100000 and OPT_CHDK_IN_EXMEM=1.  I've gone over the arithmetic in the a4000/sub/101b/makefile.inc header, and it looks good to me.  I find that, with the default quality setting (Fine, I think), JPG files over something like 4300000 bytes get corrupted.  I don't intend on using the camera to take video, so I haven't been doing any ExMem testing with that (although the quick test I did to see if the "muting during zoom" feature worked produced good video with nafraf's build).

I've built the gcc-4.5.2 toolchain using the build-arm-elf-tools.sh script found on the wiki (very handy, by the way).  I have updated to the latest CHDK source code (r2698).

I started by disabling OPT_CHDK_IN_EXMEM and enabling OPT_EXMEM_TESTING.  The first thing I found was that I wasn't getting corrupted JPG files at the default settings like I did before.  Files over 5MB were still okay.  However, when I turned on the SuperFine JPG quality override, I started seeing the corruption again in the large (8-9MB) files.  The interesting thing is that the OPT_EXMEM_TESTING indicator in the bottom left of the screen always shows OK, even when the image gets corrupted.  I've uploaded an example image to http://www3.telus.net/rkomar/IMG_1370.JPG.  The top of the image is okay, but the bottom is moved up into the middle with the left and right sides switched (probably a wrong scanline start).  This is just a guess, but I wonder if we're seeing corruption due to a lack of memory rather than due to overwriting the wrong things.

I started playing with EXMEM_BUFFER_SIZE=0x100000 setting to see if it would make a difference.  Setting it to 0x400000 causes the camera to crash immediately after it takes the shot.  Setting it to 0x200000 and 0x80000 keeps it from crashing, but doesn't stop the corruption.  The OPT_EXMEM_TESTING indicator still shows OK in all cases.

That's about all I know for now.  If anyone has some suggestions for further debugging this, I would be glad to try them out.

It sounds like you may need to use both the Canon heap and EXMEM, and minimise the memory usage in both.

To do this you enable OPT_EXMEM_MALLOC; but leave OPT_CHDK_IN_EXMEM and OPT_EXMEM_TESTING disabled.
You should also use the trunk (1.2) version of CHDK since this moves the Lua and uBasic script code to modules which greatly reduces the CHDK core size.

First disable OPT_CHDK_IN_EXMEM and enable OPT_EXMEM_TESTING - now see how much free memory you have (Miscellaneous stuff --> Show Memory Info). Do this on an SD card with no images, right after power up in playback mode (and don't run any CHDK script either manually or autostart). This will tell us how much Canon heap memory is available when CHDK is loaded in the Canon heap space.

Now disable OPT_EXMEM_TESTING (leave OPT_CHDK_IN_EXMEM disabled). Reduce the EXMEM_BUFFER_SIZE to 512KB (0x80000) and run your JPEG tests. You are now running the the CHDK core in the Canon heap; but using EXMEM for any CHDK dynamic memory allocations (loading modules, malloc/free, etc).

If you are still getting corrupted JPEGS's reduce EXMEM_BUFFER_SIZE to 256K (0x40000) and try again.

You might be able to run with EXMEM_BUFFER_SIZE down to 128K; but you will probably not be able to use Lua scripts at that size.

Phil.
CHDK ports:
  sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
  g12 (1.00c, 1.00e, 1.00f & 1.00g)
  sx130is (1.01d & 1.01f)
  ixus310hs (1.00a & 1.01a)
  sx40hs (1.00d, 1.00g & 1.00i)
  g1x (1.00e, 1.00f & 1.00g)
  g5x (1.00c, 1.01a, 1.01b)
  g7x2 (1.01a, 1.01b, 1.10b)

Re: A4000IS porting thread
« Reply #97 on: 14 / April / 2013, 21:00:54 »
It sounds like you may need to use both the Canon heap and EXMEM, and minimise the memory usage in both.

To do this you enable OPT_EXMEM_MALLOC; but leave OPT_CHDK_IN_EXMEM and OPT_EXMEM_TESTING disabled.
You should also use the trunk (1.2) version of CHDK since this moves the Lua and uBasic script code to modules which greatly reduces the CHDK core size.

First disable OPT_CHDK_IN_EXMEM and enable OPT_EXMEM_TESTING - now see how much free memory you have (Miscellaneous stuff --> Show Memory Info). Do this on an SD card with no images, right after power up in playback mode (and don't run any CHDK script either manually or autostart). This will tell us how much Canon heap memory is available when CHDK is loaded in the Canon heap space.

Now disable OPT_EXMEM_TESTING (leave OPT_CHDK_IN_EXMEM disabled). Reduce the EXMEM_BUFFER_SIZE to 512KB (0x80000) and run your JPEG tests. You are now running the the CHDK core in the Canon heap; but using EXMEM for any CHDK dynamic memory allocations (loading modules, malloc/free, etc).

If you are still getting corrupted JPEGS's reduce EXMEM_BUFFER_SIZE to 256K (0x40000) and try again.

You might be able to run with EXMEM_BUFFER_SIZE down to 128K; but you will probably not be able to use Lua scripts at that size.

Phil.

With OPT_EXMEM_TESTING enabled and OPT_CHDK_IN_EXMEM  disabled, Show Memory Info returned: Free Memory = 654360 bytes, CHDK Size = 140988, loaded at 0x157CA4.

I then disabled OPT_EXMEM_TESTING and rebuilt CHDK with EXMEM_BUFFER_SIZE=0x40000, and then with 0x20000, but I got corrupted JPG files in both cases with SuperFine quality enabled.

I ran Show Memory Info in each case, and it showed Free Memory = 254888 when EXMEM_BUFFER_SIZE=0x40000, and Free Memory = 123816 bytes when EXMEM_BUFFER_SIZE=0x20000.  It seems strange to me that the free memory went down as EXMEM_BUFFER_SIZE went down.  Does is it actually show the amount of free memory in the EXMEM area (perhaps combined with what is in the heap)?


*

Offline philmoz

  • *****
  • 3450
    • Photos
Re: A4000IS porting thread
« Reply #98 on: 14 / April / 2013, 21:11:51 »
With OPT_EXMEM_TESTING enabled and OPT_CHDK_IN_EXMEM  disabled, Show Memory Info returned: Free Memory = 654360 bytes, CHDK Size = 140988, loaded at 0x157CA4.

If you have that much free you can try disabling EXMEM altogether (disable OPT_EXMEM_MALLOC).

Quote
I then disabled OPT_EXMEM_TESTING and rebuilt CHDK with EXMEM_BUFFER_SIZE=0x40000, and then with 0x20000, but I got corrupted JPG files in both cases with SuperFine quality enabled.

If the Canon Fine mode works without corruption then just use that - the chance you will be able to tell the difference between Fine and SuperFine is pretty small.

Quote
I ran Show Memory Info in each case, and it showed Free Memory = 254888 when EXMEM_BUFFER_SIZE=0x40000, and Free Memory = 123816 bytes when EXMEM_BUFFER_SIZE=0x20000.  It seems strange to me that the free memory went down as EXMEM_BUFFER_SIZE went down.  Does is it actually show the amount of free memory in the EXMEM area (perhaps combined with what is in the heap)?

The Show Memory Info only shows the info for the heap being used by CHDK - when EXMEM is enabled this will be the EXMEM heap only, otherwise it shows the Canon heap.

Phil.
CHDK ports:
  sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
  g12 (1.00c, 1.00e, 1.00f & 1.00g)
  sx130is (1.01d & 1.01f)
  ixus310hs (1.00a & 1.01a)
  sx40hs (1.00d, 1.00g & 1.00i)
  g1x (1.00e, 1.00f & 1.00g)
  g5x (1.00c, 1.01a, 1.01b)
  g7x2 (1.01a, 1.01b, 1.10b)

Re: A4000IS porting thread
« Reply #99 on: 15 / April / 2013, 00:21:10 »
I disabled OPT_EXMEM_MALLOC and OPT_CHDK_IN_EXMEM, and now the SuperFine JPGs aren't corrupted anymore.  Thanks philmoz!  I'll keep testing for a while to see if there are any problems with building this way.

Personally, I don't intend on using SuperFine JPG quality, however, I figured that someone out there would, and then they'd come back here with more bug reports.  If we could figure out how to get it to work, then that would also get rid of that extra work of responding to those reports.

 

Related Topics


SimplePortal 2.3.6 © 2008-2014, SimplePortal