experimental alternate memory allocation - page 12 - General Discussion and Assistance - CHDK Forum
« previous next »
Pages: 1 ... 7 8 9 10 11 [12] 13   Go Down

experimental alternate memory allocation

  • 128 Replies
  • 59866 Views
*

Offline funnel

  • ****
  • 349
Re: experimental alternate memory allocation
« Reply #110 on: 03 / June / 2011, 13:49:12 »
Advertisements
Thank you philmoz for your great tutorial. I managed to load CHDK into extended memory of SX220 and that fixed a lot of problems for me. With 4MB it looks like it overlaps with the RAW buffers and crashes when shooting. With 2MB I had no problems so far.

Code: [Select]
NHSTUB(exmem_alloc, 0xFF085434)
Code: [Select]
ifdef OPT_CHDK_IN_EXMEM
MEMISOSTART=0x77867E0
else
MEMISOSTART=0x186AE4
endif

MAXRAMADDR=0x7FFFFFF
EXMEM_BUFFER_SIZE=0x200000
EXMEM_HEAP_SKIP=0x679800
Logged
*

Offline funnel

  • ****
  • 349
Re: experimental alternate memory allocation
« Reply #111 on: 09 / June / 2011, 17:27:33 »
Can somebody confirm that Quality Overrides work with chdk in exmem? I can't set the superfine quality which works in the normal memory allocation.
Logged
*

Offline philmoz

  • *****
  • 3450
    • Photos
Re: experimental alternate memory allocation
« Reply #112 on: 09 / June / 2011, 18:28:09 »
Quote from: funnel on 09 / June / 2011, 17:27:33
Can somebody confirm that Quality Overrides work with chdk in exmem? I can't set the superfine quality which works in the normal memory allocation.

Works on G12 and SX30.
Your EXMEM_HEAP_SKIP value is large (6.4MB) and if 4MB of EXMEM causes problems with RAW buffers (from previous post) it's possible there just isn't enough memory available.

Try enabling OPT_CHDK_EXMEM_TESTING (and disable OPT_CHDK_IN_EXMEM).
Enable superfine and see if you get any EXMEM corruption. If so adjust EXMEM size.

You may need to disable the superfine option override - in my opinion it's of very little value anyway. Canon's fine mode is indistinguishable from superfine under normal viewing conditions. You can see some tests I did using the SX30 here - http://forums.dpreview.com/forums/read.asp?forum=1010&message=37238416

Another option (that I've toyed with; but not implemented fully) is to load CHDK into EXMEM using just enough EXMEM memory for CHDK itself; but then use the normal Canon heap memory for malloc/free.
We would then have four scenarios:
- no EXMEM, CHDK loaded in low memory and uses Canon heap in low memory
- CHDK loaded in low memory; but CHDK heap in EXMEM memory
- CHDK loaded in EXMEM, and CHDK heap in EXMEM
- CHDK loaded in EXMEM, CHDK heap using Canon heap in low memory

Phil.
Logged
CHDK ports:
  sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
  g12 (1.00c, 1.00e, 1.00f & 1.00g)
  sx130is (1.01d & 1.01f)
  ixus310hs (1.00a & 1.01a)
  sx40hs (1.00d, 1.00g & 1.00i)
  g1x (1.00e, 1.00f & 1.00g)
  g5x (1.00c, 1.01a, 1.01b)
  g7x2 (1.01a, 1.01b, 1.10b)
*

Offline funnel

  • ****
  • 349
Re: experimental alternate memory allocation
« Reply #113 on: 10 / June / 2011, 03:02:58 »
I did a thorough exmem_testing and no corruption was found up to 2MB. Now I went for the second option - CHDK loaded in low memory; but CHDK heap in EXMEM memory. I allocated just 256KB to be on the safe side, at 512KB seemed fine too. I never thought about the fourth option. Good to know, thanks.
Logged
*

Offline ADamb

  • **
  • 65
  • sx200is
Re: experimental alternate memory allocation
« Reply #114 on: 09 / May / 2012, 14:49:16 »
Support for sx200is 100c.
Not throughly tested, but should be OK.
Logged
SX200 IS-100C
*

Offline waterwingz

  • ******
  • 12140
    • Waterwingz
Re: experimental alternate memory allocation
« Reply #115 on: 09 / May / 2012, 20:30:05 »
Quote from: ADamb on 09 / May / 2012, 14:49:16
Support for sx200is 100c.
Not throughly tested, but should be OK.
Not sure what somebody is supposed to do with the two files you posted ?   Are you proposing this as a change to the svn repository  /  autobuild server ?   If so,  the normal process is to submit a patch file here : http://chdk.setepontos.com/index.php?topic=650.0 .  If you need help with preparing a suitable patch file, just ask.

Logged
Ported :   A1200    SD940   G10    Powershot N    G16
*

Offline waterwingz

  • ******
  • 12140
    • Waterwingz
Re: experimental alternate memory allocation
« Reply #116 on: 27 / August / 2012, 20:52:06 »
I did some experimenting today with EXMEM and am looking for some advice. 

Caveat:  I set XMEM_BUFFER_SIZE=0x80000 for these experiments.  I have not done any OPT_EXMEM_TESTING - not even sure if that's still needed ?  (gulp)

In the current CHDK trunk, it seems that every port that uses EXMEM to get around low memory issues enables both of these overrides in platform/camera/sub/fw/makefile.inc  :

Code: [Select]
override OPT_EXMEM_MALLOC=1
override OPT_CHDK_IN_EXMEM=1

However, on a cheap low memory camera (A1200), my simple experiments shows :

 OPT_EXMEM_MALLOC=0     OPT_CHDK_IN_EXMEM=0    free memory =  122K
 OPT_EXMEM_MALLOC=1     OPT_CHDK_IN_EXMEM=1    free memory =  291K
 OPT_EXMEM_MALLOC=0     OPT_CHDK_IN_EXMEM=1    free memory =  399K
 OPT_EXMEM_MALLOC=1     OPT_CHDK_IN_EXMEM=0    free memory =  517K


So in a low memory camera, enabling both options doubles free memory.  But  splitting where CHDK loads from where CHDK allocates its memory makes a big difference in the free memory by almost 400%.

Other posts (recently the sx130/150) show that putting things in EXMEM cuts the buffer size available for rapid shooting.  But other than that .. why do more cameras not spit where CHDK goes from where it puts its heap ?

Or what else do I need to consider here ?  This could be very important to other "low memory" cameras like the A800 / A810.



Logged
Ported :   A1200    SD940   G10    Powershot N    G16
*

Online reyalp

  • ******
  • 14140
Re: experimental alternate memory allocation
« Reply #117 on: 27 / August / 2012, 21:05:36 »
How are you measuring "free memory" ?

Assuming you are talking about the Canon malloc heap, then something is wrong either in your implementation or measurement. Without CHDK in exmem, the CHDK binary takes away from the camera heap. Your numbers appear to show the reverse.

edit:
Note if you are not using CAM_FIRMWARE_MEMINFO method, you are getting the largest free block, meaning 1 unlucky fragmentation could skew your results a lot.

I suggest implementing  CAM_FIRMWARE_MEMINFO  (should be found fine by sig finder, on by default in 1.2 for dryos) and using chdkptp:
=return get_meminfo('system')
=return get_meminfo('exmem')

edit for posterity:
 the missing not made this confusing.
« Last Edit: 27 / August / 2012, 21:52:47 by reyalp »
Logged
Don't forget what the H stands for.
*

Offline waterwingz

  • ******
  • 12140
    • Waterwingz
Re: experimental alternate memory allocation
« Reply #118 on: 27 / August / 2012, 21:09:52 »
Quote from: reyalp on 27 / August / 2012, 21:05:36
How are you measuring "free memory" ?
The numbers are from the Miscellaneous Stuff -> Show Memory Info menu.

I played with
#define CAM_FIRMWARE_MEMINFO  1
but didn't notice a change (not that I repeated every test every way though). It was on for these tests.

If I interpret the results correctly,  the best scenerio is CHDK in low memory and CHDK allocating memory in high memory ?   Although I could be totally confuse about the semantics of those #defines.

Update : incidentally,  I started this when my A1200 port would not load the LARGE badpixel.bin file for the elcheapo sensor (it would load a smaller file from another camera).  With the options that seem to free up more memory, I can now use badpixel.bin successfully.  And from what philmoz tells me .. I need a large contiguous block to load that file !


« Last Edit: 27 / August / 2012, 21:17:32 by waterwingz »
Logged
Ported :   A1200    SD940   G10    Powershot N    G16
*

Offline philmoz

  • *****
  • 3450
    • Photos
Re: experimental alternate memory allocation
« Reply #119 on: 27 / August / 2012, 21:43:29 »
Don't have time for a detailed reply just now - look at the wiki page for exmem and the diagrams that show how the memory is mapped.

I've added some comments to the options below:

 OPT_EXMEM_MALLOC=0     OPT_CHDK_IN_EXMEM=0    free memory =  122K (1)
 OPT_EXMEM_MALLOC=1     OPT_CHDK_IN_EXMEM=1    free memory =  291K (2)
 OPT_EXMEM_MALLOC=0     OPT_CHDK_IN_EXMEM=1    free memory =  399K (3)
 OPT_EXMEM_MALLOC=1     OPT_CHDK_IN_EXMEM=0    free memory =  517K (4)


(1)
This is using the Canon heap only and 'free memory' is showing the size of the largest free block in the Canon heap.

(2)
The Canon heap is mostly untouched in this scenario, CHDK is loaded in EXMEM and uses EXMEM for it's own heap. The 'free memory' value is the largest free block in the EXMEM heap managed by CHDK.

(3)
Don't do this - very very bad
Here you are loading CHDK into EXMEM without first allocating the EXMEM buffer from the firmware - if any firmware function allocates EXMEM it could overwrite CHDK.
(I need to check for this scenario in the makefile and throw an error.)

(4)
Mixed mode. CHDK loaded into the Canon heap so the firmware loses the heap memory used by CHDK. CHDK heap is in EXMEM - the 'free memory' is the largest block in the CHDK EXMEM heap.

You'll need to experiment to see what works best - and yes OPT_EXMEM_TESTING should still be done to see if there are any video buffers that might get corrupted by CHDK EXMEM.

Phil.

Logged
CHDK ports:
  sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
  g12 (1.00c, 1.00e, 1.00f & 1.00g)
  sx130is (1.01d & 1.01f)
  ixus310hs (1.00a & 1.01a)
  sx40hs (1.00d, 1.00g & 1.00i)
  g1x (1.00e, 1.00f & 1.00g)
  g5x (1.00c, 1.01a, 1.01b)
  g7x2 (1.01a, 1.01b, 1.10b)
Pages: 1 ... 7 8 9 10 11 [12] 13   Go Up
« previous next »
 

Related Topics


SimplePortal © 2008-2014, SimplePortal