experimental alternate memory allocation

waterwingz

12140

Re: experimental alternate memory allocation

« Reply #120 on: 27 / August / 2012, 21:54:25 »

Advertisements

Quote from: philmoz on 27 / August / 2012, 21:43:29

(4) Mixed mode. CHDK loaded into the Canon heap so the firmware loses the heap memory used by CHDK. CHDK heap is in EXMEM - the 'free memory' is the largest block in the CHDK EXMEM heap.

Seems like the best of both worlds ? Better than asking all of CHDK to run from the Canon heap (assuming that it can technically do so). And it gives CHDK access to more exmem heap for things it needs to do. I wonder how many cams would run better this way ?

Quote

You'll need to experiment to see what works best - and yes OPT_EXMEM_TESTING should still be done to see if there are any video buffers that might get corrupted by CHDK EXMEM.

Thanks - now that I know I'm not going down a "rabbit hole" I'll continue to pursue this.

Logged

Ported : A1200 SD940 G10 Powershot N G16

reyalp

14109

Re: experimental alternate memory allocation

« Reply #121 on: 27 / August / 2012, 21:59:09 »

I think Phil explained it, but since I went and measured D10 anyway

chdkptp -c -e"exec return con:execwait([[return get_meminfo('system'),get_meminfo('exmem')]])"
1) no emxem
system free_size=366104
exmem N/A
2) exmem malloc, chdk not in exemem
system free_size=423808
exmem free_size=3090560
3) exmem malloc, chdk in exmem
system free_size=684032
exemem free_size=2830320

Logged

Don't forget what the H stands for.

reyalp

14109

Re: experimental alternate memory allocation

« Reply #122 on: 27 / August / 2012, 22:04:17 »

Quote from: waterwingz on 27 / August / 2012, 21:54:25

Better than asking all of CHDK to run from the Canon heap (assuming that it can technically do so). And it gives CHDK access to more exmem heap for things it needs to do.

It takes a CHDK binary size away from the Canon heap (note, with modules this is less significant, since they get malloc'd into whatever the CHDK malloc heap is). Hard to say whether that is "better" than taking the same amount away from exmem. I'd want some actual data one way or the other before changing anything.

Logged

Don't forget what the H stands for.

waterwingz

12140

Re: experimental alternate memory allocation

« Reply #123 on: 27 / August / 2012, 22:07:14 »

Quote from: reyalp on 27 / August / 2012, 21:59:09

I think Phil explained it, but since I went and measured D10 anyway

chdkptp -c -e"exec return con:execwait([[return get_meminfo('system'),get_meminfo('exmem')]])"
1) no emxem
system free_size=366104
exmem N/A
2) exmem malloc, chdk not in exemem
system free_size=423808
exmem free_size=3090560
3) exmem malloc, chdk in exmem
system free_size=684032
exemem free_size=2830320

Thanks - must be nice to have a camera with that much free exmem ! I'll take a look with chdkptp at how much the A1200 reports with the same tests.

Meanwhile, how much can I shave off the EXMEM_BUFFER_SIZE used in the MEMISOSTART calculation. As this is hard coded, getting too close to the current executable size would be dangerous going forware ( especially once 1.2.0 kicks in) .. but what I'm wondering is how to account for the loadable modules ? Presumably the CHDK size reported by Show Memory Info does not include those ?

Quote from: reyalp on 27 / August / 2012, 22:04:17

I'd want some actual data one way or the other before changing anything.

Data showing what ? How do you define "better" ? DNG badpixel.bin seems to only works with split allocations - does that make it better - at least for crappy cams like the A1200 ?

« Last Edit: 27 / August / 2012, 22:13:10 by waterwingz »

Logged

Ported : A1200 SD940 G10 Powershot N G16

reyalp

14109

Re: experimental alternate memory allocation

« Reply #124 on: 27 / August / 2012, 22:20:00 »

Quote from: waterwingz on 27 / August / 2012, 22:07:14

Thanks - must be nice to have a camera with that much free exmem !

Actually, I normally use case 1 and haven't noticed any problems...

Quote

Meanwhile, how much can I shave off the EXMEM_BUFFER_SIZE used in the MEMISOSTART calculation. As this is hard coded,

EXMEM_BUFFER_SIZE is the total amount of EXMEM you want to allocate.

MEMISOSTART under EXMEM is just the start of EXMEM. The CHDK binary size is accounted for when suba is initialized, giving whatever is left to heap.

Quote

Presumably the CHDK size reported by Show Memory Info does not include those ?

It just gives the text + data + BSS

Quote

Data showing what ? How do you define "better" ?

Data showing there is a problem being solved or a measurable performance gain.

Logged

Don't forget what the H stands for.

waterwingz

12140

Re: experimental alternate memory allocation

« Reply #125 on: 27 / August / 2012, 23:14:29 »

Quote from: reyalp on 27 / August / 2012, 22:20:00

EXMEM_BUFFER_SIZE is the total amount of EXMEM you want to allocate.

Hmm .. so I can increase the number in everything but case 1 at the expense of starving the camera for exmem if it needs it. Or I can reduce the size until I hit the code size + space needed for modules + min CHD heap size needed. Depending on whether I'm in case 2,3 or 4.

Logged

Ported : A1200 SD940 G10 Powershot N G16

reyalp

14109

Re: experimental alternate memory allocation

« Reply #126 on: 27 / August / 2012, 23:49:46 »

Quote from: waterwingz on 27 / August / 2012, 23:14:29

Quote from: reyalp on 27 / August / 2012, 22:20:00
EXMEM_BUFFER_SIZE is the total amount of EXMEM you want to allocate.
Hmm .. so I can increase the number in everything but case 1 at the expense of starving the camera for exmem if it needs it. Or I can reduce the size until I hit the code size + space needed for modules + min CHD heap size needed. Depending on whether I'm in case 2,3 or 4.

Right. For CHDK, once you have enough, more isn't really better, it's not like we do swapping or something when memory gets low. Pre-exmem, CHDK never had more than ~1mb RAM free at best. 700k would be more typical, and that was shared with the Canon firmware. So allocating 1 meg to exmem and putting CHDK (~250k) in exmem should be fine.

It's not clear to me what impact there is from taking away exmem from the factory firmware, but the ports that have it hardcoded (most of Phils at least) seem to do OK.

Logged

Don't forget what the H stands for.

waterwingz

12140

Re: experimental alternate memory allocation

« Reply #127 on: 28 / August / 2012, 00:01:45 »

Quote from: reyalp on 27 / August / 2012, 23:49:46

Right. For CHDK, once you have enough, more isn't really better, it's not like we do swapping or something when memory gets low. Pre-exmem, CHDK never had more than ~1mb RAM free at best. 700k would be more typical, and that was shared with the Canon firmware. So allocating 1 meg to exmem and putting CHDK (~250k) in exmem should be fine.

It's not clear to me what impact there is from taking away exmem from the factory firmware, but the ports that have it hardcoded (most of Phils at least) seem to do OK.

For now I have to deal with a really large badpixel.bin file that won't load without some exmem action if I want to support DNG1.1. And the only trade off I've seen is the whole Ralph Phraner wanting to shoot really fast with the sx150 and needing an non-EXMEM stripped version to do that.

Putting CHDK in low memory and CHDK's heap in high memory seems like a good compromise.

« Last Edit: 28 / August / 2012, 00:06:04 by waterwingz »

Logged

Ported : A1200 SD940 G10 Powershot N G16

srsa_4c

4451

Re: experimental alternate memory allocation

« Reply #128 on: 01 / April / 2013, 14:03:37 »

I have a question.
Has anybody tried multiple exmem allocations at startup? Since exmem is allocated downwards starting from the end of the exmem pool, what would the following scenario do:
- allocate exmem, amount: HEAP_SKIP
- allocate exmem, amount: EXMEM_BUFFER_SIZE
- free the first allocation
?
~~No, I haven't tried this yet.~~ Exmem has no conflicts on my Ixus110 (HEAP_SKIP is 0).

After trying, it seems that once I free that first allocated block, my subsequent allocations don't touch that area again. This is on a camera where HEAP_SKIP is needed. This may not help us too much...
~~It would be great if somebody with a camera that needs HEAP_SKIP and has problems with JPEG size would test it...~~

I have analysed some parts of the exmem allocator routine, and found that it will not allocate from a free exmem area, if there's an active allocation below it. So, the above scenario won't bring any improvement. Too bad...

« Last Edit: 05 / April / 2013, 12:10:18 by srsa_4c »

Logged