experimental alternate memory allocation

reyalp

14111

« on: 02 / January / 2011, 05:04:44 »

Advertisements

Based some some ideas here http://chdk.setepontos.com/index.php?topic=5929.0 I've come up with an alternate way to get memory in CHDK.

Changset 1015

This is very experimental. I've booted the camera, taken a picture, a few seconds of video, and run a couple lua scripts, but that's it. Possible risks and side effects are unknown.

This is controlled by the compile time option OPT_EXMEM_MALLOC

To use this, you must also find the function I called _exmem_alloc and add it to stubs_entry_2.S This function can be found on recent cameras by finding ExMem.AllocCacheable, and finding the first real (more than a few instructions) function that calls.

This only affects regular malloc currently, umalloc still comes from the original pool. You can set the amount of memory used in wrappers.c. You can also set which ExMem pool is used there. How much memory you can allocate isn't clear. I ran it up to 16mb, and the camera booted, but I didn't really do any testing. The default is current 2mb, which is ~2x larger than any normal CHDK camera has.

This will only work on dryos without additional hacking.

The lua script attached will record ExMem info to EXMEMINF.LOG. Native calls must be enabled.

edit:
Big thanx to mx3, the sub-allocator he found for the elf branch was just what I needed.

exmeminf.lua (0.32 kB - downloaded 165 times.)

« Last Edit: 02 / January / 2011, 14:27:27 by reyalp »

Logged

Don't forget what the H stands for.

philmoz

3450

Re: experimental alternate memory allocation

« Reply #1 on: 02 / January / 2011, 05:28:07 »

Nice, compiles and runs on SX30 - will do some testing to see how it goes.

One thing, in wrappers.c you have:

Code: [Select]

void free(void *p) {
	if(exmem_heap)
		suba_free(exmem_heap,p);
	else
		free(p);
}

Should the call to free(p) in here be _free(p) to avoid a recursive loop?

Phil.

Logged

CHDK ports:
sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
g12 (1.00c, 1.00e, 1.00f & 1.00g)
sx130is (1.01d & 1.01f)
ixus310hs (1.00a & 1.01a)
sx40hs (1.00d, 1.00g & 1.00i)
g1x (1.00e, 1.00f & 1.00g)
g5x (1.00c, 1.01a, 1.01b)
g7x2 (1.01a, 1.01b, 1.10b)

reyalp

14111

Re: experimental alternate memory allocation

« Reply #2 on: 02 / January / 2011, 05:35:43 »

Quote from: philmoz on 02 / January / 2011, 05:28:07

Nice, compiles and runs on SX30 - will do some testing to see how it goes.

One thing, in wrappers.c you have:
Code: [Select]
void free(void *p) { if(exmem_heap) suba_free(exmem_heap,p); else free(p); }
Should the call to free(p) in here be _free(p) to avoid a recursive loop?

Phil.

Probably

Really it should panic or something if exmem_heap doesn't get initialized.

Logged

Don't forget what the H stands for.

Bernd R

430

Re: experimental alternate memory allocation

« Reply #3 on: 02 / January / 2011, 06:52:46 »

great if that can work stable, i use chdk shell maybe its possible to add OPT_EXMEM_MALLOC here too for more easy test.

i think to do a worst case test use the handheld nightscene 10 megapixel(on IX1000 HS) mode.here the camera shoot 5 images in series and mix them together.

this 5 shoots work also if there is no SD Card in(so its sure this feature use internal RAM), but of course after the processing image bar is gone, and the image is show in preview, the image is not store.

I think when the handheld nightscene mode work ok when chdk alloc 16 megabyte of mem, then this memory is really free available to chdk.

there is also a best picture mode that shoot 5 images and choose the sharpest then, but it use only 2.5 megapixel, so i think it use not so much memory.but this mode can too test if it work ok

« Last Edit: 02 / January / 2011, 06:57:37 by Bernd R »

Logged

Ixus 1000 HS

philmoz

3450

Re: experimental alternate memory allocation

« Reply #4 on: 02 / January / 2011, 07:21:30 »

I've hit one problem on both the SX30 and G12.
Recording a movie results in a corrupt movie file, and after stopping the recording the CHDK menu font is also corrupted.

If I disable OPT_EXMEM_ALLOC movie recording works fine on both camera.

Regards,
Phil.

Logged

CHDK ports:
sx30is (1.00c, 1.00h, 1.00l, 1.00n & 1.00p)
g12 (1.00c, 1.00e, 1.00f & 1.00g)
sx130is (1.01d & 1.01f)
ixus310hs (1.00a & 1.01a)
sx40hs (1.00d, 1.00g & 1.00i)
g1x (1.00e, 1.00f & 1.00g)
g5x (1.00c, 1.01a, 1.01b)
g7x2 (1.01a, 1.01b, 1.10b)

Bernd R

430

Re: experimental alternate memory allocation

« Reply #5 on: 02 / January / 2011, 11:16:03 »

@reyalp
>To use this, you must also find the function I called _exmem_alloc and add it to stubs_entry_2.S This >function can be found on recent cameras by finding ExMem.AllocCacheableMemory, and finding the first >real (more than a few instructions) function that calls.

Can you explain that more, i search whole chdk dir (which contain asm txt source of firmware) but the string AllocCacheableMemory(i copy with CTRL+c from your post) is not find in IX1000 Firmware.but IX 1000 is release in same time as sx30 and G12.

Logged

Ixus 1000 HS

asm1989

527
SX720, SX260, SX210 & SX200

Re: experimental alternate memory allocation

« Reply #6 on: 02 / January / 2011, 12:19:57 »

@Bernd R,
for ixus1000 100D ExMem.AllocCacheableMemory is at FF890950

Logged

waldo

238

Re: experimental alternate memory allocation

« Reply #7 on: 02 / January / 2011, 12:37:35 »

Is there a way to get a list of currently allocated memory blocks?

Logged

reyalp

14111

Re: experimental alternate memory allocation

« Reply #8 on: 02 / January / 2011, 13:19:52 »

Quote from: philmoz on 02 / January / 2011, 07:21:30

I've hit one problem on both the SX30 and G12.
Recording a movie results in a corrupt movie file, and after stopping the recording the CHDK menu font is also corrupted.

If I disable OPT_EXMEM_ALLOC movie recording works fine on both camera.

Regards,
Phil.

Interesting. Have you tried using a different pool for the heap, say EXMEM_SCRIPT instead of EXMEM_RAMDISK ?

Have you tried a smaller size ? Even 500kb would be better than some of the new cameras, and at least the cameras normal heap wouldn't run out so unpredictably. On the other hand, maybe going larger will make the problem more obvious...

On thing I realized is if there is anything in CHDK that frees memory returned by firmware functions, or passes pointers to the firmware that are freed, heap corruption will result. I'll have a look and try to figure out if we do this anywhere.

Did you check with ExMem.View (with the lua script above) to see if it really allocated 2mb for EXMEM_RAMDISK ? Maybe check while recording a move too, so you can see what that is using.

Quote from: waldo on 02 / January / 2011, 12:37:35

Is there a way to get a list of currently allocated memory blocks?

You mean each thing that was malloc'd ? You can probably hack something into suba.c

Quote from: asm1989 on 02 / January / 2011, 12:19:57

@Bernd R,
for ixus1000 100D ExMem.AllocCacheableMemory is at FF890950

Just to be clear this isn't the function you want. It's a signpost to the function you want. _exmem_alloc is FF8908E8 on this camera.

Logged

Don't forget what the H stands for.

Bernd R

430

Re: experimental alternate memory allocation

« Reply #9 on: 02 / January / 2011, 14:25:58 »

ok thanks i see now func is named as

ExMem.AllocCacheable and not ExMem.AllocCacheableMemory as stand in your intro post.

I test that tomorrow what happen on IX1000.old mem is only 512 kb, and when have 1 megabyte is more than enough for next years.

only not good is that canon firmware trash mem.normaly when record video cam should show out of memory.

Logged

Ixus 1000 HS