Responding to Microfunguys post from
http://chdk.setepontos.com/index.php/topic,4465.msg42798.html#msg42798This is something I am not clear about ..
Does malloc reserve memory in the memory cache and a copy in regular memory and write to both ?
If so, can the cached memory be 'out-of-date' and how would you know ?
When would you use uncached memory allocation ?
This is pretty far from the OPs question, so I'm replying in a more appropriate thread.
First, some backgroundCamera has two views of main memory. These address spaces refer to the
same physical memory. The differences is how the CPU access it.
0 through <memory size> has the cache and write buffer bits enabled.
<uncached start address> through <uncached start + memory size> has the cache and write buffer bits disabled.
The uncached start address is defined in CHDK with CAM_UNCACHED_BIT. Memory sizes vary between 16 and 128 megs or so. The values for a specific camera can be found from the CP15 instructions at the very start of the firmware.
If the CPU reads via an address in the cached range, it will check the cache first, and will copy the value from main memory to cache if needed. If it writes a value, the value will be written to the cache, and only updated in main memory later. This is normally completely transparent from the programmers POV. There's also an independent instruction cache, but we'll ignore that for now.
The specific rules are described in the ARM documentation (see ARM946E-S Technical Reference Manual, chapter 3 "Caches", and chapter 6, section 5 "The write buffer"). Mostly we can ignore the details, but the rules concerning coherency are important:
3.3.4 Data cache validity
...
The ARM946E-S processor does not support external memory snooping. Any shared data memory space therefore, must not be cachable.
In other words, if an external device modified memory, the CPU won't know. If the CPU happens to have a value for this address in cache, it will happily use that old value. Similarly, if the CPU writes to an address in the cached range, external devices may not see that value until it has made its way through the cache and write buffer.
Since there are a bunch of external devices that are directly tied to main memory (the sensor frame buffer, the display frame buffers) the canon firmware uses the uncached range to interact with them. It appears that some lower level IO functions (read(), write()) also use DMA, and so only work reliably on uncached addresses.
CHDK actually breaks this rule with the "raw buffer cached" option for DNG, accessing the raw buffer via the cached range. This is technically wrong (we should flush before we start and flush and drain write buffer when done, or something like that) but appears to work. The data volume is so much larger than the cache (4k on most or all CHDK cameras) that most of it will have been flushed out anyway.
OK that's very exciting, but what should I use where ?
- All normal operations should be on cached memory. You don't have to manage the caching, unless you are dealing with external hardware that maps directly to main memory. All the CHDK code and variables is accessed through the cached range.
- malloc()/free() operate on addresses in the cached ranged. This should be used for any dynamically allocated data, except the following:
- umalloc()/ufree() operate on uncached addresses. Use them if and only if that memory has to be passed to a low level function that expects uncached memory. Generally in CHDK the only functions with this requirement are read() and write().
Some of this is also covered in
http://chdk.wikia.com/wiki/CHDK_Coding_Guidelines
