SD info and benchmarks (was Re: EOS M3 porting)

This is the response from SD card to some commands. We need CMD41:

Thanks, I'll take a look at this when I have some time.

And Upgrader.bin works the same?

As far as we know, there's no way to automatically load a binary in a way that doesn't trash the running OS. It's possible something has been missed, but it's my impression that everything in the normal firmware goes though essentially the same codepath. There is separate code in the romstarter, but this is not run without hardware or ROM modifications.

It's possible to inject code into the running OS using canon basic, but this requires pressing set and happens long after everything is started.

Quote

But I'm not sure about this solution: SD power is switched off but the card still can get power via data lines...

Yes, that's a concern, the SD spec specifically mentions that "DAT, CMD, and CLK should be disconnected or driven
to logical 0 by the host" when the card is powered down.
Perhaps there's a more proper version of SD power control somewhere that also drives the above mentioned pins to a low state?

Do we know the SetSDPwrPort function doesn't already do this? It kinda seems like the reason Canon would have a function like this would be to reset the card, but OTOH, the behavior is somewhat confusing: https://chdk.setepontos.com/index.php?topic=5563.0

Also, just FWIW, there's a ChangeSDPortVoltage function.

Do we know the SetSDPwrPort function doesn't already do this?

No. On M3 it switches only GPIO 6.
But there is another function 0x010E16C8 called from SD1stInit task that changes the state of six GPIO ports including GPIO 6.

On M3 it switches only GPIO 6.

On the M10 also.

But there is another function 0x010E16C8 called from SD1stInit task that changes the state of six GPIO ports including GPIO 6.

The first five GPIOs are the same in the 3 cameras I checked (M3, M10, SX280). I found them referenced in another routine, where they are set to an unusual value (0x8a0040), not used for regular GPIO pins like the ones with LEDs. I find it a bit odd that SD I/O pins are available as regular GPIOs, in a specialized HW like the DIGIC.

Did not test the new routine yet, though.

edit:
Now I have tested the new routine (0x10e173c for M10), same behaviour as with the simple SetSDPwrPort (fast boot, UHS active). It needs a running OS (calls SleepTask), so I called it from Startup task.

It worked, but card speed remained below 21MB/s. On this camera, startup times increased significantly when I left the SD powered off.

In the previous thread on SD power, I noticed that turning it off and then trying file access worked, after a quite long delay. My guess is that something errors out and resets, but from G7X it seems like the reset doesn't include getting back to high speed mode. It wouldn't be surprising for the firmware to stay on a slow mode once an error has been encountered, although it could easily be on the card side too.

Now I have tested the new routine (0x10e173c for M10), same behaviour as with the simple SetSDPwrPort (fast boot, UHS active). It needs a running OS (calls SleepTask), so I called it from Startup task.

I tried this on g7x (100d sub_010e638b), called where diskboot start code used to be. Using the wintec-2012 card SD info function shows 1.8V Support, and benchmarks are much improved (writes around ~32 MB/s, though I did have one run around 20)

I notice that in the Canon code (sub_010e647c, SdPower.c), this function is called based on the return value of sub_010e659a.

In the previous thread on SD power, I noticed that turning it off and then trying file access worked, after a quite long delay. My guess is that something errors out and resets, but from G7X it seems like the reset doesn't include getting back to high speed mode. It wouldn't be surprising for the firmware to stay on a slow mode once an error has been encountered

That would be my opinion (too). Those low speeds (around 10 MB/s) seem very much like the "default speed mode" mentioned in the SD spec (which all cards should support, I guess).

Quote

Now I have tested the new routine (0x10e173c for M10), same behaviour as with the simple SetSDPwrPort (fast boot, UHS active). It needs a running OS (calls SleepTask), so I called it from Startup task.

I tried this on g7x (100d sub_010e638b), called where diskboot start code used to be. Using the wintec-2012 card SD info function shows 1.8V Support, and benchmarks are much improved (writes around ~32 MB/s, though I did have one run around 20)

If this routine is indeed doing what we think it's doing (lowering SD signal lines followed by removing SD power), calling it during boot should have the potential of clearing up much or all our card problems (on previous DIGICs too). Setting GPIOs to 0x4c0003 does look like commanding them to L logical level output. 0x8a0040 might then be something like "assign GPIO to the SD hw"...? Too bad that I can't check what really happens on the SD lines.

I notice that in the Canon code (sub_010e647c, SdPower.c), this function is called based on the return value of sub_010e659a.

I did not check further yet.

I'm considering releasing test builds with this "card off during boot" mod. It doesn't seem like it could be harmful.

The sx280 probably needs more testing, to determine whether it is capable of UHS speed (the hw obviously is).

If this routine is indeed doing what we think it's doing (lowering SD signal lines followed by removing SD power), calling it during boot should have the potential of clearing up much or all our card problems (on previous DIGICs too).

Agreed, it seems like it might have some connection with https://chdk.setepontos.com/index.php?topic=12795.0

I'm considering releasing test builds with this "card off during boot" mod. It doesn't seem like it could be harmful.

Agreed with this too, I was thinking about adding it to the G7X builds.

The sx280 probably needs more testing, to determine whether it is capable of UHS speed (the hw obviously is).

Hmm... it should be possible to load a simple benchmark binary using canon basic. Writing it position independent might be annoying, but you could probably find some small fixed piece of memory to use, maybe the top of ARAM?

edit:
I tested a few more cards with the fix, and the results are confusing.

team-16g
RAW  5-7 MB/s   (previously was 6-11, but the 11 was only seen once)
MEM 3-6 MB/s (previously 4-15, 15 was only seen once )
Read speeds are improved: consistently 26 MB/s vs previous 16.

sandisk-16g
RAW 13-16 MB/s (previously 9-18)
MEM 7-16 MB/s (previously 7-13)
Read speeds are again improved, consistently 32 MB/s vs 18 before.

Read speed on the wintec card was also improved, consistently 32 MB/s vs 10-18 before.

All of the cards show 1.8v in the sd info function.

I also feel like the boot time is slower, on both cards that benchmark faster and the ones that don't.

 

edit:
I will try to make a benchmark that can be run without CHDK.

Too bad that I can't check what really happens on the SD lines.

I did it using the multimeter:
SetSDPwrPort function sets to 0.1v sd power pin, but there is 0.7v on DAT0-DAT3 and CMD pins.
0x010E16C8 function sets all sd pins to 0v.
It's strange that calling 0x010E16C8 function in the beginning of task_Startup has no affect.

IIUC there should be enough 1ms to reset the card:

I will try to make a benchmark that can be run without CHDK.

I think the benchmark should read/write at least 0.5GB to get more stable results.

I will try to make a benchmark that can be run without CHDK.

You can find my AdditionAgent based examples (they include some macros and build instructions), probably better via google here's the latest. As far as I remember, the upper (half?) part of the AdditionAgentRAM area is affected by CBasic.
edit: The LCDMsg functions or an LED can be used to provide feedback.

@Ant
Can you measure the CLK pin too? That is likely not included in the 5 GPIOs (but it's host driven, so should be at L logical level, I guess).

It's strange that calling 0x010E16C8 function in the beginning of task_Startup has no affect.

According to my M10 notes, hw (including GPIOs) is initialized at the start of Startup task. The diskboot-related routine's location should be safe.

Another idea: perhaps we could try executing the diskboot-related routine (after powering down the card), only removing the part that actually loads the diskboot...?

but it's host driven, so should be at L logical level, I guess

You are right. Normally it's at L logical level.

Another idea: perhaps we could try executing the diskboot-related routine (after powering down the card), only removing the part that actually loads the diskboot...?

I was trying to comment only sub_fc063804: errors and no UHS...

Here's a CHDK-less SD benchmark.

The top of the extend.m must be edited with the correct addresses for Open, Write, Close and GetSystemTime (all should be available in funcs_by_*.csv)

The SD info function can optionally be set. If not known, it should be set to 0.

Make sure the card is prepared as a script disk
If running under CHDK, remember to use a build without ARAM enabled
Put extend.m and cbbench.bin on the card
Start the camera and press set

The results are logged to cbbench.log, using UART redirection. If you are running CHDK the console output hooked, this won't work. The log is appended each run.

The script attempts 8 runs of 128 MB each, so it may take some time (e.g >100 sec at 10 MB/s). It tries to use LCDMsg to display, but on g7x I only see the message switching to record mode after the script is done.

It will leave a 128 MB file called cbbench.dat on the card.

I've only used it on G7X so far, but I think it should theoretically work on recent non-digic 6 cameras edit: Runs fine on elph130 too, without sdinfo. The binary is position independent ARM, and gets all entry points as function arguments. It assumes the start of uncached memory is at 0x40000000, and does 16 8MB writes to a file from that address each time it's called. The time measured is for the write() calls, without the open and close.


Results
I tested under CHDK with the SD reset hack and without CHDK. In all cases, the results with and without were essentially identical. The results were also broadly similar to CHDK benchmark results.

wintec-2012 ~31 MB/s (edit: as a sanity check, I tried this card using CHDK without SD reset. Result was ~20 MB/s as expected.)
patriot-32g-2015  ~25 MB/s
team-16g ~10 MB/s
sandisk-16g  ~9 MB/s

So, the [admin: avoid swearing please] performance of the last two allegedly UHS1 cards appears to have nothing to do with CHDK.

Other stuff:

I made the loadcbbench.lua file in the zip for development. It can be used to load the bin with chdkptp rather than Canon basic. Usage:
=<loadcbbench.lua
the load address is printed

To run, use
.return call_func_ptr(<load address>,<Open address>,<Write address>,<Close address>,<GetSystemTime address>)
wait for it to finish
getm
to get the elapsed time. You can run multiple times without reloading. "." is used instead of "=" to avoid polling while the benchmark runs.

edit:
updated 2017-05-09 to fix some typos in the cache clearing code