CHDK Forum
Using CHDK => General Help and Assistance on using CHDK stable releases => Topic started by: Mlapse on 05 / May / 2019, 11:42:42
-
I tried
http://www.ridgecrop.demon.co.uk/index.htm?guiformat.htm
to format a 256GB sd card to FAT32
After placing the CHDK build on that card I've had an error after boot stating missing modules.
I reformatted back to exFAT and placed the same CHDK build on it: boots CHDK without error.
started over, same result.
Does anybody have a clue what I did wrong or what tool might be better suited to create a large FAT32 partition?
...it's to see if stick can make a dual boot with that size, but now it does not recognise the card
-
Sounds more like you failed to copy CHDK correctly to the card rather than a card format problem?
When you say "placing the CHDK build on that card" does that mean you expanded the zipped download file fully onto the card - with all the associated folders and subfolders created and filled with the files that are supposed to be in those folders?
Also, what camera are you using?
-
yes, same set of files and folders works when the card is formatted exFAT, but not when it's formatted FAT32...
both times CHDK seems to work abeit the 'missing modules' notion when in FAT32
done it twice to see if i did something wrong.
-
Again, what camera?
Also, are you trying to do this with dual partitions?
-
I did this with a 128GB Card for a while:
cmd (start prompt)
diskpart (new command window)
list disk (displays volumes)
select disk 2 (select the number depending on the drive, of course!)
clean
create partition primary
Now go back to the standard "cmd" command window
d: (change to drive where H2Format.exe is located)
cd temp (if the h2format.exe is located there)
h2format.exe (enter)
h2format h: 64 (then wait about 1minute)
https://www.heise.de/download/product/h2format-40825
-
It's my S95, that supports dual boot.
just now I tried formatting FAT32 with
https://rufus.ie/
and h2format...
same result, missing files when i boot via the firmware update method on the FAT32 formatted 256GB card.
working when same card is formatted exFAT
for now it is a single partition FAT32 with firmware update method.
-
It's my S95, that supports dual boot.
just now I tried formatting FAT32 with
https://rufus.ie/
same result, missing files when i boot via the firmware update method.
So do you have one partitions on the card or two?
-
After placing the CHDK build on that card I've had an error after boot stating missing modules.
When that happens, can you still take photos and review them?
-
yes i can, also the intervalometer script runs.
stick does not recognise it.
-
yes i can, also the intervalometer script runs.
You get the "missing modules" error, but can still run script?
Can you open the CHDK file browser?
-
if you mean to select a script, yes i can. it started with no script selected.
-
if you mean to select a script, yes i can. it started with no script selected.
I meant "Miscellaneous -> file browser" but script selection is the same thing.
This suggests there is some kind of timing/initialization error, because the file select module (CHDK/MODULES/FSELECT.FLT) is the one used for the modules check. CHDK checks once at startup whether it can access this file, as a sanity check for a correct install. If not, it shows the message.
-
It's my S95
Can you try another, older camera (for example a490 if you still have it)?
-
same with the a490,
error:missing modules
and no script loaded, should be, but i can load it and run it, take and see pictures.
-
same with the a490,
error:missing modules
and no script loaded, should be, but i can load it and run it, take and see pictures.
Looks like the firmware spends too much time initializing the file system. CHDK only waits up to 4 seconds for this to complete.
-
@Mlapse:
What Canon firmware does your s95 have?
edit:
Here's patch that reports startup time in the misc values area. It displays 3 values
T0: tick when spytask got to the start of the spytask_can_start wait
FS: when core_spytask_can_start() was called, when we think the filesystem is ready
ST: when spytask exited the can_start wait. Normally 10ms after the previous item
It seems surprising to me that an empty 256 GB FAT32 card would take > 4s to be ready, but I don't have one to try. Elph130 with a 32 GB card takes about 700ms, while elph180 with an 8 GB card takes ~200.
Another possibility is that our filesystem ready logic is failing somehow (so core_spytask_can_start is called before it actually should), although that failing based on card size would also surprise me.
-
when the card is formatted in FAT32 it takes noticeably more time than when formatted exFAT for the cam to be ready (orange led flashing at start).
it looks like 4-6 seconds...I will test the patch this afternoon, where should I place it, what to do with it?
the card is rated class10, U3, V30, 4K, A1 (up to: read:95MB/s, write:55MB/s, 1500 iops) so it should exceed whatever the S95 can use
i am testing it on a S95-100K, but have several versions lying around to crossreference results.
-
it looks like 4-6 seconds...I will test the patch this afternoon, where should I place it?
the card is rated class10, U3, v30, 4K, A1 (up to: read:95MB/s, write:55MB/s, 1500 iops)
That's useful information.
i am testing it on a S95-100K, but have several versions lying around to crossreference results.
Here's a test build with the patch applied for S95 100k. You can just unzip the files on the root of the card.
Please post the 3 numbers described in the previous post.
-
here are the results.
256GB/FAT32
t0:60
fs:5510
st:4170
so st comes in early
256GB/exFAT
t0:60
fs:860
st:860
fs=st
16gb class10/FAT32 for reference
t0:60
fs:650
st:660
my first conclusion is that rebooting with a cam that only support FAT16 or FAT32 card lock method is possible by increasing wait time.
but not really viable for 256GB or larger sd cards, because of that extra time needed....not only the 5sec. start is something to consider.
in my experience startup before buttons work (BUSY) also takes more time if pictures are present on the card...if that time should be included in total reboot time it will easy go over 5 minutes.
however a longer wait at boot might still have extra value if you prefer the autoboot to the firmware update method.
-
Thanks. That appears to confirm that it's just the Canon firmware taking a long time to be ready with large FAT32 cards. I wonder if this would be influence by cluster size or something, I would guess the time is reading/checking some FAT related structures.
I believe the current 4 second timeout is just what someone figured was longer than a properly functioning port should take, so there shouldn't be any particular issue increasing it.
Attached is a build with it set to 10s. Let me know if it resolves the problem. I'll check it in if it works and doesn't create any new problems.
-
@reyalp the error is gone when starting up, but it's a long wait. :D
to test the reboot function, i first have to make it a dual partition card.
since stick does not recognise the large sd card although it's formatted in FAT32, what are the other options?
the method of creating a dual boot card in camera does not work anymore because the resulting size of the first partition is too small..or is that fixed in CHDK1.5?
-
to test the reboot function, i first have to make it a dual partition card.
since stick does not recognise the large sd card although it's formatted in FAT32, what are the other options?
I'd suggest gparted under linux. There are various liveusb / livecd distros that provide this. You can make the card bootable in camera afterward.
Since windows 10 now has some support for multi-partition removable media, you might be able to do the partition there, either with CLI commands or though the computer management storage / disk management snapin.
the method of creating a dual boot card in camera does not work anymore because the resulting size of the first partition is too small..or is that fixed in CHDK1.5?
The partition is not too small, it only needs to contain DISKBOOT.BIN.
That said, the in camera partitioning has always seemed pretty dubious to me. You still need to *format* the partitions outside of the camera.
-
The partition is not too small, it only needs to contain DISKBOOT.BIN.
That said, the in camera partitioning has always seemed pretty dubious to me. You still need to *format* the partitions outside of the camera.
I have used it with succes up until 1.4,
the trick is as you say in the fact that you had to format in windows, so before windows could accept dual partition cards, you'd have to place the small build on partition one to be able to swap partitions in the camera and set up CHDK on partition 2.
With only 2Mb that was, and is, not enough for this method since CHDK1.4.
I'm guessing i'll go with the linux option.
Windows may recognise dual partition SD nowadays, but still can't make them.
-
With only 2Mb that was, and is, not enough for this method.
You should only need DISKBOOT.BIN or PS.FI2 to swap partitions.
-
thanks to @zeno excellent work I first tried Stick, but since it does not support large SD cards, this method does not work (yet).
luckely he does have this link to fat32format on his STICK page...and that helped me later.
thanks to @dmitrys I had another option: Chimp, that does support large cards, but it had to download a build instead of me pointing it to an existing one and then I got this error: 1 is not a supported code page.
Since i found no answers to what went wrong, I did not know how to solve this and so I tried something old.
I used CHDK to make a dual partition card.
Put it into a windows 10 system, since that does recognise both partitions there is no need to swap active partitions anymore.
Formatted the 2mb partition with windows, did the large partition with fat32format.
Unpacked the build reyalp made for me yesterday
Placed it back in the camera and tried to use the card lock method, failed to autoboot as I expected after a format in windows.
So then I tried CHDK to make it bootable, but that failed to autoboot too....something i did not expect.
I tried several times but to no avail.
So I fired up Chimp again, just to make the card bootable, you do not need to download a build for that.
worked first time as described on the tin.
And I now have a dual partition bootable large SD card...have to do some testing.
After that i will set up a rig to test the reboot somewhere close to 24 days.
It is really a shame that this dual boot method only works with FAT16+FAT32 and not FAT16+exFAT, but i'm guessing that is a Canon limitation.
Because as we discovered some posts back, the speed of the card has suffered dearly from this FAT32 conversion
-
best to continue this old thread i thought, since it is the same issue.
against my better judgement I picked up an even larger card for a next year project ;)
and have again that error at boot: missing modules.
PC test: exfat vs fat32 approx 10% slower at fat32...still above 90MB/s read 70 MB/s write
card is v30, class 10, U3, I.
i tried on the g1x 101a and s110 102b, both moved from the spash screen in a few seconds when it was still exfat.
after formatting to fat32 it took some 14 seconds on the s110 and G1X before the canon splash screen was over when started in play....before loading chdk, no card lock.
when starting regular with card lock, only very long press starts cam. so the lens extended and then i got the chdk splash... error:missing module
Reyalp, I thought you changed that timing to change with card size since we had that issue with the s95.
but maybe not dynamically enough :)
is it possible to extend that wait time some more? serious request :D
I know i don't like my question...more time lost while (re)booting :(
-
@mlapse
Cameras that can autoboot from FAT32 cards don't support dual partitions because the code to do so was not added - it was not thought necessary.
However there is no reason they cannot do so - it just requires a change to the CHDK startup code to detect and swap partitions.
I have tried this on my G1X and it seems to work fine.
In addition supporting a dual partition card with the 2nd (large) partition formatted as exFat is also just a code change to the CHDK boot code (provided the camera natively supports exFat). The G12 has always had this and I have tested this in my G1X as well.
If you can tell me the firmware version of your G1X I can create a test build that handle dual partition cards.
-
Reyalp, I thought you changed that timing to change with card size since we had that issue with the s95.
but maybe not dynamically enough :)
is it possible to extend that wait time some more? serious request :D
In r5183, I changed the timeout from 4 seconds to 10. This is a fixed value, it doesn't know anything about card size or file system or anything like that.
Here's a test build with it set to 20 for g1x 101a.
I'm a bit reluctant to make it arbitrarily long, because if there are any ports that don't reliably call core_spytask_can_start, users will be forced to wait the full timeout for every boot.
It's also not a very good workaround for people using large cards. As I mentioned when the topic first came up, it's possible that different fat parameters could have some effect. However, I don't have any cards larger than 64 MB to test, and haven't noticed those being particularly slow.
An alternative solution might be to extend multi-partition support to allow a FAT boot partition and exFAT, but I'm not sure how much work that would take, and multipartition is such a source of installation problems and help requests I'm not keen to bring it to modern cameras :-[
edit: I see phil is already there, and I have no objection to it being added if it works.
-
Philmoz, i tried a dual partition fat16/exfat on the s95 with a 256gb card, but that did not work.
but if you say fat32/exfat will work i will give that a try to see if it speeds things up... the g1x is 101a
Reyalp, the longer time gives no warning of missing modules.
it works normally, but only if you start via play.
if started via on/off you have too keep the button pressed for 5 seconds for it to respond and, as expected, it pushes the cam to record mode immediately.
and yes, the idea of even larger cards worries me a bit too, on the other hand the standard only runs up to 2TB with fat32, so we are almost there :)
next year I've got this project where i will have a cam for 3 month running without me being able to physically touch it for that period.
if it breaks down in that period all is lost and i have to start again one year later.
since i've had little time and progress to work on chdkptp this larger card is my fall back solution so the project can go onwards instead of being put on hold until i've become a decent programmer ;)
-
Attached is G1X 1.01a with dual partition support. 2nd partition can be FAT32 or exFat.
The S95 does not currently support exFat for the 2nd partition; but this should also be possible. Let me know which firmware version you have if you want to try a version with this for the S95.
-
thanks philmoz, i will test tomorrow....and if it works, the next 2 month or so.
and you don't have to make a version for the s95 since these work good enough as they are now.
with 256 gb cards dual partition you can shoot >100.000 images in one run...my best one yet made over 2.5m images before it mechanically died.
-
i tried creating a dual partition card via the camera, that works despite the missing modules.
but when i tried to format the small partition to fat32 it stated:
This drive is too small for fat32 - there must be at least 64k clusters
however, that does not matter fat16 + exfat works....and the first impression is that it is fast! instant startup as with 1 partition exfat....maybe that wait time can go down to 6 sec again? to verify if it only works on the g1x or similar on other cams, i can use a version for the s110-102b if you like...rest is occupied.
Philmoz, Reyalp, this should be implemented on all cams that boot from fat32, it is a mayor step forward in my opinion.....maybe with a disclaimer 'for crazy large cards that create a missing module error at startup' use dual partition with exfat !
although it might already improve performance at 64gb cards it might not be as noticable. i don't know if this is wishfull thinking on my part, but the g1x seems quicker ready after a shot and more responsive in between now it's working with dual partition fat16/exfat opposed to a smaller size fat32 formatted card.
while my collection of working s95's is dwindling down i thought there would come a time that i would never, ever had to use those frustrating dual partition cards anymore. and now i'm promoting it's return :D
-
Test version with dual partition support for S110 1.02b.
-
tested the s110, works too and again, faster boot with exfat dual partition on a larger sd card than fat32 single partition on a much smaller card.
as for response and speed...it feels better, but have no figures or test to back that up.
if you have 2x 64gb or larger cards i would advise you to try yourself. the difference is amaizing. :) :D 8) :-* :lol
while the experience seems smoother on the s110, the g1x is almost like a different camera.
Reyalp, if you rather have that wait time set back to 6 seconds, i'm willing to try if that works dual boot on the g1x and s110. seeing the results i think it will.
-
The two test builds have the startup wait time at 10 seconds not the 20 seconds from reyalp's test build.
It used to be 4 seconds; but I would probably leave this at 10 as it still helps with other cameras like the S95.
-
The two test builds have the startup wait time at 10 seconds not the 20 seconds from reyalp's test build.
It used to be 4 seconds; but I would probably leave this at 10 as it still helps with other cameras like the S95.
Yeah, it's been at 10 seconds for a long time (May 2019, meaning it was included in 1.5 release) without complaints.
If I were confident there were no cameras missing core_spytask_can_start, I'd increase to to 20 or more, since other people are likely using large FAT32 cards as well. I suppose if we set it very long, people with broken ports would complain.
-
i was there when it was upped to 10 seconds in may 2019 because of my large sd card for the s95 had missing modules.
and it seemed like a good idea then.
but then exfat on dual partition cards did not work yet as i tested that back then as mentioned in this thread.
now that's solved i thought you might want to reconsider.
even if your port is not broken.
the test with 20 seconds from a few days back is failing on the g1x since it won't start up normally but needs a 5 sec [on] press, so i wouldn't recommend that.
and obviously it is nice to be able to choose between fat32 single or exfat dual partition.
but because of the speed boost i think i will follow industry standard and will format all larger than 32gb cards to exfat from now on....if the cams chdk supports it.
-
My biggest FAT32 card is 128GB. So far I had no problems with the M3 and G1x. I do have a 256GB card for the G1x but I only use it with exFAT.
-
philmoz, i was aware that your test build had only a 10 sec delay. since it stated missing modules before i made the card dual boot.
c_joerg, as pointed out hereabove, the 256gb card size was solved in may 2019 by extending wait time to 10 seconds
this is not about those problems anymore, both formats will work. it is about speed.
didn't you see a startup difference with the 128 gb card fat32 vs the 256gb exfat?
with my new card it went from >14 seconds startup to <2 seconds for the canon splash screen...i'm guessing it might be less distinct on a 128gb card, but it's such a difference it should be noticable.
-
didn't you see a startup difference with the 128 gb card fat32 vs the 256gb exfat?
When starting via firmware update I see no difference. I can't start automatically with the exfat, so no compare.
-
even if you unlock your fat32 and just start the cam without chdk you can compare with the exfat
this has nothing to do with chdk, it's all canon. so the results are consistent with empty exfat and fat32 cards