"refresh" feature for ROM area of Flash memory - General Discussion and Assistance - CHDK Forum  
« previous next »
Pages: [1] 2   Go Down

"refresh" feature for ROM area of Flash memory

  • 14 Replies
  • 1024 Views
*

Offline koshy

  • *****
  • 1051
"refresh" feature for ROM area of Flash memory
« on: 20 / December / 2019, 13:23:57 »
Advertisements
Given my recent "camera broken due to a deteriorated bit in FW, or rather flash memory" incident I pondered that the cameras lack a functionality to keep their flash memory contents healthy.The charge surely gets applied in writing and probably only gets measured thereafter it's bound to fail over time. Modern SSDs would have some refresh logic of their own but I don't expect that in the cameras.

So, I wondered what could we do about this? Wouldn't it make sense to have a functionality that iterates over the entire ROM area in a read a byte, write a byte, re-read the byte and if it's the same go on while otherwise write again kind of fashion? Just address by address so that even a power outage should ruin little? That is one step closer to something that could really brick a camera than what CHDK normally does but being a user invooked process I see no problem with decent documentation.

Should this kind of thing be a lua script or should it be a CHDK built in feature or should it be Canon basic or should one not do this at all for some reason I'm missing? I think if it existed I'd subject the cameras to the treatment every few years.
Logged
Koshy had a little ELPH which wasn't white as snow but everywhere that Koshy went the ELPH was sure to go. (actually an SD, but that detail ruins the rhyme...)
*

Online reyalp

  • ******
  • 12686
Re: "refresh" feature for ROM area of Flash memory
« Reply #1 on: 20 / December / 2019, 15:34:18 »
Quote from: koshy on 20 / December / 2019, 13:23:57
Modern SSDs would have some refresh logic of their own but I don't expect that in the cameras.
AFAIK, modern SSDs have error correction, which would flag corrupted bits before data loss. I don't know if they have any pro-active refresh, under normal use cases wear leveling would probably take care of it naturally.

In any case, I agree the cameras likely don't. Flash isn't archival storage, and a decade plus is well beyond the design life for consumer products like this. Unlike old film cameras, these things shouldn't be expected to be heirlooms that will work 50 or 100 years down the road. I suspect even Leicas and Hasselblads will bit rot about the same :haha

Quote
So, I wondered what could we do about this? Wouldn't it make sense to have a functionality that iterates over the entire ROM area in a read a byte, write a byte, re-read the byte and if it's the same go on while otherwise write again kind of fashion? Just address by address so that even a power outage should ruin little?
Generally, flash is only written in larger blocks, and can only be written after an "erase". While the firmware function lets you write a single byte, under the hood what almost certainly happens is the whole block is read, the byte(s) are modified in memory, the block is erased, and then written. It's not immediately obvious to me how much of this happens in firmware code vs in the flash controller.

The flash in these cameras is presumably NOR flash.

To further complicate things, on most of these cameras the functions to write to flash are also in flash. Refreshing the bit of firmware currently being executed could be sporty ;) The Canon firmware update process doesn't have this issue, because it uses a standalone executable which runs from RAM.

IMO, a more tractable approach would be to checksum the code and constant data areas in flash and compare them to a known good dump. This isn't trivial either, since there are regions used to store settings and calibration data. Of course, this doesn't give you proactive refresh, if a bit in the bootloader or startup code goes bad, you'd still be hosed, but statistically that should be a lot rarer.
« Last Edit: 20 / December / 2019, 16:54:20 by reyalp »
Logged
Don't forget what the H stands for.
*

Offline koshy

  • *****
  • 1051
Re: "refresh" feature for ROM area of Flash memory
« Reply #2 on: 20 / December / 2019, 16:45:44 »
Quote from: reyalp on 20 / December / 2019, 15:34:18
Unlike old film cameras, these things shouldn't be expected to be heirlooms that will work 50 or 100 years down the road.
An image formed on my mind when you said that, couldn't resist actually creating it.

EDIT: The S80 is in that for a reason other than being a cute camera. While I saw one S80 with FW 1.01B the handful of others I looked at were 1.00G or below and there is an "unofficial FW upgrade to 1.00G" which will even work on an existing 1.00G and thus effectively do the refreshing. Whether it works on the 1.01B I didn't dare instruct to try at the time which is where the S80 1.01B port originated... Anyway, this gives the S80 a headstart over most others.
« Last Edit: 21 / December / 2019, 14:20:09 by koshy »
Logged
Koshy had a little ELPH which wasn't white as snow but everywhere that Koshy went the ELPH was sure to go. (actually an SD, but that detail ruins the rhyme...)
*

Offline koshy

  • *****
  • 1051
Re: "refresh" feature for ROM area of Flash memory
« Reply #3 on: 20 / December / 2019, 17:03:05 »
Thank you so much for the extensive reply, this takes the musings to a different level.
How neat if one has someone who can do that to a conversation.

So, byte wise re-writing won't work as naively expected. An interesting remark was the one on using the Firmware updating routine to refresh "ROM". Is there any functionality to read/write block wise? Even if writing a byte actually writes more it may still be feasible to use that. The actual difference seems to be on the risk assessment side. If something goes wrong after the erase of a block the camera could be bricked. In a time (today) where many cameras still work and the cameras I'm thinking about are cheap to replace with working ones, at least for a while, it would still be something I'd subject the cameras to every few years if it were available.
It's a question of targeted life span, something will break eventually but I'd hate it to be something as dumb as Flash memory data storage. The clock on the Ixus at hand ends in 2037. Project 2037  ;). In that line of thinking the boot code would need to be refreshed sometime. All more complicated than I hoped, though.
Logged
Koshy had a little ELPH which wasn't white as snow but everywhere that Koshy went the ELPH was sure to go. (actually an SD, but that detail ruins the rhyme...)
*

Offline koshy

  • *****
  • 1051
Re: "refresh" feature for ROM area of Flash memory
« Reply #4 on: 21 / December / 2019, 17:10:28 »
Quote from: koshy on 20 / December / 2019, 16:45:44
EDIT: The S80 is in that for a reason other than being a cute camera. While I saw one S80 with FW 1.01B the handful of others I looked at were 1.00G or below and there is an "unofficial FW upgrade to 1.00G" which will even work on an existing 1.00G and thus effectively do the refreshing. Whether it works on the 1.01B I didn't dare instruct to try at the time which is where the S80 1.01B port originated... Anyway, this gives the S80 a headstart over most others.
Given that S80 *.fir file and the known 1.00G S80 dump, how complex is the FIR file architecture in this use case? Can a .fir file be created for a given camera with the intent of overwriting the existing FW with a copy of itself? That would do what is needed. Not to be confused with producing FW updates for cameras that did not originally have the given FW. Who knows if a hardware detail changed in the course of production or whatever. No, the intent being overwriting a given FW with itself and thus refreshing the flash memory.
Logged
Koshy had a little ELPH which wasn't white as snow but everywhere that Koshy went the ELPH was sure to go. (actually an SD, but that detail ruins the rhyme...)
*

Online reyalp

  • ******
  • 12686
Re: "refresh" feature for ROM area of Flash memory
« Reply #5 on: 21 / December / 2019, 18:26:08 »
Quote from: koshy on 21 / December / 2019, 17:10:28
Given that S80 *.fir file and the known 1.00G S80 dump, how complex is the FIR file architecture in this use case? Can a .fir file be created for a given camera with the intent of overwriting the existing FW with a copy of itself? That would do what is needed.
Someone would need to reverse engineer it. I would expect it's relatively straightforward for old cameras like this, but I haven't looked at it. I'm not aware of the details being written down anywhere, but it's likely they were discussed in early CHDK history. The hacks for the Digic I DOS/x86 models were based on firmware update hacking.

The bottom line is that reverse engineering is very time consuming. Personally, I have an essentially unlimited list of things that would be interesting to investigate and this is pretty low on it.

Another issue is testing, since there's a real risk of bricking if you get something wrong. If the romstarter is intact, it should be possible to recover but likely requires physical UART access.

Speaking of the romstarter (essentially, bootloader), there's a good chance that normal firmware updates only do the main firmware, so would not refresh the romstarter. It's quite possible there are other areas a regular firmware update doesn't touch. Though again, statistically romstarter is a tiny part of the firmware, so if bit flips are randomly distributed, the chances of it going bad in a fatal way should be fairly low.

There is another age factor too: Camera settings are stored in flash, and written quite frequently (likely every power cycle, at minimum). Once these wear out the camera is probably toast. I don't know if there's any wear leveling happening behind the scenes. There have been a couple of instances of recovering cams with corrupted settings.

Quote
Not to be confused with producing FW updates for cameras that did not originally have the given FW.
Yes, updating other models would be far more difficult. The .FIR/FI2 file presumably has all the drivers for that specific camera built in, and would only be valid for cameras with exactly the same MMIO and ROM layouts.

Quote
Is there any functionality to read/write block wise?
There is EraseSectorOfRom. I'd guess WriteToRom does the right thing if given a full block aligned to whatever the block size is.
Logged
Don't forget what the H stands for.
*

Offline c_joerg

  • *****
  • 933
Re: "refresh" feature for ROM area of Flash memory
« Reply #6 on: 22 / December / 2019, 05:59:29 »
Quote from: reyalp on 21 / December / 2019, 18:26:08
Speaking of the romstarter (essentially, bootloader), there's a good chance that normal firmware updates only do the main firmware, so would not refresh the romstarter. It's quite possible there are other areas a regular firmware update doesn't touch.

This is exactly what I would expect that the loader and firmware are separate.

There are several reports on the www that people have deliberately removed the battery during a firmware update. Nothing should have happened.
I can well imagine that the memory for the firmware is a ping pong memory. Only when the firmware has been completely saved and checked, is it switched over.

My two S45 from 2003 are still working without problems  :xmas...


« Last Edit: 22 / December / 2019, 06:01:50 by c_joerg »
Logged
M100 100a, M3 101a, 2*G1x (101a,100e), S110 (103a), SX50 (100c), SX230 (101a), S45,
Flickr https://www.flickr.com/photos/136329431@N06/albums
YouTube https://www.youtube.com/channel/UCrTH0tHy9OYTVDzWIvXEMlw/videos?shelf_id=0&view=0&sort=dd
*

Offline srsa_4c

  • ******
  • 4262
Re: "refresh" feature for ROM area of Flash memory
« Reply #7 on: 22 / December / 2019, 10:42:05 »
Quote from: c_joerg on 22 / December / 2019, 05:59:29
I can well imagine that the memory for the firmware is a ping pong memory. Only when the firmware has been completely saved and checked, is it switched over.
Certainly not on Canons. There is no extra flash memory on their cameras.
Logged
*

Offline koshy

  • *****
  • 1051
Re: "refresh" feature for ROM area of Flash memory
« Reply #8 on: 23 / December / 2019, 19:57:09 »
Quote from: srsa_4c on 22 / December / 2019, 12:33:42
Quote from: c_joerg on 22 / December / 2019, 11:15:47
My question is actually whether it is possible to carry out a Canon firmware update without a Canon menu?
For example, if parts of the firmware in the camera are destroyed.
If the main firmware is able to start a diskboot.bin, one could create a special one that is able to reflash the damaged areas. If the damage is extensive enough, many parts of the rom would need to be copied to make diskboot functional.
In some cases (erased or damaged main fw start), the bootloader can also resort to starting diskboot.bin - if it finds one on card.
AFAIK the bootloader cannot deal with firmware update files.
I figured this should be quoted here as well, basically as long as the romstarter is not dead there is hope. Right now the functionality to put a FW back in place does not exist but we mused about it here. Also right now the problem is not too common, if it becomes common with cameras more likely dead than alive the priority of this may well change if able folks are still around then...

I think this is pretty important to this general consideration. No immediate need to do it now. Even with seemingly dead cameras they could be brought to life later. If reyalp is correct on the assumption that the FW Update does not overwrite the actual boot loader it really makes no difference whether to "refresh flash" or to "restore flash gone bad"...

Q: If one has two FW dunps, how to best compare the static areas of the two to see if a bit has flipped by comparing the dumps?
Logged
Koshy had a little ELPH which wasn't white as snow but everywhere that Koshy went the ELPH was sure to go. (actually an SD, but that detail ruins the rhyme...)
*

Offline Caefix

  • ****
  • 253
  • Sorry, busy deleting test shots...
Re: "refresh" feature for ROM area of Flash memory
« Reply #9 on: 24 / December / 2019, 10:46:02 »
 ;) How about treating them as RAW and marking bad pixels?  :haha
Logged
All lifetime is a loan from eternity.
Pages: [1] 2   Go Up
« previous next »
 

Related Topics