DryOS - some success

[jeff666]

During the last days I poked around a bit in DryOS. This posting reports what I did and what works.

After making sure that the restart-function does actually restarts the cam (and provoke a boot-loop by reloading diskboot.bin) I took the original CHDK-Loader-framework and duplicated parts of the original firmware into the loader.

Copying data and bss is done using the existing CHDK-functions. I didn't relocate anything, yet.

I had to duplicate more than previously necessary (6 functions, till now) since the SD is initialized right before diskboot.bin is loaded. So I had to change the diskboot-loader which is called later than in VxWorks.

What works:
Right now I control the boot-process until right after my modified diskboot-loader is called. Right after that I return control to the firmware which does the rest of the start.

What needs to be done:
- Hook own code so it is called during normal operation. There is a function called "CreateProxyOfEventProcedure". Maybe this can be used as a hook. The other thought is to register a Task using "CreateTask" (and hope it gets called regularly).
- turn off the card lock.
- find the location of useful functions, create signature-files for findsig

(unsolved) Problem:
I duplicated the function that calls the diskboot-loader. This function calls about 20 other subs. My copy called a modified diskboot-loader that just initializes the SD. For some unknown reason one of the following calls doesn't return.
However if I jump into the original function, the camera starts correctly. Is it possible that my code is overwritten?

Here is my progress so far: http://www.zshare.net/download/5952071a9847ef/
The interesting file is platform/a720/sub/101c/boot.c

To demonstrate that something happens, the blue LED blinks and is turned on at the start.
The code works on a A720IS ver. 1.01C

Cheers.

[jeff666]

Minor update:

I solved the problem about the possibly overwritten code. It seems as it has actually been overwritten. The bug disappeared after moving the .bss offset up by the usual amount (0x20000).

My copy of task_Startup now runs completely and the camera boots as expected.

I also had success creating a new task, but I'm a little short on information about tasks.
Is a task comparable to a process on a modern OS?
Will it get interrupted by a preemptive scheduler?
Any other interesting details about handling tasks?

Cheers.

[lukg]

I'm with you jeff, my s5 thanks you!
I hope you will succeed to port chdk on dryos camera

[jeff666]

Good news for the S5 users. I had a quick look into the S5 firmware and it's pretty much identical to the A720. There are a few minor changes, I noticed a source file was renamed but the functions remain the same.

Everything that's necessary to control the boot process can be identified within a few minutes.

Also some further progress on DryOS in general.
I created a new task that blinks an LED during normal operation. Task handling seems to be identical to the old OS. C-calls work well (no need to write everything in assembly - yay )
Bad news: there's probably no equivalent to taskCreateHookAdd. This is a little unfortunate but we can still create own tasks and at least "PhySw" can be redirected easily since it is started directly from task_Startup.

Tomorrow I plan to
- do graphics output (using existing CHDK code)
- hook the keyboard (using CHDK as well)
- get rid of the card lock.

Cheers.

[Mantulen]

Jeff, you are the man!  I have a S5, and I am looking forward to your progress! 

[Altsoph]

Hi, Jeff.

I've got a720 recently, and going to investigate it.
Earlier I done some work for s2is.

For now, I got the dump and making some RE on it.

I'll be online after 2008-01-13. so mail me, and let's do it together.

Sorry my poor english

[jeff666]

I've got a720 recently, and going to investigate it.
Earlier I done some work for s2is.

It's great that there's more devs interested in DryOS support. I think it will be helpful that you know how CHDK should work, both internally as well as how it "looks".

Before you start, check the version of your A720 firmware. The available firmware (from my camera) is a 1.00C. You might need to dump yours if you have a different version.

I suppose you read the DryOS-porting-article in the wiki.

I'll be online after 2008-01-13. so mail me, and let's do it together.

That means you won't be available till Jan, 13th? I'm back at work from Jan, 8th and will work on CHDK till then. Won't have much time afterwards, though.

Anyways, here's the current progress:
- boot works (all in platform/a720/sub/100c/boot.c)
- keyboard is hooked but calls firmware-functions. No kbd-specific CHDK-code, yet.
- some entry-points have been found out and manually added to stubs_entry_2.S
- a proof-of-concept task is started that blinks an LED but it might be useful to tweak stuff that's not available during startup.

The thing that nags me the most, currently, is the card lock. In case you're available these days, you could check how to solve this.

Here's the file: http://www.zshare.net/download/6009710faa4a9a/
I also added a dump of the IDA function list to the archive. It might be useful.

Sorry my poor english

What's you native language?

Btw: Happy hacking

Cheers.

[Altsoph]

I suppose you read the DryOS-porting-article in the wiki.

I've already read that article and done some research on dump (by IDA).

That means you won't be available till Jan, 13th? I'm back at work from Jan, 8th and will work on CHDK till then. Won't have much time afterwards, though.

Tomorrow I going to my "new year" rest, jan 13th'll be back for my main work and work on CHDK.

I also added a dump of the IDA function list to the archive. It might be useful.

Thanks, I've downloaded it, will check it after holidays...

What's you native language?

Russian.

[jeff666]

Tomorrow I going to my "new year" rest, jan 13th'll be back for my main work and work on CHDK.

I assume you will find a much more advanced state of the dryos port when you return. I will document major progress in this thread and fill you in with details if things are unclear.

Enjoy your holidays.

Cheers.

[DataGhost]

Nice work!
I have an S5 and quite some programming experience, so I looked into it a bit... unfortunately I figured out that it's quite hard to start from scratch on ARM ASM and CHDK in general (I don't really know any of the CHDK code, though ARM ASM is quite easy... but still I get lost in the disassembly), so I couldn't really contribute there. All I really managed was to get some lights to blink in C and ASM, but that's nothing compared to what you did.
Anyway, if you need any help on the S5 part (I understand you do not have it), I'd be happy to help out whereever I can. In the meantime, I hope your progress (reports) will help familiarize myself with the code a bit. For that, it would be nice to have some more detailed reports on what you did exactly, though, if that's not too big of a hassle for you.

[jeff666]

I have an S5 and quite some programming experience, so I looked into it a bit... unfortunately I figured out that it's quite hard to start from scratch on ARM ASM and CHDK in general (I don't really know any of the CHDK code, though ARM ASM is quite easy... but still I get lost in the disassembly), so I couldn't really contribute there. All I really managed was to get some lights to blink in C and ASM, but that's nothing compared to what you did.

You should have seen me nervously happy running around in my room after having some LED blinking and realizing that a hack would be possible So blinking some LEDs is still quite important (and looses importance only after the display works).

As you noticed, ARM Asm is fairly simple and easy to read, but there are still some powerful statements that might do three things intermixed with each other in five lines of code. I had to rewrite a fraction of code to clarify what it does more than once.

Anyway, if you need any help on the S5 part (I understand you do not have it), I'd be happy to help out whereever I can.

Right, I don't have an S5, I just have an A720.

So let's get you started.

You will need:
Firmware-dumps of S5IS, A720, A710 and maybe S3 or S2.
Reason:
S5 because it's your own (make sure that your cam has the same version as the dump)
A720 because it's the reference-dryos-firmware, lots of functions have been named and I use it for my CHDK-port
A710 because I use the A710-platform-dependent code as starting point for my port. There may be references to this model.
S3/S2 because S-Series cameras may do things totally different as A-Series (kbd-handling is said to be different) and you will need to merge S3/S2-chdk-code into my dryos-modified sources.
You find the files http://www.zshare.net/download/590221805aee35/ and http://www.zshare.net/download/3711689d229885/ (thx to GrAnd for uploading).

You also need CHDK trunk. I used it to start, made a copy of the A710-directories and removed everything else device-dependent (to minimize confusion). So neither clean A710 nor S2/S3-code is in the archives I posted and you will need to get them from a clean CHDK trunk.

I suppose you already read everything in the wiki. If you didn't then read at least the developer section. Useful information might also be mentioned in articles that don't obviously seem interesting (i.e. the camera articles).

Nearly forgot to mention: You need IDA, but I guess you already have it. You also need something to compile CHDK. I assume you have that also. I build CHDK in linux (http://chdk.wikia.com/wiki/Compiling_CHDK_under_Linux). I also tried the win32 toolchain but it didn't compile CHDK!

Now you're ready to start.

What you do to get the port is basically:

• read a function in existing CHDK code
• find out how it interacts with the firmware
• read the part of the (Vx) firmware that it interacts with
• find the corresponding part in the (Dry) firmware
• change the CHDK function code to interact with the (Dry) firmware

To know how to proceed, here's a few details about CHDK:

• The entrypoint for the code is always the entry.S file. No libc, no "int main()".
• There are three entry.S files:
• loader/<dev>/entry.S: This is the starting point.
        At this point, diskboot.bin has been load to 0x1900 by the firmware and needs to be relocated to a safe location.
        This entry.S calls my_restart which calls code in loader/<dev>/resetcode.
        There's a little glitch. my_restart defines a function called copy_and_restart. This one points actually to the next entry.S.
• loader/<dev>/resetcode/entry.S
        Calls copy_and_restart which relocates the CHDK-code, executes the restart-code and continues execution in the relocated CHDK.
        The reset-code (asm-fraction) is a duplicate from the original firmware. Look for the string "Restart called".
        At the end of the reset-code execution continues in the next entry.S
   
• core/entry.S
        calls "startup()" in platform/generic/main.c. From here on everything is in its place and works like a normal c program. No weird relocations anymore.
   
• "startup()" calls "boot()" in platform/<dev>/sub/<ver>/boot.c
  This function controls the boot process and calls functions that were relocated from the original firmware since they need minor modifications like changed addresses or just skipping of a particular function.
   

There are a couple differences between Vx and Dry in boot.c. Reasons:

• Both OSes have "Tasks". They are somewhat like processes on a full-blown OS, only here we have cooperative multitasking. It means that a Task has to call SleepTask to return control to the OS and allow other tasks to run.
• A new task is created by calling CreateTask and passing it a reference to the c-function that has to run as task as well as a descriptive name of the task.
• during Startup a number of tasks are spawned.
• VxWorks had a function called taskCreateHookAdd(cb) allowing an override to CreateTask. A callback would be executed if a new task has to be spawned. By knowing the function names it was easy to replace a few particular tasks by own ones.
• DryOs OTOH doesn't seem to have this function. This requires us to duplicate quite a lot of code from the firmware until we can change the CreateTask call.
• There's another reason why more code has to be duplicated than earlier needed: The diskboot-loader. In VxWorks a OS-function was used to create a device named "A/DISKBOOT.BIN". When the diskboot-loader tried to open the file of that name it got the device - not our file - thus preventing a boot loop. If DryOS had a function that allowed similar action it is at least not found (yet?). What I did was copying more code and just skipping the call to the diskboot-loader.

An important function is called "Startup" or "task_Startup" (as it is installed as task). It exists in both Vx and Dry and directly calls the diskboot-loader ("StartDiskboot") as well a task called "PhySw". The latter handles the keyboard and needs to be diverted.

CHDK also spawns a task called "spytask" which is the core of CHDK. I'm currently trying to figure out why this function doesn't even get called after CreateTask-ing it. Probably some stupid misunderstanding between source files.

To find out how task hooking worked in Vx, check createHook() in platform/generic/main.c

The things you will want to do now are

• Take my code (link at the end of the posting), duplicate/rename the A720-directories in loader/ and platform/ to S5IS. Don't forget the firmware-version in platform/s5is/sub/
• change makefile.inc to point to your camera and firmware-revision
• make fir (to test if compile works), resulting diskboot.bin is in bin/
• manage to get the resetcode to work. If you did this, you will find the rest by yourself.

Here's a few little hints to get things working:

• Use the LEDs to check if code is executed.
• Progress in little steps. ie. don't change the whole resetcode and boot.c at once but first add a jump to the original resetcode and see if you get a boot-loop.
  Then duplicate the resetcode to your source and see if you still have a boot-loop.
  Then change the jump at the end of the reset-code, change boot() to jump to the firmware start and check for boot-loop.
  And so on until you get the camera starting.

Occasionally you will want to break out of the boot-loop to check if a complete startup still works. It's pretty unsatisfying if you duplicate all the boot-code until finally skipping StartDiskboot just to find out that the cam doesn't boot anyways.
This is what I did: I blinked an LED on startup, thus delaying the boot process by a short time. I also made some minor hack that allowed me to run the camera with opened battery/card-cover. This way I could remove the card after diskboot.bin has started, stopping the boot-loop and get a regular startup.

I hope I mentioned everything. If you still have questions: Just ask.

Edit:
Nearly forgot the link: http://www.zshare.net/download/6055482b209930/

Cheers.[/list]

[oboka]

WOW Jeff!! what a great start tutorial!! I own an S5 cam, if I've time I'll try this to see if I can help.

Thank you!

[DataGhost]

Great post, thanks

[..]
You should have seen me nervously happy running around in my room after having some LED blinking and realizing that a hack would be possible So blinking some LEDs is still quite important (and looses importance only after the display works).

That's what I did, after that I coded "Hello, world!" in morse but it quickly got boring, especially after I did the same in asm and familiarized myself with the syntax.

[..]
Right, I don't have an S5, I just have an A720.
[..]
Firmware-dumps of S5IS, A720, A710 and maybe S3 or S2.
S5 because it's your own (make sure that your cam has the same version as the dump)
[..]
You find the files http://www.zshare.net/download/590221805aee35/ and http://www.zshare.net/download/3711689d229885/ (thx to GrAnd for uploading).

Thanks, I've been looking for those to figure out how everything is supposed to interact, but I failed to locate them. About the dump version... after overlooking it initially, I found out that I have 1.01b, the dump I downloaded is 1.01a. I don't have an optical input on my sound card or anything like that, so I'll have to come up with something to dump it (I expect writing it to the SD isn't that straightforward yet).

... trunk ... wiki ... IDA ... ARM toolchain ... porting information ... libc ... hints ...

I got that far, yes. I'm using version 3.4.3, which I already had to compile iPod (same arch) applications / bootloader, building the bootdisk.bin files for my camera worked fine.
Wiki... haven't read it all but I think I read a fair share of the relevant information for this port.
IDA... check
Also, I'm quite familiar with programming in the linux kernel, reverse engineering on a device with limited output capabilities (printf/k debugging ) and such, so that's not a problem. I just hope I can find enough time to work on it

[jeff666]

Great post, thanks

Pure selfishness. If I get other people to work on the subject, I have to do less 

That's what I did, after that I coded "Hello, world!" in morse

You must have been really bored

You find the files http://www.zshare.net/download/590221805aee35/ and http://www.zshare.net/download/3711689d229885/ (thx to GrAnd for uploading).

Thanks, I've been looking for those to figure out how everything is supposed to interact, but I failed to locate them. About the dump version... after overlooking it initially, I found out that I have 1.01b, the dump I downloaded is 1.01a. I don't have an optical input on my sound card or anything like that, so I'll have to come up with something to dump it (I expect writing it to the SD isn't that straightforward yet).

SD writing isn't available (yet) - unfortunately. MX3 repeatedly had the idea about a universal dumper and was about to look at a portion of the firmware called "RomStarter". It's a separate function that doesn't intermix with the remaining firmware and just loads DISKBOOT.BIN in case the original firmware got "destroyed". We hope to gain information about low-level card-access and use it to write a firmware-file to the card. Maybe he already made some progress. I'll write a post in the other thread.

The optical soundcard-input wouldn't help much since the firmware-blinker doesn't blink the SPDIF protocol. You will need a phototransistor. It is mentioned in the wiki and some threads in the firmware-dumping section.

I got that far, yes. I'm using version 3.4.3, which I already had to compile iPod (same arch) applications / bootloader, building the bootdisk.bin files for my camera worked fine.

What platform?
Check if the archive I posted compiles ("make fir").
The arm-gcc that is built using the instructions in the wiki apply a patch to the gcc-sources. In general I suggest we all use the same compiler to minimize differences (different camera models are enough to cope with) and thus reduce the chance of "strange" errors. This way either everybody or nobody gets this error and the situation will be less confusing. You might already know: different versions of a software are support-hell

Wiki... haven't read it all but I think I read a fair share of the relevant information for this port.
IDA... check
Also, I'm quite familiar with programming in the linux kernel, reverse engineering on a device with limited output capabilities (printf/k debugging ) and such, so that's not a problem.

Sounds great.

For starters I set up a little problem for you to solve:
In platform/a720/sub/100c/boot.c I have three functions called "CreateTask_blinker", "CreateTask_spytask" and "CreateTask_PhySw", each just calling "CreateTask" and a reference to a function that has to be installed as a task. Two of them work (blinker and physw), one of them doesn't (spytask). I circled the problem down to the particular source-file which holds the installed function.
"task_blinker" is in the same source-file (platform/a720/sub/100c/boot.c),
"mykbd_task" (for PhySw) is in platform/generic/kbd.c,
"spytask" is in in core/main.c

I moved task_blinker to core/main.c as well to see if it stops working, and it did. So references between platform and core seem to be broken. Do you have any Idea what to look for?

I just hope I can find enough time to work on it

http://search.ebay.com/time? Maybe someone sells his

Cheers.

[GrAnd]

I moved task_blinker to core/main.c as well to see if it stops working, and it did. So references between platform and core seem to be broken. Do you have any Idea what to look for?

I have a guess. Thumb mode?
ARM processor has two modes of code execution: native (32bits instructions) and thumb (16bits instructions). They can't work together directly. To make a call of procedure from one type of code to another, a special interwork mechanism is used.
The original firmware has native ARM format.
But in the CHDK, only platform dependent code is compiled in native mode. The 'core' and 'libs' are compiled in thumb. It was done for reducing size of code. The functions from wrappers.c are used for interwork communications.
To determine, which file compiled in which mode, look into makefiles (see CTHUMB flag).

PS. Generally speaking, I'm not an expert in ARM at all. All this information I got from the founder of CHDK, Vitaly.