DryOS - some success - page 2 - DryOS Development - CHDK Forum supplierdeeply
« previous next »
Pages: 1 [2] 3 4 5 6 7 ... 23   Go Down

DryOS - some success

  • 220 Replies
  • 174580 Views
*

Offline jeff666

  • ****
  • 181
  • A720IS
Re: DryOS - some success
« Reply #10 on: 31 / December / 2007, 09:05:25 »
Advertisements
    Quote
    I have an S5 and quite some programming experience, so I looked into it a bit... unfortunately I figured out that it's quite hard to start from scratch on ARM ASM and CHDK in general (I don't really know any of the CHDK code, though ARM ASM is quite easy... but still I get lost in the disassembly), so I couldn't really contribute there. All I really managed was to get some lights to blink in C and ASM, but that's nothing compared to what you did.

    You should have seen me nervously happy running around in my room after having some LED blinking and realizing that a hack would be possible :) So blinking some LEDs is still quite important (and looses importance only after the display works).

    As you noticed, ARM Asm is fairly simple and easy to read, but there are still some powerful statements that might do three things intermixed with each other in five lines of code. I had to rewrite a fraction of code to clarify what it does more than once.

    Quote
    Anyway, if you need any help on the S5 part (I understand you do not have it), I'd be happy to help out whereever I can.

    Right, I don't have an S5, I just have an A720.

    So let's get you started.

    You will need:
    Firmware-dumps of S5IS, A720, A710 and maybe S3 or S2.
    Reason:
    S5 because it's your own (make sure that your cam has the same version as the dump)
    A720 because it's the reference-dryos-firmware, lots of functions have been named and I use it for my CHDK-port
    A710 because I use the A710-platform-dependent code as starting point for my port. There may be references to this model.
    S3/S2 because S-Series cameras may do things totally different as A-Series (kbd-handling is said to be different) and you will need to merge S3/S2-chdk-code into my dryos-modified sources.
    You find the files http://www.zshare.net/download/590221805aee35/ and http://www.zshare.net/download/3711689d229885/ (thx to GrAnd for uploading).

    You also need CHDK trunk. I used it to start, made a copy of the A710-directories and removed everything else device-dependent (to minimize confusion). So neither clean A710 nor S2/S3-code is in the archives I posted and you will need to get them from a clean CHDK trunk.

    I suppose you already read everything in the wiki. If you didn't then read at least the developer section. Useful information might also be mentioned in articles that don't obviously seem interesting (i.e. the camera articles).

    Nearly forgot to mention: You need IDA, but I guess you already have it. You also need something to compile CHDK. I assume you have that also. I build CHDK in linux (http://chdk.wikia.com/wiki/Compiling_CHDK_under_Linux). I also tried the win32 toolchain but it didn't compile CHDK!

    Now you're ready to start.

    What you do to get the port is basically:
    • read a function in existing CHDK code
    • find out how it interacts with the firmware
    • read the part of the (Vx) firmware that it interacts with
    • find the corresponding part in the (Dry) firmware
    • change the CHDK function code to interact with the (Dry) firmware

    To know how to proceed, here's a few details about CHDK:
    • The entrypoint for the code is always the entry.S file. No libc, no "int main()".
    • There are three entry.S files:
    • loader/<dev>/entry.S: This is the starting point.
            At this point, diskboot.bin has been load to 0x1900 by the firmware and needs to be relocated to a safe location.
            This entry.S calls my_restart which calls code in loader/<dev>/resetcode.
            There's a little glitch. my_restart defines a function called copy_and_restart. This one points actually to the next entry.S.
    • loader/<dev>/resetcode/entry.S
            Calls copy_and_restart which relocates the CHDK-code, executes the restart-code and continues execution in the relocated CHDK.
            The reset-code (asm-fraction) is a duplicate from the original firmware. Look for the string "Restart called".
            At the end of the reset-code execution continues in the next entry.S
       
    • core/entry.S
            calls "startup()" in platform/generic/main.c. From here on everything is in its place and works like a normal c program. No weird relocations anymore.
       
    • "startup()" calls "boot()" in platform/<dev>/sub/<ver>/boot.c
      This function controls the boot process and calls functions that were relocated from the original firmware since they need minor modifications like changed addresses or just skipping of a particular function.
       

    There are a couple differences between Vx and Dry in boot.c. Reasons:
    • Both OSes have "Tasks". They are somewhat like processes on a full-blown OS, only here we have cooperative multitasking. It means that a Task has to call SleepTask to return control to the OS and allow other tasks to run.
    • A new task is created by calling CreateTask and passing it a reference to the c-function that has to run as task as well as a descriptive name of the task.
    • during Startup a number of tasks are spawned.
    • VxWorks had a function called taskCreateHookAdd(cb) allowing an override to CreateTask. A callback would be executed if a new task has to be spawned. By knowing the function names it was easy to replace a few particular tasks by own ones.
    • DryOs OTOH doesn't seem to have this function. This requires us to duplicate quite a lot of code from the firmware until we can change the CreateTask call.
    • There's another reason why more code has to be duplicated than earlier needed: The diskboot-loader. In VxWorks a OS-function was used to create a device named "A/DISKBOOT.BIN". When the diskboot-loader tried to open the file of that name it got the device - not our file - thus preventing a boot loop. If DryOS had a function that allowed similar action it is at least not found (yet?). What I did was copying more code and just skipping the call to the diskboot-loader.

    An important function is called "Startup" or "task_Startup" (as it is installed as task). It exists in both Vx and Dry and directly calls the diskboot-loader ("StartDiskboot") as well a task called "PhySw". The latter handles the keyboard and needs to be diverted.

    CHDK also spawns a task called "spytask" which is the core of CHDK. I'm currently trying to figure out why this function doesn't even get called after CreateTask-ing it. Probably some stupid misunderstanding between source files.

    To find out how task hooking worked in Vx, check createHook() in platform/generic/main.c

    The things you will want to do now are
    • Take my code (link at the end of the posting), duplicate/rename the A720-directories in loader/ and platform/ to S5IS. Don't forget the firmware-version in platform/s5is/sub/
    • change makefile.inc to point to your camera and firmware-revision
    • make fir (to test if compile works), resulting diskboot.bin is in bin/
    • manage to get the resetcode to work. If you did this, you will find the rest by yourself.

    Here's a few little hints to get things working:
    • Use the LEDs to check if code is executed.
    • Progress in little steps. ie. don't change the whole resetcode and boot.c at once but first add a jump to the original resetcode and see if you get a boot-loop.
      Then duplicate the resetcode to your source and see if you still have a boot-loop.
      Then change the jump at the end of the reset-code, change boot() to jump to the firmware start and check for boot-loop.
      And so on until you get the camera starting.

    Occasionally you will want to break out of the boot-loop to check if a complete startup still works. It's pretty unsatisfying if you duplicate all the boot-code until finally skipping StartDiskboot just to find out that the cam doesn't boot anyways.
    This is what I did: I blinked an LED on startup, thus delaying the boot process by a short time. I also made some minor hack that allowed me to run the camera with opened battery/card-cover. This way I could remove the card after diskboot.bin has started, stopping the boot-loop and get a regular startup.

    I hope I mentioned everything. If you still have questions: Just ask.

    Edit:
    Nearly forgot the link: http://www.zshare.net/download/6055482b209930/

    Cheers.[/list]
    « Last Edit: 31 / December / 2007, 09:23:19 by jeff666 »
    Logged
    *

    Offline oboka

    • *
    • 3
    Re: DryOS - some success
    « Reply #11 on: 01 / January / 2008, 06:41:22 »
    WOW Jeff!! what a great start tutorial!! I own an S5 cam, if I've time I'll try this to see if I can help.

    Thank you!
    Logged
    *

    Offline DataGhost

    • ****
    • 314
    • EOS 40D, S5IS
      • DataGhost.com
    Re: DryOS - some success
    « Reply #12 on: 01 / January / 2008, 08:12:19 »
    Great post, thanks :)

    Quote from: jeff666 on 31 / December / 2007, 09:05:25
    [..]
    You should have seen me nervously happy running around in my room after having some LED blinking and realizing that a hack would be possible :) So blinking some LEDs is still quite important (and looses importance only after the display works).

    That's what I did, after that I coded "Hello, world!" in morse but it quickly got boring, especially after I did the same in asm and familiarized myself with the syntax.

    Quote
    [..]
    Right, I don't have an S5, I just have an A720.
    [..]
    Firmware-dumps of S5IS, A720, A710 and maybe S3 or S2.
    S5 because it's your own (make sure that your cam has the same version as the dump)
    [..]
    You find the files http://www.zshare.net/download/590221805aee35/ and http://www.zshare.net/download/3711689d229885/ (thx to GrAnd for uploading).

    Thanks, I've been looking for those to figure out how everything is supposed to interact, but I failed to locate them. About the dump version... after overlooking it initially, I found out that I have 1.01b, the dump I downloaded is 1.01a. I don't have an optical input on my sound card or anything like that, so I'll have to come up with something to dump it (I expect writing it to the SD isn't that straightforward yet).

    Quote
    ... trunk ... wiki ... IDA ... ARM toolchain ... porting information ... libc ... hints ...

    I got that far, yes. I'm using version 3.4.3, which I already had to compile iPod (same arch) applications / bootloader, building the bootdisk.bin files for my camera worked fine.
    Wiki... haven't read it all but I think I read a fair share of the relevant information for this port.
    IDA... check
    Also, I'm quite familiar with programming in the linux kernel, reverse engineering on a device with limited output capabilities (printf/k debugging :) ) and such, so that's not a problem. I just hope I can find enough time to work on it :)
    « Last Edit: 01 / January / 2008, 08:15:16 by DataGhost »
    Logged
    *

    Offline jeff666

    • ****
    • 181
    • A720IS
    Re: DryOS - some success
    « Reply #13 on: 01 / January / 2008, 12:38:24 »
    Quote
    Great post, thanks :)

    Pure selfishness. If I get other people to work on the subject, I have to do less  8)

    Quote
    That's what I did, after that I coded "Hello, world!" in morse

    You must have been really bored :D


    Quote
    Quote
    You find the files http://www.zshare.net/download/590221805aee35/ and http://www.zshare.net/download/3711689d229885/ (thx to GrAnd for uploading).

    Thanks, I've been looking for those to figure out how everything is supposed to interact, but I failed to locate them. About the dump version... after overlooking it initially, I found out that I have 1.01b, the dump I downloaded is 1.01a. I don't have an optical input on my sound card or anything like that, so I'll have to come up with something to dump it (I expect writing it to the SD isn't that straightforward yet).

    SD writing isn't available (yet) - unfortunately. MX3 repeatedly had the idea about a universal dumper and was about to look at a portion of the firmware called "RomStarter". It's a separate function that doesn't intermix with the remaining firmware and just loads DISKBOOT.BIN in case the original firmware got "destroyed". We hope to gain information about low-level card-access and use it to write a firmware-file to the card. Maybe he already made some progress. I'll write a post in the other thread.

    The optical soundcard-input wouldn't help much since the firmware-blinker doesn't blink the SPDIF protocol. You will need a phototransistor. It is mentioned in the wiki and some threads in the firmware-dumping section.

    Quote
    I got that far, yes. I'm using version 3.4.3, which I already had to compile iPod (same arch) applications / bootloader, building the bootdisk.bin files for my camera worked fine.

    What platform?
    Check if the archive I posted compiles ("make fir").
    The arm-gcc that is built using the instructions in the wiki apply a patch to the gcc-sources. In general I suggest we all use the same compiler to minimize differences (different camera models are enough to cope with) and thus reduce the chance of "strange" errors. This way either everybody or nobody gets this error and the situation will be less confusing. You might already know: different versions of a software are support-hell :)

    Quote
    Wiki... haven't read it all but I think I read a fair share of the relevant information for this port.
    IDA... check
    Also, I'm quite familiar with programming in the linux kernel, reverse engineering on a device with limited output capabilities (printf/k debugging :) ) and such, so that's not a problem.

    Sounds great.

    For starters I set up a little problem for you to solve: :D
    In platform/a720/sub/100c/boot.c I have three functions called "CreateTask_blinker", "CreateTask_spytask" and "CreateTask_PhySw", each just calling "CreateTask" and a reference to a function that has to be installed as a task. Two of them work (blinker and physw), one of them doesn't (spytask). I circled the problem down to the particular source-file which holds the installed function.
    "task_blinker" is in the same source-file (platform/a720/sub/100c/boot.c),
    "mykbd_task" (for PhySw) is in platform/generic/kbd.c,
    "spytask" is in in core/main.c

    I moved task_blinker to core/main.c as well to see if it stops working, and it did. So references between platform and core seem to be broken. Do you have any Idea what to look for?

    Quote
    I just hope I can find enough time to work on it :)
    http://search.ebay.com/time? Maybe someone sells his :)

    Cheers.
    Logged
    *

    Offline GrAnd

    • ****
    • 916
    • [A610, S3IS]
      • CHDK
    Re: DryOS - some success
    « Reply #14 on: 01 / January / 2008, 13:39:05 »
    Quote from: jeff666 on 01 / January / 2008, 12:38:24
    I moved task_blinker to core/main.c as well to see if it stops working, and it did. So references between platform and core seem to be broken. Do you have any Idea what to look for?
    I have a guess. Thumb mode?
    ARM processor has two modes of code execution: native (32bits instructions) and thumb (16bits instructions). They can't work together directly. To make a call of procedure from one type of code to another, a special interwork mechanism is used.
    The original firmware has native ARM format.
    But in the CHDK, only platform dependent code is compiled in native mode. The 'core' and 'libs' are compiled in thumb. It was done for reducing size of code. The functions from wrappers.c are used for interwork communications.
    To determine, which file compiled in which mode, look into makefiles (see CTHUMB flag).

    PS. Generally speaking, I'm not an expert in ARM at all. All this information I got from the founder of CHDK, Vitaly.
    Logged
    CHDK Developer.
    *

    Offline DataGhost

    • ****
    • 314
    • EOS 40D, S5IS
      • DataGhost.com
    Re: DryOS - some success
    « Reply #15 on: 01 / January / 2008, 14:42:22 »
    I just had a look at the dump and disassembly and I noticed that 0x9F3CC is used for pointing to mykbd_task, 0xA008C for task_blinker and 0x9D0E5 for core_spytask. Now, the last one looked a bit strange, since the address is not 4-byte aligned. It was only after examining the dump that I saw 2-byte instructions for that particular part of code, at 0x9D0E4, that I remembered that odd addresses are used to identify thumb code. I also remembered some reference to 'this code is supposed to be in thumb mode', and the Makefile in core/ reflects that. I guess _CreateTask has some sort of problem with thumb addresses/code (doesn't exchange properly?), but I'm not that experienced with ARM ASM or this firmware in general. As I'm typing this, I'm loading the A720 firmware in IDA to see what it's supposed to do, but it's still processing.

    Edit: Oh, yes. What GrAnd said, heh.
    by the way, I modified the code a bit so I could locate my stuff in the disassembly (I still had to figure out what was getting loaded where and how to get to it), because I didn't know the dump file was being generated. That's why addresses might be a bit different, if you're concerned :)

    Edit2(late): I just realized that I failed to reply to the rest of your post
    Quote
    The optical soundcard-input wouldn't help much since the firmware-blinker doesn't blink the SPDIF protocol. You will need a phototransistor. It is mentioned in the wiki and some threads in the firmware-dumping section.
    Hm, I thought it could do something with 'analog' signals as well, but as I'm typing this (I never really carefully thought it through) it sounds stupid... well, I don't have an optical-in, so that's probably why I didn't know. I did read about the transistor, but stores are closed for today so I was actually trying to find something else, perhaps an optical mouse, so I could do it today. Unfortunately I don't have one with the right chip, so I'll have to go out and buy some stuff tomorrow, I guess.

    Quote
    What platform?
    Check if the archive I posted compiles ("make fir").
    The arm-gcc that is built using the instructions in the wiki apply a patch to the gcc-sources. In general I suggest we all use the same compiler to minimize differences (different camera models are enough to cope with) and thus reduce the chance of "strange" errors. This way either everybody or nobody gets this error and the situation will be less confusing. You might already know: different versions of a software are support-hell :)
    Linux. It compiles, apart from a lot of warnings and 'dangerous error' every now and then, though it looks like it compiles well. I haven't changed the LED addresses and looked into other to-be-changed things yet, so I haven't yet tested it on my camera. It does compile my own programs, as I said earlier.
    I will look into the newer/uptodate compiler though, and I'll try to set it up somewhere private so I can use them both (all three actually) without too much trouble.

    Quote
    http://search.ebay.com/time? Maybe someone sells his :)
    I doubt I can buy enough time to not have to worry about the upcoming three weeks of exams ;)
    « Last Edit: 01 / January / 2008, 16:16:35 by DataGhost »
    Logged
    *

    Offline jeff666

    • ****
    • 181
    • A720IS
    Re: DryOS - some success
    « Reply #16 on: 02 / January / 2008, 07:25:47 »
    The problem really seems to be the arm/thumb-mode connection. It seemed to be the easiest way to get rid of it by just disabling thumb-code generation at all. Unfortunately the result didn't work either (no further investigation done).

    Next thing I tried was calling a new function in core (turns on/off an LED depending on parameter) to see if any call fails - it does.

    I tried two ways:
    1. calling core-test-function from blinker-task spawn in boot.c
    2. calling core-test-function from startup_my.

    To find out what goes wrong, I load core/main.bin into IDA (additionally to the firmware) and find this:

    Code: [Select]
    startup_my (fraction) (arm-mode):
    seg000:A0054                 MOV     R0, #1
    seg000:A0058                 BL      sub_A1F40 ; core-test-function(1); 1 => turn LED on

    arm-thumb-interconnection-code (arm-mode):
    seg000:A1F40 sub_A1F40                               ; CODE XREF: seg000:A0058p
    seg000:A1F40                 LDR     R12, =0x9D101
    seg000:A1F44                 BX      R12    ; B with mode change

    core-test-function (thumb-mode)
    seg000:9D100 sub_9D100                               ; CODE XREF: sub_A1F40+4j
    seg000:9D100                 PUSH    {LR}      ; LR: return-address of BL in startup_my
    seg000:9D102                 LDR     R2, =0xC02200C8 ; LED addr
    seg000:9D104                 CMP     R0, #1
    seg000:9D106                 BEQ     loc_9D110
    seg000:9D108                 MOVS    R3, #0x44 ; 'D'
    seg000:9D10A
    seg000:9D10A loc_9D10A
    seg000:9D10A                 STR     R3, [R2]
    seg000:9D10C                 POP     {R1}     ; restore LR and ...
    seg000:9D10E                 BX      R1       ; ...continue there (and change mode)
    seg000:9D110 loc_9D110
    seg000:9D110                 MOVS    R3, #0x46 ; 'F'
    seg000:9D112                 B       loc_9D10A

    I guess the problem is the target address in the arm/thumb-interconnection-code. It's 0x9D101 while the function itself starts at 0x9D100. So the address is wrong in two ways. a) it points to the wrong entry-point and b) it's not aligned.

    Since the other addresses are correct I don't think it's a general offset-problem (caused by wrong load offset in IDA) but rather something the linker messes up.

    Any Ideas?

    Cheers.
    Logged
    *

    Offline DataGhost

    • ****
    • 314
    • EOS 40D, S5IS
      • DataGhost.com
    Re: DryOS - some success
    « Reply #17 on: 02 / January / 2008, 10:45:36 »
    Nope, that's what I thought initially, before I remembered the actual meaning of that instruction. The address is correct, the non-alignedness (in conjunction with BX over B) indicates thumb mode at the destination :) You may find this document useful, at least I did.

    Quote
    BX
    Branch, and optionally exchange instruction set.

    Syntax
              BX{cond} Rm

       where:
              cond is an optional condition code (see Conditional execution on page 4-4).
              Rm is an ARM register containing the address to branch to.
       
          Bit 0 of     Rm is not used as part of the address.
          If bit 0 of     Rm is set, the instruction sets the T flag in the CPSR, and the 
          code at the destination is interpreted as Thumb code.
          If bit 0 of     Rm is clear, bit 1 must not be set.
    This way, the instructions MUST be 4-byte aligned for ARM mode and 2-byte aligned for Thumb mode :) I'm just not sure how the interwork thing actually works yet, but I guess you'll need to look into that to get this working.
    Logged
    *

    Offline jeff666

    • ****
    • 181
    • A720IS
    Re: DryOS - some success
    « Reply #18 on: 02 / January / 2008, 13:05:28 »
    Quote
    Nope, that's what I thought initially, before I remembered the actual meaning of that instruction. The address is correct, the non-alignedness (in conjunction with BX over B) indicates thumb mode at the destination :)

    Didn't know that. I already wondered how the CPU finds out which mode to switch to.

    Quote
    I'm just not sure how the interwork thing actually works yet, but I guess you'll need to look into that to get this working.

    The code generated by gcc looks ok. I also tried to compile the same code completely in arm-mode. The result looked a bit different but I had expected it to work - still it didn't. I'm getting the idea that the code is overwritten at some point. I'll see what I can find out.

    Cheers.
    Logged
    *

    Offline jeff666

    • ****
    • 181
    • A720IS
    Re: DryOS - some success
    « Reply #19 on: 02 / January / 2008, 17:53:35 »
    That was a nasty one, but I got the core-problem fixed.

    After finding out that the code should work I suspected that it must have gotten "damaged" in the camera. Since the OS runs I should have file access available and tried some things with open, Open (yeah, they're different), Fopen_Tut and related, just to find out that none of them worked straight forward. Fortunately I found an existing function which does exactly what I wanted - write a portion of memory into a file (and shut down the camera afterwards, investigation why has to be done).

    Now I had a file which I expected to contain the memory location that (roughly) contains CHDK code. I was a little confused to find zeroed portions of memory where CHDK should start so I made a dump of the firmware (0xFFC00000+4M) to check if the function did what I expected and it did. So obviously code has been deleted.

    As a quick fix I relocated CHDK up by 64k, pushed the memory pool start up as well and core works now - even in thumb mode :)

    I'll check if gui works now.

    Quote
    It compiles, apart from a lot of warnings and 'dangerous error' every now and then, though it looks like it compiles well.

    If it looks like it runs well, everybody will be happy :)

    Quote
    I haven't changed the LED addresses and looked into other to-be-changed things yet, so I haven't yet tested it on my camera. It does compile my own programs, as I said earlier.
    I will look into the newer/uptodate compiler though, and I'll try to set it up somewhere private so I can use them both (all three actually) without too much trouble.

    The win32-compiler built my test programs (blinker and such), too, but didn't build CHDK. Though it was probably not the compiler itself.
    To compile the "official" chdk-arm-gcc just follow the instructions in the wiki. They're very straight forward and simple.


    Btw, GrAnd, still think it can't be done? ;)

    Cheers.

    Logged
    Pages: 1 [2] 3 4 5 6 7 ... 23   Go Up
    « previous next »
     

    Related Topics