Pages: Prev 1 ... 4 5 [6] 7 8 ... 15 Next   Go Down
  Print  
Author Topic: DryOS - some success  (Read 57863 times)
0 Members and 1 Guest are viewing this topic.
DataGhost
Developers
Sr. Member
****

Karma: +64/-0
Offline Offline

Posts: 312


EOS 40D, S5IS


« Reply #75 on: 09 / January / 2008, 16:54:48 »

Well, I shot a few RAW images from my S5 Smiley I've never really worked with RAW images before (let alone dcraw), so I don't know if I'm doing it right... but it seems like I only get 'good' output when I set the top margin to 15 px (it should be 16 px according to the offset relative to the generated .jpg file, but I don't get *ANY* green except for in overexposed areas in that case, my image will be purple and I lose quite a bit of information when I do manage to get the colors somewhat correct). Actually, I do not see a positioning difference between 15 and 16 but I do still see a difference between these and the 'reference'. This is possibly caused by the sharpening the camera applies, though. Judge for yourself, I'll try setting it to 17 in a bit Smiley Anyway, even then, it's very green (is that supposed to happen?). It can be fixed by telling dcraw where to find something white, else it's a bit hard to fix... but I just need minor modifications afterwards, so that's fine, I guess Smiley .

RAW image converted to jpeg, set the white balance to something sensible, NO noise reduction (notice the hot pixels, especially in the dark areas), no shapening, and I made it a bit brighter.
'Reference' jpg image made by the camera, auto white balance (it's way off) and quite dark.
Settings: ISO 80, 1.6", F8. I should have shot with a lower F-number, I guess.. but it's a bit hard to tell through a live hexdump :p

About the RAW conversion strangeness:
top_margin=15 No whitebalance, no postprocessing (wall color is really close though, better than in my processed image, but the screen should be #FFFFFF)
top_margin=16 No whitebalance, no postprocessing. I don't know what causes this but it's clearly unusable. Possibly a misalignment (starts at byte 2 instead of byte 0 or vice versa?) but I'd expect it to be correct for even offsets, not for odd ones..


Adding support in dcraw was straightforward, just search for 'S3 IS', copy that part and paste it right below there, changing the appropriate values, so you'll end up with the following code:
Quote
} else if (!strcmp(model,"PowerShot S5 IS")) {
    height = 2448;
    width  = 3264;
    raw_height = 2480;
    raw_width  = 3336;
    top_margin  = 15;
    left_margin = 24;
    load_raw = &CLASS canon_a5_load_raw;
I *think* these margin values are correct, but it's mostly trial-and-error.

Also, you'll need to add the filesize to the table (located elsewhere, search for "S3 IS" again) so the file can be identified
Quote
    { 10341600, "Canon",    "PowerShot S5 IS",0 },

This might cause a problem though, but it's just minor. The offsets, sizes and such appear to be exactly the same as in the A720, so the only problem this will cause is that A720 files are identified as S5IS files. This shouldn't really matter, though, as I expect both cameras to generate the same data.
The above mentioned problems could also be caused by the canon_a5_load_raw function, but as I said, I don't know too much about RAW photography and I especially don't know how to interpret the sensor data (yet), so this will have to do.
Logged
ewavr
Developers
Hero Member
****

Karma: +138/-1
Offline Offline

Posts: 652

A710IS


« Reply #76 on: 09 / January / 2008, 17:59:06 »

Good work!
Can you also upload RAW file (zipped)?
Logged
DataGhost
Developers
Sr. Member
****

Karma: +64/-0
Offline Offline

Posts: 312


EOS 40D, S5IS


« Reply #77 on: 09 / January / 2008, 18:59:16 »

Oh yes, of course. That's what I forgot Smiley There you go. It didn't compress that well, but it at least shaves 3MB off.

Edit: my dcraw.c might be useful as well.
« Last Edit: 09 / January / 2008, 19:10:29 by DataGhost » Logged
jeff666
Developers
Full Member
****

Karma: +42/-8
Offline Offline

Posts: 181


A720IS


« Reply #78 on: 10 / January / 2008, 00:06:22 »

Quote
my dcraw.c might be useful as well.

Very good.

While you are on it, could you have a look at the NR-handling? I'm back at work and don't have much time anymore.

If we control NR, we should have a (mostly) complete port of trunk and can start to port the changes to a newer branch with more features.

Cheers.
Logged
DDMan
Newbie
*

Karma: +0/-0
Offline Offline

Posts: 12

A720 IS


« Reply #79 on: 10 / January / 2008, 00:10:08 »

Hi Jeff,
could you maybe upload your latest CHDK port for the A720, so people like me can test things like DataGhosts findings.
I'm very anxious to see the port and will help as much as I can. By the way I'm a software developer as well, so if time allows it I will dive into programming too. ;-)
Thank you in advance!

Cheers and greetz from Fellbach at night ;-)
Logged

A720 IS Firmware 1.00c
DataGhost
Developers
Sr. Member
****

Karma: +64/-0
Offline Offline

Posts: 312


EOS 40D, S5IS


« Reply #80 on: 10 / January / 2008, 00:45:36 »

While you are on it, could you have a look at the NR-handling? I'm back at work and don't have much time anymore.

If we control NR, we should have a (mostly) complete port of trunk and can start to port the changes to a newer branch with more features.
Yeah... about that....  Embarrassed I'm cheating. For starters, the keyboard stuff seems to be a lot different, I haven't figured it out anyway (some functions don't exist and the layout seems to differ from both the S3 and the A720, though I could be terribly wrong). The only keys I can see now is the shoot key (none, half and full) and the power-'key' (the twist-wheel thingy). Both are still handled by the OS, I can use the shoot key for browsing through my on-screen hexdump if I'm using a locked memory card, because it won't take a picture in that case.
Anyway, what I did was... add a LONG delay to the boot process, so I have time to actually switch my SD card. As you can see in this short clip, I put in an 8GB SDHC card (not locked) and it re-reads it and that works just fine. I can take pictures now Smiley Anyway, I keep the gui code running from spytask and fetch the shoot-button status. When I press it, it'll reset a counter and it will keep resetting it until I release that button. I coded it so that it would save the contents of the RAW buffer as soon as I release it (counter > 1) :p So what I did was... shoot an image at a couple of seconds shutter time and release the shoot-button as soon as it's done shooting and busy calculating a dark frame. It'll save the buffer before that's done, and the OS will output a jpeg file after that.
I'm really not anywhere near actually hooking the shooting functions yet, so I can't really have a look, sorry. I'm still in the middle of my exams so I don't have a lot of time, either Smiley

Anyway, in the meantime I got an idea... I can create several partitions on an SD card. I have 2 8GB SDHC cards and a 32MB card I got from canon (and several 2GB miniSD cards, but they're really for my phone). Now I found out that the OS will read only one partition ( http://dataghost.com/chdk/S5_parse_MBR.txt ) and completely ignore the other, so I'll have to do something about that. Now, as you can see in the code, it should really not be that hard to alter it so that it'll read the second partition instead of the first one (offset everything by 0x10). The thing is, I ran out of time for today and that code seems to be called at least from 5 entries in a jump table just above there, I haven't traced them back yet. If this works out, I can create a small FAT16 partition to boot CHDK and a large FAT32 partition which is then read by the camera after loading the code Smiley That way, it'll be possible to use large SDHC cards with CHDK, as they're becoming increasingly popular, I think.
Logged
intrinsic
Newbie
*

Karma: +3/-1
Offline Offline

Posts: 29

S5IS


« Reply #81 on: 10 / January / 2008, 01:16:20 »

I'm really not anywhere near actually hooking the shooting functions yet, so I can't really have a look, sorry. I'm still in the middle of my exams so I don't have a lot of time, either Smiley

Given your lack of time would you please post your code so far, that way we can continue work on the S5 without having to duplicate your efforts.

Thanks,
intrinsic
Logged
DataGhost
Developers
Sr. Member
****

Karma: +64/-0
Offline Offline

Posts: 312


EOS 40D, S5IS


« Reply #82 on: 10 / January / 2008, 01:20:45 »

Yeah, sure, but it'll be tomorrow at best (I'm going to get some sleep now). I also have an exam tomorrow and didn't study that much, so I hope I can quickly clean up the code a bit because it's a real mess, unless that's fine with you Smiley I'll also add some comments in a few places.
Logged
intrinsic
Newbie
*

Karma: +3/-1
Offline Offline

Posts: 29

S5IS


« Reply #83 on: 10 / January / 2008, 01:23:19 »

Yeah, sure, but it'll be tomorrow at best (I'm going to get some sleep now). I also have an exam tomorrow and didn't study that much, so I hope I can quickly clean up the code a bit because it's a real mess, unless that's fine with you Smiley I'll also add some comments in a few places.

Excellent, I look forward to your post ;oD
Logged
DataGhost
Developers
Sr. Member
****

Karma: +64/-0
Offline Offline

Posts: 312


EOS 40D, S5IS


« Reply #84 on: 10 / January / 2008, 21:25:23 »

I'm sorry I haven't posted anything yet... I got this idea (regarding bootable SDHC cards) so I traced a lot of stuff and figured out that I wrongfully assumed the code was being called outside of the already-hooked stuff. With the current code, I'm looking at 7 more subroutines I need to bypass, 14 almost-worst-case and it's hooked *somewhere* in the absolute worst case. I already documented a bit of this in boot.c (I should really get some more formal documentation going, outside source files Tongue ), so you can even have a look at that if you want.
Anyway, because of that, I haven't really paid much attention to cleaning up the code a bit, making it readable, adding comments, commenting out junk... so I'll have a quick break and get to that straight away.

By the way, the next problem is that Windows completely ignores an SD-card partition's type and opens up the first one (and refuses to open the second one in any way), so I guess that'll be Linux-only or through USB.

Edit: there we go, http://stack.dataghost.com/chdk-dryos-s5is-20080110.tar.bz2. I hope I cleaned it up well enough, this should at least run Smiley
« Last Edit: 10 / January / 2008, 22:51:18 by DataGhost » Logged
jeff666
Developers
Full Member
****

Karma: +42/-8
Offline Offline

Posts: 181


A720IS


« Reply #85 on: 10 / January / 2008, 23:30:22 »

Quote from: DDMan
could you maybe upload your latest CHDK port for the A720, so people like me can test things like DataGhosts findings.

I just removed some annoying debug-stuff and rebuilt the code with the correct addresses for RAW-support. It's just for testing, though.

If you enable raw in the ALT-Menu, a file called "rawtest.crw" will be saved to the card. No counter or anything like that, yet, but if you get a bunch of original Canon 16mb SD-cards more than one file won't fit anyway Smiley

First the code: http://www.zshare.net/download/6386888cd05c7f/

Next an example, 4" exposure, F between 6 and 8 (don't remember exactly). It's a picture of my "chaos-dominates-order"-corner Big Grin
The lighting is a little difficult, since the scene is lit by a normal neon-lamp behind the sofa and the black-light lamp on it. The black border on the right is the end of the wall.

raw: http://img341.imageshack.us/my.php?image=rawtestxh7.jpg
cam-jpg: http://img184.imageshack.us/my.php?image=img1191cg1.jpg

Both images shot with 8mpx. Raw converted using DataGhost's dcraw-patch. Scaled down with gimp.
Here's the output of dcraw:
Code:
-> dcraw -v rawtest.crw
Loading Canon PowerShot S5 IS image from rawtest.crw ...
Scaling with black 36, multipliers 1.582762 1.000000 1.718605 1.000000
AHD interpolation...
Converting to sRGB colorspace...
Writing data to rawtest.ppm ...

I guess dcraw's whitebalance failed this one. The actual scene looks much more like the camera's jpeg.

Quote
I'm very anxious to see the port and will help as much as I can. By the way I'm a software developer as well, so if time allows it I will dive into programming too. ;-)

I could use someone who finds the entrypoint for "GetParameterData". We need it for the filecounter. You made yourself familiar with IDA? Smiley

Quote from: DataGhost
Edit: there we go, http://stack.dataghost.com/chdk-dryos-s5is-20080110.tar.bz2. I hope I cleaned it up well enough, this should at least run Smiley

Great. Now I don't have to write a hexdump-function myself Smiley

Edit:
Just remembered a glitch I noticed when saving RAW.
I had the resolution of the jpeg set to 5mpx for the first try. The raw-file was full size (as expected), but the upper 10% of the image were overwritten with garbage. After changing to 8mpx I got the result I posted above. Probably the resolution needs to be set to maximum.

Cheers.
« Last Edit: 10 / January / 2008, 23:36:55 by jeff666 » Logged
digit
Newbie
*

Karma: +0/-0
Offline Offline

Posts: 13


« Reply #86 on: 11 / January / 2008, 00:10:30 »

Is the full IDA Pro version required?  Or would I be able to get away with the freeIDA one?
Logged
jeff666
Developers
Full Member
****

Karma: +42/-8
Offline Offline

Posts: 181


A720IS


« Reply #87 on: 11 / January / 2008, 00:15:10 »

Quote from: digit
Is the full IDA Pro version required?  Or would I be able to get away with the freeIDA one?

Unfortunately the free one doesn't support ARM (just x86).
Logged
ewavr
Developers
Hero Member
****

Karma: +138/-1
Offline Offline

Posts: 652

A710IS


« Reply #88 on: 11 / January / 2008, 01:08:13 »

Hi all,
I got for some days A720 and played with previous jeff's sources.
I made:
-add  functions: mkdir, set(get)parameterdata (and correct RAW numbering)
-keyboard now works correctly (no shooting mode change in <alt> mode)
-RAW shooting with noise reduction

Still have a lot to do: histogram, scripts etc....

src: http://ewavr.nm.ru/hdk/A720_001.ZIP (makefile's slightly modified for compliation under Windows)

edit: sorry, histogram works
« Last Edit: 11 / January / 2008, 01:18:42 by ewavr » Logged
DataGhost
Developers
Sr. Member
****

Karma: +64/-0
Offline Offline

Posts: 312


EOS 40D, S5IS


« Reply #89 on: 11 / January / 2008, 01:40:00 »

Histogram still crashes my camera, but that's probably because I haven't looked into it yet Smiley I had more pressing issues that needed addressing :p

Like booting from a big SDHC card which requires FAT32. I managed to dump a function-call graph from IDA, so that greatly simplified locating the functions. Canon either planned this or it's just a very bad coincidence, but I only got it right on the last try. I first tried the shortest (number-of-subroutines-wise) paths to that MBR code, and of course, it turned out that the path required was the last one I actually tried, so I ended up cloning ALL 15 subroutines, good for a whopping 1500 lines and the assembler complaining it couldn't properly place the literal pool(s) by itself. I'll look into reducing that again, as I think I 'only' need 12 subroutines (the 3 small ones aren't necessary, the others are Sad ), unless someone knows a better way. In my opinion, it's worth it Smiley

jeff666: do be sure to read the comments if you actually want to use the prev/next functions in the hexdump. Actually, while I'm writing this... you do have proper keyboard support so you probably don't need my stuff to make it compatible with the shoot button (which triggers two 'buttons' when I want only one). If you still want/need to use that code, however, I must tell you that I'm not entirely sure anymore what I changed exactly. It should work from the comments, but it might need adjusting Smiley
About your raw image: yours also looks a bit greenish! What options did you pass to dcraw? I only use -k <somevalue>, I don't really know how the others are going to help me. I did use the whitebalance functions, but I just keep getting some sort of fog over my image unless I adjust it by hand. Maybe I'm just not used to postprocessing RAW images, though, you seem to be quite cool about that whitebalance.
Logged
Pages: Prev 1 ... 4 5 [6] 7 8 ... 15 Next   Go Up
  Print  
 
Jump to: