chdk in the DIGIC6 world - page 7 - General Discussion and Assistance - CHDK Forum
« previous next »
Pages: 1 2 3 4 5 6 [7] 8 9 10 11 12 ... 21   Go Down

chdk in the DIGIC6 world

  • 201 Replies
  • 170156 Views
*

Offline srsa_4c

  • ******
  • 4451
Re: chdk in the DIGIC6 world
« Reply #60 on: 22 / May / 2016, 13:54:20 »
Advertisements
I have revisited a dump made from the 0x80a00000 area, and found that...
... it's now very likely that the core with the MZRM (etc.) firmware is an Xtensa.
It was apparent from the beginning that there are chunks of addresses in this dump. For example
(LE byte order, stupid HxD can't change endianness)
Code: [Select]
Offset(h) 00       04       08       0C

000001D0  00000000 00000000 5094A280 7C94A280
000001E0  A894A280 00000000 00000000 F096A280
000001F0  2097A280 4C97A280 7897A280 A897A280
00000200  D497A280 E094A280 1095A280 3C95A280
00000210  6C95A280 A495A280 D495A280 0496A280The data found at the above addresses has made me suspicious.
Code: [Select]
Offset(h) 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

00029450  36 41 00 0C 8A B1 1C D0 C1 BE FE B8 0B 81 53 D0  6A..Š±.ĐÁľţ¸..SĐ
00029460  E0 08 00 B1 45 D0 B8 0B 8C 4B A1 B9 FE E0 0B 00  ŕ..±Eи.ŚKˇąţŕ..
00029470  A2 22 00 E5 FA 02 22 A0 00 90 00 00              ˘".ĺú." ....

00029470                                      36 41 00 0C              6A..
00029480  8A B1 11 D0 C1 B4 FE B2 2B 00 81 48 D0 E0 08 00  Š±.ĐÁ´ţ˛+..HĐŕ..
00029490  B1 3A D0 B8 0B 8C 4B A1 AF FE E0 0B 00 65 F7 02  ±:и.ŚKˇŻţŕ..e÷.
000294A0  0C 02 1D F0 00 00 00 00                          ...đ....

000294A0                          36 41 00 0C 8A B1 06 D0          6A..Š±.Đ
000294B0  C1 AA FE B8 0B 81 3D D0 E0 08 00 B1 2F D0 B8 0B  ÁŞţ¸..=Đŕ..±/и.
000294C0  8C 4B A1 A5 FE E0 0B 00 B2 22 01 C2 12 04 D2 12  ŚKˇĄţŕ..˛".Â..Ň.
000294D0  05 A2 22 00 25 F7 02 A0 2A 20 1D F0 00 00 00 00  .˘".%÷. * .đ....

000294E0  36 41 00 0C 8A B1 F8 CF C1 9D FE B8 0B 81 2F D0  6A..Š±řĎÁťţ¸../Đ
000294F0  E0 08 00 B1 21 D0 B8 0B 8C 4B A1 98 FE E0 0B 00  ŕ..±!и.ŚKˇ.ţŕ..
00029500  A8 02 B8 12 81 97 FE E0 08 00 0C 02 1D F0 00 00  ¨.¸..—ţŕ.....đ..
Start is typically 0x36 0x41 0x00, end is usually 0x1d 0xf0 plus zero bytes. Start is always 4 byte aligned.
I then looked up the RET instructions in the "Xtensa Instruction Set Architecture (ISA) Reference Manual" (found via google).
And found that 0xf01d (the manual is using LE byte order) is indeed 'retw.n'.
Built an objdump binary that knows this architecture (using this script).

Below disassemblies are using big endian byte order for the instruction hex dumps, but specifying endianness for objdump doesn't change the resulting disassembled instructions.

Code: [Select]
80a29450:   364100                  entry   a1, 32
80a29453:   0c8a                   movi.n   a10, 8
80a29455:   b11cd0                  l32r   a11, 0x80a1d4c8
80a29458:   c1befe                  l32r   a12, 0x80a28f50
80a2945b:   b80b                   l32i.n   a11, a11, 0
80a2945d:   8153d0                  l32r   a8, 0x80a1d5ac
80a29460:   e00800                  callx8   a8
80a29463:   b145d0                  l32r   a11, 0x80a1d578
80a29466:   b80b                   l32i.n   a11, a11, 0
80a29468:   8c4b                   beqz.n   a11, 0x80a29470
80a2946a:   a1b9fe                  l32r   a10, 0x80a28f50
80a2946d:   e00b00                  callx8   a11
80a29470:   a22200                  l32i   a10, a2, 0
80a29473:   e5fa02                  call8   0x80a2c420
80a29476:   22a000                  movi   a2, 0
80a29479:   900000                  retw

80a2947c:   364100                  entry   a1, 32
80a2947f:   0c8a                   movi.n   a10, 8
80a29481:   b111d0                  l32r   a11, 0x80a1d4c8
80a29484:   c1b4fe                  l32r   a12, 0x80a28f54
80a29487:   b22b00                  l32i   a11, a11, 0
80a2948a:   8148d0                  l32r   a8, 0x80a1d5ac
80a2948d:   e00800                  callx8   a8
80a29490:   b13ad0                  l32r   a11, 0x80a1d578
80a29493:   b80b                   l32i.n   a11, a11, 0
80a29495:   8c4b                   beqz.n   a11, 0x80a2949d
80a29497:   a1affe                  l32r   a10, 0x80a28f54
80a2949a:   e00b00                  callx8   a11
80a2949d:   65f702                  call8   0x80a2c414
80a294a0:   0c02                   movi.n   a2, 0
80a294a2:   1df0                   retw.n
80a294a4:   000000                  ill

80a294a8:   364100                  entry   a1, 32
80a294ab:   0c8a                   movi.n   a10, 8
80a294ad:   b106d0                  l32r   a11, 0x80a1d4c8
80a294b0:   c1aafe                  l32r   a12, 0x80a28f58
80a294b3:   b80b                   l32i.n   a11, a11, 0
80a294b5:   813dd0                  l32r   a8, 0x80a1d5ac
80a294b8:   e00800                  callx8   a8
80a294bb:   b12fd0                  l32r   a11, 0x80a1d578
80a294be:   b80b                   l32i.n   a11, a11, 0
80a294c0:   8c4b                   beqz.n   a11, 0x80a294c8
80a294c2:   a1a5fe                  l32r   a10, 0x80a28f58
80a294c5:   e00b00                  callx8   a11
80a294c8:   b22201                  l32i   a11, a2, 4
80a294cb:   c21204                  l16ui   a12, a2, 8
80a294ce:   d21205                  l16ui   a13, a2, 10
80a294d1:   a22200                  l32i   a10, a2, 0
80a294d4:   25f702                  call8   0x80a2c448
80a294d7:   a02a20                  or   a2, a10, a10
80a294da:   1df0                   retw.n
80a294dc:   000000                  ill
Seems legit to me. Here's some demo disassembly, found on the web: http://kozos.jp/books/asm/sample/xtensa-elf.d

Blind disassembly is similarly "successful" as on thumb2, fortunately small chunks can be disassembled like this:

objdump -D --adjust-vma=0x80a00000 -b binary --start-address=0x80a294a8 --stop-address=0x80a294df -mxtensa 80a00000.bin >80a294a8.d

Above doesn't mean I now understand this architecture or its disassembly.

Those with recent IDA can probably use this: https://github.com/themadinventor/ida-xtensa (which is specific to the core in ESP8266).
Logged
*

Offline reyalp

  • ******
  • 14125
Re: chdk in the DIGIC6 world
« Reply #61 on: 22 / May / 2016, 15:12:05 »
Nice work :D

Pity capstone doesn't support this architecture :(

One thing I wondered about was whether addresses seen by this core are the same as seen by the main CPU. Looks like they are, which is nice. The start of my 0x80a0000 dump mostly bff2* addresses.

I assume 0xbff00000 is the initialized (.data) segment. 0xbff20000 looks like it might be more xtensa code.

This also should help explain some MPU regions on the main arm CPU:
Code: [Select]
MPU region 2 base 0x80000000
  Base address         0x80000000 -2147483648
MPU region 2 size & enable 0x0000003D
  Enabled              0x1 1
  Size                 0x1E 30 [2G]
  -                    0x0 0
  Sub-regions disabled 0x0 0 [00000000]
MPU region 2 access control 0x00001105
  Region attributes    0x5 5 [Shareable device; Shareable]
  -                    0x0 0
  Access permission    0x1 1 [P:RW U:--]
  -                    0x0 0
  Execute never        0x1 1
MPU region 3 base 0xBFE00000
  Base address         0xBFE00000 -1075838976
MPU region 3 size & enable 0x00000029
  Enabled              0x1 1
  Size                 0x14 20 [2M]
  -                    0x0 0
  Sub-regions disabled 0x0 0 [00000000]
MPU region 3 access control 0x00000124
  Region attributes    0x24 36 [Inner Non-cacheable; Outer Non-cacheable; Shared]
  -                    0x0 0
  Access permission    0x1 1 [P:RW U:--]
  -                    0x0 0
  Execute never        0x0 0

edit:
One obvious question is how this core is configured. From strings in the firmware it seems safe to assume it's configured with exception handling and an FPU at least.

I don't see any obvious indication whether there are any caches, MPU, DSP extensions etc. How it relates to the TAKUMI GPU is also unclear.

The GK.log generated with D6 romlogs appears to come from this core.

Another question is whether there is some kind of console that allows interacting with dryos on this core. From the strings, it seems like there could be, there are references to /term and things that look like CLI commands.

The following look suggestive
Code: [Select]
fcb47e64 cpusel
fcb47e6c [-marius|-omar|-zico] <-podmain|-podsub>
fcb47e98 -marius
fcb47ea0 -omar
fcb47ea8 -zico
fcb47eb0 option error.
fcb47ec0 -podmain
fcb47ecc -podsub
fcb47ed4 Usage:
fcb47edc  debsio %s %s
fcb47eec %s is not found.
fcb47f00 DEBSIO
fcb47f08 debsio
(I'm guessing debsio is DEBug Serial IO.)

edit:
Quote
I wish I had names for these different ROMs...
IMO, debsio above gives names of the cores running DryOS
Marius = main ARMv7 CPU
Omar = ??? (arm?)
Zico = Xtensa / GPU interface (mzrm.. = Marius/Zico... ?)

The respective Dryshell prompts are Dry[MARIUS]>, Dry[OMAR]> and Dry[ZICO]>, where pre-digic 6 cams just have Dry>
« Last Edit: 22 / May / 2016, 20:28:16 by reyalp »
Logged
Don't forget what the H stands for.
*

Offline reyalp

  • ******
  • 14125
Re: chdk in the DIGIC6 world
« Reply #62 on: 29 / May / 2016, 18:17:20 »
I've been thinking about adding d6 ports to the autobuild. My feeling is that sx280, g7x and sx60hs are stable enough be useful to the broader user community, and outside of the display related issues, are pretty much at the level of other existing ports.

Missing features include
* Zebra
* histogram
* Custom auto-ISO (depends on histogram)
* MD (does this work on sx280 with current live buffer?)
* Script get_live_histo
* Edge overlay
* PTP live view

Some options

1) Enable as-is
2) Add code to disable the features that are not yet supported
3) Don't enable in autobuild until more issues are resolved

My current preference is #2. Using an ifdef to disable the menu items and possibly default the cfg options seems like it shouldn't be too bad. Not sure about script functions.

I'd prefer to avoid #1, since it's likely to cause frustration and duplicate bug reports.

Another question is whether the autobuild should include FI2 files. Being able to manually load to make the card bootable is convenient, but all these cameras crash in some cases if you try to shoot after booting an FI2.
Logged
Don't forget what the H stands for.
*

Offline srsa_4c

  • ******
  • 4451
Re: chdk in the DIGIC6 world
« Reply #63 on: 29 / May / 2016, 20:50:15 »
Quote from: reyalp on 29 / May / 2016, 18:17:20
I've been thinking about adding d6 ports to the autobuild. My feeling is that sx280, g7x and sx60hs are stable enough be useful to the broader user community, and outside of the display related issues, are pretty much at the level of other existing ports.

Missing features include
* Zebra
* histogram
* Custom auto-ISO (depends on histogram)
* MD (does this work on sx280 with current live buffer?)
* Script get_live_histo
* Edge overlay
* PTP live view

Some options

1) Enable as-is
2) Add code to disable the features that are not yet supported
3) Don't enable in autobuild until more issues are resolved
I'd tend to say 3 but ... I can't predict when those missing features will become implemented. So, maybe it's OK to make these ports officially available.
Question: how should we call these? Is there a designation more appropriate than 'prealpha'?
Quote
My current preference is #2. Using an ifdef to disable the menu items and possibly default the cfg options seems like it shouldn't be too bad. Not sure about script functions.
If we choose to release, then this is probably the way to go. Since THUMB_FW can also used in module code, the non-working functions can be disabled made properly non-working (no framebuffer access, etc.).
Quote
Another question is whether the autobuild should include FI2 files.
Definitely not. Some people seem to prefer the firmware update boot for some reason, they would expect it to be fully usable.

One more thing to consider: the current (horrible) drawing code uses a computed palette (and not lookup tables), which is not really optimal.
If this ever gets reworked, the palette will likely change more or less.

I also need to fix a few display related things in the sx280 port. Some dimensions are incorrect and could in theory corrupt memory if zebra is on.
I don't think MD works (but I'm not sure if I re-tried it when I semi-fixed display routines for the PTP live view demo implementation).
Logged
*

Offline reyalp

  • ******
  • 14125
Re: chdk in the DIGIC6 world
« Reply #64 on: 31 / May / 2016, 01:11:21 »
Quote from: srsa_4c link=topic=11316.msg128797#msg128797
Question: how should we call these? Is there a designation more appropriate than 'prealpha'?
I don't know, "prealpha" makes me think "mostly not working" rather than "some features aren't available" but I haven't come up with a better word.
Quote
I don't think MD works (but I'm not sure if I re-tried it when I semi-fixed display routines for the PTP live view demo implementation).
Yeah, looking at the code, I don't think it would work. It might trigger on motion, but the values will be all wrong.

I spent some time poking around the firmware display code and ram dumps, but no breakthroughs yet.

I found the uyvy_old format viewport buffers on g7x. They are lower res than the main viewport: 640x426, and there are 8 of them. Just before that in RAM, there are also 8 360x240 in the same format. If it's really an 8 frame circular buffer, not using the most recent one would add a lot of latency.

There appear to be 4 normal viewport buffers (720x480 on 736x480 buffer, uyvy_d6). Playback mode viewport only appears to be available in this format, at different addresses from rec.

I also found the 32 bit RGBA version of the bitmap, which appears to be 720x480 on a 960x480 buffer.

Thinking about MD, it shouldn't be too hard to make it work (except the grid display) it just needs different YUV decoding. Having different d6 ports using different different formats would complicate things.
Logged
Don't forget what the H stands for.
*

Offline srsa_4c

  • ******
  • 4451
Re: chdk in the DIGIC6 world
« Reply #65 on: 03 / June / 2016, 20:39:38 »
I tried finding movie_status for the sx280. So far, none of the candidates behave like those on pre-D6 ports. And, unfortunately, this is a problem, because several parts of the CHDK code are kind of hardwired to expect the old values.
Constants are defined in include/shooting.h which has this:
Code: [Select]
// Note: used in modules and platform independent code.
// Do not add platform dependent stuff in here (#ifdef/#endif compile options or camera dependent values)Affected parts include:
- CAM_VIDEO_CONTROL feature
- is_video_recording()
- set_movie_status and get_movie_status script functions, including any scripts that use them
Logged
*

Offline reyalp

  • ******
  • 14125
Re: chdk in the DIGIC6 world
« Reply #66 on: 03 / June / 2016, 22:56:55 »
Quote from: srsa_4c on 03 / June / 2016, 20:39:38
I tried finding movie_status for the sx280. So far, none of the candidates behave like those on pre-D6 ports. And, unfortunately, this is a problem, because several parts of the CHDK code are kind of hardwired to expect the old values.
Constants are defined in include/shooting.h which has this:
Code: [Select]
// Note: used in modules and platform independent code.
// Do not add platform dependent stuff in here (#ifdef/#endif compile options or camera dependent values)Affected parts include:
- CAM_VIDEO_CONTROL feature
- is_video_recording()
- set_movie_status and get_movie_status script functions, including any scripts that use them
I'm in favor of turning directly variable accesses like this into functions. For old cameras it could just get/set the existing variable, new ones could re-map the values.

It's actually kind of amazing that so many values have stayed stable for so long.
Logged
Don't forget what the H stands for.
*

Offline srsa_4c

  • ******
  • 4451
Re: chdk in the DIGIC6 world
« Reply #67 on: 05 / June / 2016, 19:21:43 »
Quote from: reyalp on 03 / June / 2016, 22:56:55
I'm in favor of turning directly variable accesses like this into functions. For old cameras it could just get/set the existing variable, new ones could re-map the values.
Attached is my (sketchy) try to allow using a movie_status variable that behaves differently. All 3 candidates I found so far on the sx280 behave like: 0 when not recording, nonzero (usually 1) while recording.
The patch assumes this behaviour for the alternative movie_status. I don't know how future-proof this is.

I'm doing this because I'd like to eventually enable the movie AF related stuff on the sx280, which won't work until is_video_recording() becomes usable.

Quote from: reyalp on 03 / June / 2016, 22:56:55
It's actually kind of amazing that so many values have stayed stable for so long.
There's more stuff that relies on evaluation of Canon variables directly in CHDK core (detection of Canon menus and various states for example), fortunately those are still working.
Logged
*

Offline reyalp

  • ******
  • 14125
Re: chdk in the DIGIC6 world
« Reply #68 on: 11 / June / 2016, 19:30:36 »
Quote from: srsa_4c on 05 / June / 2016, 19:21:43
Attached is my (sketchy) try to allow using a movie_status variable that behaves differently. All 3 candidates I found so far on the sx280 behave like: 0 when not recording, nonzero (usually 1) while recording.
Thanks for doing this.

I've attached a patched with some changes (improvements? Maybe :-[ but feel free to treat these as suggestions)

* Made CAM_VIDEO_CONTROL exclusive with CAM_SIMPLE_MOVIE_STATUS, with #error in camera.h

Control doesn't currently make sense if the variable is only on-off.  This means VIDEO_CONTROL related ifdefs don't have to check both, and ensures menu items don't show up on cameras with SIMPLE. We might theoretically find firmware functions that allow control on these cams, but we can cross that bridge if we get to it.

* Made movie_status only declared in core/shooting.c, converted all other references to function calls. This involves a search/replace in a bunch of platform files.

* Renamed the "simple" variable simple_movie_status, also restricted to shooting.c. This makes it clearer that they refer to different firmware variables (helpful for people looking at stubs, eventual sigfinder addition), and will trigger compiler errors if the  wrong one is used with wrong ifdef.

I haven't tested this exhaustively. I did built d10 and verified that lua get_movie_status() and  get_video_recording() return the expected values. I also built it with a dummy simple_movie_status and verified it always returned the "not recording" values. I built a540 and verified that fast video control still works.

I did not build all the cameras affected by the search/replace platform.

Other notes:
A number of the references to video_status are essentially check "is the camera in a video mode or recording" e.g. the touch camera kbd.c files and some of the lib.c files. A dedicated function for this would be good.

ixus65 has a dummy movie_status variable. Could possibly be converted to simple.

I've never been fond of the whole VIDEO_CONTROL thing, it always seemed a bit too hacky to be useful to me.

Quote
The patch assumes this behaviour for the alternative movie_status. I don't know how future-proof this is.
It seems pretty likely there will be some variable that behaves like this.

Cameras without a dedicate video mode could just use the fact that video recording is indicated by <still mode> | 0x400. This could be implemented with a different value of SIMPLE_MOVIE_STATUS (e.g. 1 = variable, 2 = propcase)

Quote
There's more stuff that relies on evaluation of Canon variables directly in CHDK core (detection of Canon menus and various states for example), fortunately those are still working.
Yes,
Code: [Select]
canon_menu_active!=(int)&canon_menu_active-4
was one I had in mind ;)

edit:
Oops, first version of patch had d10 dummy simple_movie_status. Updated
« Last Edit: 11 / June / 2016, 21:05:02 by reyalp »
Logged
Don't forget what the H stands for.
*

Offline srsa_4c

  • ******
  • 4451
Re: chdk in the DIGIC6 world
« Reply #69 on: 12 / June / 2016, 16:19:10 »
Quote from: reyalp on 11 / June / 2016, 19:30:36
I've attached a patched with some changes (improvements? Maybe :-[ but feel free to treat these as suggestions)
I don't think I see any problems with your version. If you feel like it, you could just check it in.
Quote
ixus65 has a dummy movie_status variable. Could possibly be converted to simple.
I have one of these, but never bothered implementing movie_record_task  :-[ . It's very likely that the 'proper' movie_status is easy to find - I might do that sometimes.

I have attached a version of your patch with the following (for reference only, I wouldn't commit it like that):
- implemented sx280 simple_movie_status (and more because it's taken from another patch)
- fix for 2 division-by-zero conditions in core/gui_osd.c - needed for testing movie status on D6 cams, obviously

With these, I get the CHDK bitrate display while recording, but it keeps displaying zero until I stop the recording. Then, it displays some other values for a few moments (I'm likely not using the most appropriate fw variable as simple_movie_status).
Logged
Pages: 1 2 3 4 5 6 [7] 8 9 10 11 12 ... 21   Go Up
« previous next »
 

Related Topics


SimplePortal © 2008-2014, SimplePortal