CHDK Forum
Using CHDK => RAW Shooting and Processing => Topic started by: SticK on 22 / August / 2012, 17:21:22
-
This post is with gui_context_plus=true in CHDKPTP for the SX110IS.
Please have a look at live viewfinder image quality in the attached. After enabling the lib, the intensity of severe horizontal artifacts improved significantly. However, some lower intensity artifacts still remain (red and violet annotations).
I don't know what the cause of the "out-of-focus" in effect is (yellow annotations), or whether the two might be related. Because I was careful to focus with the camera LCD, I know the images are OK that way.
First row of screen shots is SCENE 1 and second row is SCENE 2, to demonstrate consistent results.
Is there a way to increase the quality/strength of the interpolator, in X and in Y, regardless of CPU cost? If possible, I would like to see the best quality result we can get.
-
A note, if you have the "contextplus" stuff correctly configure, you will see
ContexIsPlus iup:true cd:true
in the command prompt window (not the gui console, the command prompt you started from, or created when you launched chdkptp). If you don't see that, then you don't have it set up correctly.
Re the screenshots:
If you turn aspect correct off, then there is no scaling at all. So any artifacts you see are not related to that.
If you have Viewport 1:1 unchecked, then half the Y values are discarded when converting to RGB. If you check it, then you have exactly the data that was received from the camera, only converted from YUV to RGB.
The YUV->RGB conversion is probably not optimal.
I have to say I don't understand the point of this exercise. Given that typical cameras only have 240 lines, you are never going to get great detail. The current implementation is sufficient for composing a shot or getting a general idea of what is in the scene. It's not great for focusing, but no ~240 line display is going to be.
Note that some cameras have modes that are higher resolution. On my a540, 640x480 video mode changes the live view frame buffer to 704x528. Even if you are shootings stills, you could switch back and forth if you really need the viewport resolution for something. Your camera may or may not do this. Plugging something into the video out connector may also change the viewport resolution.
-
Please allow me to address your most important comment first ...
Quote "I have to say I don't understand the point of this exercise .... you are never going to get great detail."
I felt you might bring this up. If you take a *very* close look at the RemoteCapture image, you will find it is sharper by comparison to the CHDKPTP screen shots. While I agree that most photographic needs may not require that high a performance, my line of thinking is why not provide it if it is being delivered out of the CCD as RemoteCapture says, despite its own conversions to the data as you outlined in an earlier post. Furthermore, in my specialized instrument (where this SX110 is destined if CHDKPTP can provide the performance) I don't use autofocus, in fact, I don't use the lens at all.
Quote "if you have the "contextplus" stuff correctly configure, you will see
ContexIsPlus iup:true cd:true"
Yes that is correct, in the DOS window. In fact it is also easy to see the difference between the stock and contextplus version viewports because the jaggies are very severe in the stock version, and obviously improved in the context plus version.
Quote "If you turn aspect correct off, then there is no scaling at all. So any artifacts you see are not related to that." .... "If you have Viewport 1:1 unchecked, then half the Y values are discarded when converting to RGB. If you check it, then you have exactly the data that was received from the camera, only converted from YUV to RGB."
Your comments prompted me to do an another experiment. This time I kept the native resolutions intact. Panel A is native 360x270, Viewfinder 1:1 unchecked. Panel B was created by a) checking Viewfinder 1:1, b) taking screen shot, c) resizing to 50% in Microsoft Paint, and d) pasting into attachment. Panel C is native RemoteCapture 320x240.
Although I thought CHDKPTP panel B could be better than A, both appear very similar. In RemoteCapture panel C you can best see JPEG artifacts in the background of the cables circled red. I claim that the RemoteCapture image is sharper than the CHDKPTP images. Your images are clean and free of artifacts, at native resolution. I was wondering if the sharpening in RemoteCapture could be the result of the JPEG artifacts. What is your opinion?
Overall, this test shows that CHDKPTP live images cleaner and artifact free, but RemoteCapture is sharper. Hmmm. Any thoughts?
-
I felt you might bring this up. If you take a *very* close look at the RemoteCapture image, you will find it is sharper by comparison to the CHDKPTP screen shots.
I'm not asking whether they are different, I'm asking in what application does the difference actually matter ?
If I could just snap my fingers and make it better I would, but in reality I have limited time.
Furthermore, in my specialized instrument (where this SX110 is destined if CHDKPTP can provide the performance) I don't use autofocus, in fact, I don't use the lens at all.
So is the quality of the viewport image critical ? If so, using a device with a 216 line output is probably not a good choice. What sort of instrument is this ?
Panel A is native 360x270, Viewfinder 1:1 unchecked. Panel B was created by a) checking Viewfinder 1:1, b) taking screen shot, c) resizing to 50% in Microsoft Paint, and d) pasting into attachment. Panel C is native RemoteCapture 320x240.
If you are going to scale it yourself, you might as well take chdkptp scaling out of the equation completely: Start with the aspect correct off, viewport 1:1 (720x216) image, aspect correct by scaling it to something large, e.g. 1440x1080 and then scale it back down to 360x270. Gimp may be preferrable to paint...
I was wondering if the sharpening in RemoteCapture could be the result of the JPEG artifacts. What is your opinion?
No particular opinion. However, I would note that the Canon image processing pipeline certainly has the capability to apply sharpening to still jpegs, so it wouldn't be a surprise if they applied it remote capture too.
Overall, this test shows that CHDKPTP live images cleaner and artifact free, but RemoteCapture is sharper.
The scaling done with contextplus may look smoother than the other method because it's doing some smoothing, not because it's preserving the data better.
It's also possible the the YUV/RGB conversion isn't optimal.
-
Your fine dedication to CHDK is notable and much appreciated. That's why this entire package is so good, but understandably, is still under development. I think that unless there are obvious gross bugs like the one you addressed with A/R which from the sky-view needed a 2-stage resolution, then I feel addressing more refined areas in a snap of a finger is not often the best approach anyway. On the other hand, discussing these seemingly silly issues however sheds light on what is possible for me, gives you a better understanding of my requirements, and hopefully implants some thought seeds for you if we can meet on common ground. Then a good prudent approach can be developed that can better suit your priority schedule, and naturally your judgment of a new or enhanced feature fit in the global requirements.
Quote "So is the quality of the viewport image critical ?"
Yes it is, very critical. Depth of field is measured in units of microns. RemoteCapture is just on the positive edge of the border of being acceptable. From the last comparison image, the result that came out yesterday is stunning compared to what it was just a few days ago, but still is just on the negative edge of being acceptable. Thus I am primarily interested in three functions: RAW image, liveview, and the ability to operate the camera in manual mode remotely (setting exposure profiles, Tv, Ev, digital zoom, and shooting -- essentially a mix of ptpCamGUI and CHDKPTP, for the SX110). My instrument currently uses an S50 CCD, but that camera has both liveview and RAW capability transfer with RemoteCapture. The SX110 however does have the liveview but does not have RAW out-of-box. That's where your CHDK comes in.
Quote "If so, using a device with a 216 line output is probably not a good choice."
This is a good point. Andor makes scientific cameras that could be used in my application with 640x480 liveview at 30 fps, but they are all monochrome, among several other technical drawbacks that make them unsuitable for my application. Hence commercial scientific cameras cannot be used. I designed this instrument to be very compact and flyable, and therefore it relies on small CCDs, ie 1/2.5" to 1/2" range, and those, complete with readout and data transfer electronics that I don't have to worry about, are only available in small consumer cameras. For a prototype instrument that is intended to validate basic functionality and scientific objectives, that's good enough.
Quote "What sort of instrument is this ?"
It is a microimager, something like MAHLI on MSL, but much more sophisticated. Because I have a publication pending, that is all I can say in a forum. If we chat privately, I can tell you more.
Quote "If you are going to scale it yourself, you might as well take chdkptp scaling out of the equation completely"
I did the scaling in the center panel to test a simple hypothesis and demonstrate the results to you (and to me too). The final objective would be for the liveview to provide a crisp image, not to take CHDKPTP out.
Quote "Start with the aspect correct off, viewport 1:1 (720x216) image, aspect correct by scaling it to something large, e.g. 1440x1080 and then scale it back down to 360x270."
That's a good idea ... I have a tool that does Lanczos interpolation and will try it, but again, that would be a concept test and is not meant to replace chdkptp.
Quote. "Canon image processing pipeline certainly has the capability to apply sharpening to still jpegs, so it wouldn't be a surprise if they applied it remote capture too."
Thank you for this insight, and that is more probable than a side-effect of JPEG artifacts. I agree.
Quote "It's also possible the the YUV/RGB conversion isn't optimal."
It is strange that Canon chose this archaic TV format. I really wonder why // perhaps because their cameras support video // speculating. It is amazing to me how you managed to dig that out. Nonetheless, could something in your routines related to the conversion have an impact on image quality, and give clarity a little boost?
Thank you for your help.
-
Here are the results to your suggestion. I used RAWTherapee 4 (I find GIMP difficult to use). The difference between what you suggested is that here I scaled the chdkptp original 720x216 by 2X instead going to a 4:3 A/R right away. The reason is that I can't seem to get RAWTherapee to operate on W & H independently, even though it has that option (a bug?). Thus to scale back down to 360x270, I used Paint, which in my experience does OK.
Fig 1, center panel. Direct scaling up with Lanczos interpolation on the way up, follow by down scale.
Fig 2, center panel. Mild unsharp mask applied between the Lanczos image in Fig 1 center, before down scale.
The figures are coregistered so you can see the differences better in the center panel by flipping them in a viewer.
Please let me know what you think.
-
@SticK
By looking at your screenshots I get the feeling that Canon's remote live view works with a source vertical resolution about twice as much than what the regular live view provides in shooting mode. CHDK can't do this, you only have those 216 lines, you can't get detail which isn't there.
What you need to understand is, the camera's remote controlled mode (which is what you see when you use RemoteCapture) is something that's independent of the regular shooting mode every camera has. This mode has many specialities (the camera generates MJPEG frames from the live feed the CCD provides, the shooting code is different, you have the option not to use the card for shooting), which are not present in a camera without Canon's remote control support. Since Canon seems to have dropped support for this years ago in their compacts, it's unlikely that CHDK will ever support it.
That means, we can only use features which are available in every camera. That limits CHDK remote live view to what the camera provides as live view. In your case (SX110), it has the lowest vertical resolution I've seen. You could try to see whether plugging something into the TV-out connector changes this, as reyalp suggested.
The only higher resolution live feed CHDK could use is only available in movie mode, but currently there's no support for that.
I can imagine that cameras with HDMI output may provide bigger live view resolutions even in shooting mode.
-
I noted that with all the running around between two computers, switching camera cables, and getting used to working with unusual size formats etc, I made an error in the captions. Where it read "720x540" it should read "720x216". That makes more sense and addresses reyalp's suggestion, but, my process was correct, it was just a label mistake so there is no change to the images.
For your future reference, here they are with captions corrected and added, shown in violet.
-
@srsa_4c
Quote "Canon's remote live view works with a source vertical resolution about twice as much than what the regular live view provides in shooting mode"
That is a very interesting point. If I understand you correctly, when *Canon* operates live view, they use something like 432 lines, and you're saying you don't have access to that mode.
If so, that seems to agree with the following, and also with my remark in blue in [16].
Given the amount of image quality push I had to do to plate [16] center panel 720x216, and yet it still is slightly inferior to RemoteCapture, your explanation seems to me plausible. Taking it a step further, even though JPEG artifacts are visually obvious in RemoteCapture, their resolution is still visually notably better to CHDKPTP even when pushed in [16] w/o JPEG artifacts. I am on the right track?
Quote "That limits CHDK remote live view to what the camera provides as live view. "
Is my understanding correct? What you mean is "... provides as live view [to its LCD as opposed to RemoteCapture]" ?
I am not familiar with other modes, like video, so I would have to explore them to test your suggestions. If a special video cable is needed I would not have it.
By the way, movie mode is not required by my application.
-
That is a very interesting point. If I understand you correctly, when *Canon* operates live view, they use something like 432 lines, and you're saying you don't have access to that mode.
You can only get there using Canon's ptp commandset, which we do not use.
Given the amount of image quality push I had to do to plate [16] center panel 720x216, and yet it still is slightly inferior to RemoteCapture, your explanation seems to me plausible. Taking it a step further, even though JPEG artifacts are visually obvious in RemoteCapture, their resolution is still visually notably better to CHDKPTP even when pushed in [16] w/o JPEG artifacts. I am on the right track?
As you can actually see that setup, you should know better. I can only see screenshots with artifacts.
Quote "That limits CHDK remote live view to what the camera provides as live view. "
Is my understanding correct? What you mean is "... provides as live view [to its LCD as opposed to RemoteCapture]" ?
Correct. (to the LCD or the connected TV)
I am not familiar with other modes, like video, so I would have to explore them to test your suggestions. If a special video cable is needed I would not have it.
No special cable is needed (usually you get one with the camera anyway), you can use a toothpick* or something :)
* non-conductive :)
BTW, I wouldn't expect much higher resolution in movie mode either (LCD/Tv-out will continue to use the same framebuffers with probably unchanged resolution).
By the way, movie mode is not required by my application.
I understand, but that would be the only way to get higher res. live view (and it isn't implemented, as I wrote).
-
By the way, movie mode is not required by my application.
I understand, but that would be the only way to get higher res. live view (and it isn't implemented, as I wrote).
As I mentioned earlier, on some cameras, switching the camera to movie mode does change the resolution of the normal live view buffer, without any need to capture the movie buffers specifically. On the a540, 640x480 mode gets you 704x528 live view. I suspect this doesn't work on a digic III cam like sx110 but if it does, SticK could to movie mode when the live view was needed, and switch back to still to take the actual shots.
It's also worth noting that some recent cameras (e.g. the G12) have a 480 line live view.
-
@srsa_4c
This camera uses a special USB male that has pins on both sides. That means the AV cable plugs in where the USB cable does, and I don't have the AV cable // ebay logic.
@reyalp
Very very interesting, if it works that would be great ... because I have noticed that response of CHDKPTP is way faster than ptpCamGUI. Would I be able to see the results, or at least a difference that points to a good direction in CHDKPTP right away? If so, how can I make the camera go into movie mode from the command line in CHDKPTP, and back again to still mode?
-
Hence commercial scientific cameras cannot be used.
This seems quite incredible. Maybe you mean that the cost is more than you want to pay, or the engineering to integrate them is more difficult that you would like ?
It is a microimager, something like MAHLI on MSL, but much more sophisticated.
A lofty goal.
As an aside, have you verified you can operate the sx110 without the lens hardware ? If so, this is definitely something other members of the forum would be interested in. If not, then you might want to do that before you invest too much time in other stuff.
Because I have a publication pending, that is all I can say in a forum. If we chat privately, I can tell you more.
You can PM me on the forum if you like. I have no specific need to know about it, but if I find the project interesting I might be more willing to spend time supporting it.
I did the scaling in the center panel to test a simple hypothesis and demonstrate the results to you (and to me too). The final objective would be for the liveview to provide a crisp image, not to take CHDKPTP out.
I understand that. My suggestion was to take the scaling done by CHDKPTP out of the equation, so you can see if that is responsible for the image quality difference.
It is strange that Canon chose this archaic TV format.
Technically, I guess we should say YCbCr rather than YUV. It's very common in all sorts of digital video and imaging.
Nonetheless, could something in your routines related to the conversion have an impact on image quality, and give clarity a little boost?
Sure, it's possible. I'm not aware of any specific problems, but I'm not an expert in this sort of thing either. One question in my mind is how the UV pair relates to the four Y values. Currently each Y in a given UYVYYY chunk uses the same U and V values, but it's possible the Ys on the ends should blend the U and V values from the neighboring blocks (with special cases for the line ends...) However, this should only affect chroma, which doesn't really seem like your issue.
Would I be able to see the results, or at least a difference that points to a good direction in CHDKPTP right away?
If you enter
set gui_verbose=2
in the console, changes in the live view dimensions will cause the frame buffer description to be printed out in the console. You can set this in the same file as the contest plus option. If you are using viewport 1:1 the difference should also be fairly visible if you go from ~200 lines to ~400.
If so, how can I make the camera go into movie mode from the command line in CHDKPTP, and back again to still mode?
In the gui, there's a dropdown on the right side that lets you change mode.
In the command line, you can get to manual still mode using something like
lua require('capmode').set('M')
and to movie with
lua require('capmode').set('VIDEO_STD')
You can get the exact names from the dropdown.
Note on the A540 at least, the low resolution video modes reduce the viewport width.
-
This is what I was able to do ... I enabled the movie mode by using dial. With [ ] Viewfinder 1:1 and
- Scale A/R set, movie mode changed the size of the viewport to 1/2 the still mode size (looks like a large thumbnail).
With - Viewfinder 1:1 and
- Scale A/R the viewport is 360x270, but image looks pretty bad, with rather gross horizontal artifacts. The same artifacts are visible on the camera LCD.
MOVIE MODE ANSWER: the movie mode on the SX110 seems to go in the other direction where we want it to. Too bad.
@reyalp
G12 - thanks for the suggestion. I looked at the tech specs and the 1/1.7" CCD is OK and would work perfectly in my instrument. I read that Canon discontinued RemoteCapture operation and the G12 does not support it. Thus CHDKPTP must be the only mechanism to see live view, correct, because it reads the same frame buffer as the camera LCD, right?
Is the 480 line view in still mode on the G12 ?
Do you know if G12 RAW files are 12-bit or more?
This is becoming interesting.
I will address your new message shortly.
-
Quote "This seems quite incredible. Maybe you mean that the cost is more than you want to pay, or the engineering to integrate them is more difficult that you would like ?"
Neither ... as I mentioned earlier .. "there are *other* [technical] drawbacks, in addition to being monochrome."
Quote "A lofty goal."
Yes, but it is already functional, and has been doing science for a few years. Thus the lofty goal is behind me. A paper is coming likely around January.
Quote "have you verified you can operate the sx110 without the lens hardware ?
No, not with the SX110, that is why I am doing all this first. Although I had checked out G12 before, I did not know then CHDKPTP would work live. So my ears went up like a puppy when you mentioned it // looking fwd to your answers.
Quote "If so, this is definitely something other members of the forum would be interested in."
My instrument has been working with the S50 CCD for many years, but the job was horrendous, especially too when there is need to cool the CCD. If we find a good potential camera for this "upgrade", then perhaps we can work together to get a great result. More later on this topic, if all this pans out.
Quote, "If not, then you might want to do that before you invest too much time in other stuff."
Yes, that is what I doing now, in part with you.
Quote " However, this should only affect chroma, which doesn't really seem like your issue."
That's what I thought. If you're worried about the different hues I was showing in my images, no need .. I had white set differently when connect to RemoteCapture. In the recent figures I corrected that.
I will try your command suggestions after dinner.
-
G12 - thanks for the suggestion. I looked at the tech specs and the 1/1.7" CCD is OK and would work perfectly in my instrument. I read that Canon discontinued RemoteCapture operation and the G12 does not support it. Thus CHDKPTP must be the only mechanism to see live view, correct, because it reads the same frame buffer as the camera LCD, right?
Is the 480 line view in still mode on the G12 ?
Do you know if G12 RAW files are 12-bit or more?
The G12 live view buffer is 720 x 480 YUV when shooting normal images in 4:3 aspect ratio.
It has a 12 bit, 1/1.7" CCD sensor.
If you need the best image quality possible then also look at the G1X.
It also has the same 720 x 480 YUV live view buffer; but it has a 1.5" (near APS-C size), 14 bit RAW sensor.
Neither is supported by Canon RemoteCapture; but both work with chdkptp for remote live view.
The best frame rate I have been able to get on both the G12 and G1X with chdkptp is 20fps.
Phil.
Edit: according to the current CHDK code other cameras with 480 line live view are G10, G11, IXUS310, S90, S95, S100, SX220, SX230, SX240, and SX260. Some of these are quite new and may not have complete ports of CHDK yet.
-
I have a new *very* encouraging result. The balance has just tipped in favor of a good future here.
Attached, center panel.
Here's what happened ...... after setting video mode with the camera dial and got the useless "thumbnail-size" live video I reported before dinner, I went on a tip from reyalp and saw that the dropdown was set to VIDEO_COMPACT. I changed that to VIDEO_STD and result is *amazing*. There are still some artifacts we can chat over, but that I think all is very doable and definitely on a good track. If you can demonstrate that we can handle the minor artifacts shown in yellow, then it's downhill from here.
I will converge on a YES/NO for the SX110 through iteration, perhaps potentially investigate a G12, so we may revisit live view later if needed. Next is to do some response and automation tests (script-driven I assume as I am just a week old with CHDK) and may need a bit of help.
@philmoz
Thanks for the useful additional info. APS-C sized sensors are much too big for my instrument, and is one of the main reasons I can't use commercial cameras .. they are all large format.
Quote "Neither is supported by Canon RemoteCapture; but both work with chdkptp for remote live view."
Very nice.
Folks, you have my attention.
-
Sorry, scale for A/R must be checked for 360x270 in the center panel, not unchecked. Two computer problem.
-
Quote "have you verified you can operate the sx110 without the lens hardware ?
No, not with the SX110, that is why I am doing all this first. Although I had checked out G12 before, I did not know then CHDKPTP would work live. So my ears went up like a puppy when you mentioned it // looking fwd to your answers.
Just to be clear, I was absolutely NOT stating that the the G12 would run with the lens hardware removed, these are two completely unrelated questions!
Quote "If so, this is definitely something other members of the forum would be interested in."
My instrument has been working with the S50 CCD for many years, but the job was horrendous, especially too when there is need to cool the CCD. If we find a good potential camera for this "upgrade", then perhaps we can work together to get a great result. More later on this topic, if all this pans out.
Quote, "If not, then you might want to do that before you invest too much time in other stuff."
Yes, that is what I doing now, in part with you.
I don't think you have understood. Nothing we have discussed related to making the camera work without the lens. Current Canon firmware has lots of error checking. When it encounters what it thinks is a hardware error, it stops. There is a very high chance that if you remove the lens hardware, the camera firmware will refuse to function. The electronics have almost certainly gotten more complicated over time, so even if you were able to spoof the s50, doing the same thing for a newer camera is likely more difficult.
Some uses have reported success on some models by dismounting the lens hardware but leaving it as unit with all the electronics attached. I believe some people have managed to override some errors in software (srsa_4c might know more about this), but I don't recall anyone pulling this off for a complete lens removal.
If you don't have a solution for this, then jaggies in the live view are a quite minor issue...
Here's what happened ...... after setting video mode with the camera dial and got the useless "thumbnail-size" live video I reported before dinner, I went on a tip from reyalp and saw that the dropdown was set to VIDEO_COMPACT. I changed that to VIDEO_STD and result is *amazing*.
A dump or the output from gui_verbose should provide a simple Yes/No answer about the viewport resolution. If you are getting higher resolution, but haven't checked viewport 1:1, then it will be downscaled which might be responsible for the jaggies.
-
Quote "I was absolutely NOT stating that the the G12 would run with the lens hardware removed"
What I meant by "checking out G12," was in far in the past when I was selecting the best candidate for a higher density CCD, and never implied anything of the sort. To clarify, now that I know you can drive a G12 with PTP live view, I will rekindle my search, because there may be a better candidate than the SX110. Despite that, the SX110 is still an excellent CCD candidate. I see I mixed the two thoughts in one place // oops!
Quote "When it encounters what it thinks is a hardware error, it stops"
I agree, there is no argument // well known here.
Quote "high chance that if you remove the lens hardware, the camera firmware will refuse to function"
That's definite.
Quote "Some uses have reported success on some models by dismounting the lens hardware but leaving it as unit with all the electronics attached."
Yes, that obviously works.
Quote "I believe some people have managed to override some errors in software (srsa_4c might know more about this), but I don't recall anyone pulling this off for a complete lens removal."
This is the more guru zone and is where I felt I might ask your input in the future and we could chat. But since you raised it ... why not? Keeping the lens assembly on restricts access to both sides of the CCD because in my case, I have to cool it. Thus the best solution is to in principle "cut the glass out" and leave the CCD mounted on the camera frame. That way one gets a clear optical path to front, and TEC access to the back, wherein an accurate mechanical frame can be machined for alignment of the whole camera body. For example, I might be able to track down and identify the error detector signals and for those that I could not provide an electrical state=good impostor for, perhaps you might know where the table is in firmware and disable the error detection for a specific signal. That could possibly mean changing value(s) in Canon EEPROM. Is that something in your sphere?
Quote "If you are getting higher resolution, but haven't checked viewport 1:1, then it will be downscaled which might be responsible for the jaggies."
If I check Viewfinder 1:1, then I get the attached. Does this help?
-
DUMPS
Viewfinder 1:1 unchecked (small view)
http://www.sendspace.com/file/7rfhkv (http://www.sendspace.com/file/7rfhkv)
Viewfinder 1:1 checked (big view)
http://www.sendspace.com/file/uxkjjw (http://www.sendspace.com/file/uxkjjw)
-
For example, I might be able to track down and identify the error detector signals and for those that I could not provide an electrical state=good impostor for, perhaps you might know where the table is in firmware and disable the error detection for a specific signal. That could possibly mean changing value(s) in Canon EEPROM. Is that something in your sphere?
It's unlikely to be a single table. This thread may be of interest http://chdk.setepontos.com/index.php?topic=6725.0 (http://chdk.setepontos.com/index.php?topic=6725.0)
As a rule, we do not modify camera ROM. Even though it could be useful in some cases, most of us don't have cameras to sacrifice, and even if we did, the odds of end users bricking theirs and blaming us would be too high for any sort of general distribution.
DUMPS
As i've said before, the size/aspect buttons don't affect the dump.
Anyway, here's the relevant output from gui_verbose
vp.buffer_width:nil->720
vp.visible_width:nil->720
vp.visible_height:nil->240
so it's 240 rather than 216. The extra 24 rows probably aren't worth it, and may be interpolated by the canon firmware anyway.
edit:
I'm not sure why you are seeing more jaggies than in still mode...
-
Thank you for the reminders and patience. I will post only dump per incident from now.
Quote "This thread may be of interest http://chdk.setepontos.com/index.php?topic=6725.0 (http://chdk.setepontos.com/index.php?topic=6725.0)"
Interesting. This fella from Argentina hit it right on the money. You refer readers directly here. At the top of his thread, his targets are labeled **. The first link at the top goes to his site where he shows some electrical mods. That's the right idea ...
http://chdk.setepontos.com/index.php?topic=3409.0 (http://chdk.setepontos.com/index.php?topic=3409.0)
Quote "Even though it could be useful in some cases, most of us don't have cameras to sacrifice, and even if we did, the odds of end users bricking theirs and blaming us would be too high for any sort of general distribution."
For those users perhaps it would be worthwhile to remind them in no uncertain terms what "H" stands for. Folks are human and might forget. In my case, I could send you a sacrificial lamb, already in a partially dismantled state, but electrically functional. For example, I already have two SX110s // one is a spare. The camera model could change now that I know the reach of CHDKPTP, and especially the new fact to me, that the camera does not have to support RemoteCapture in order to obtain the live view CHDKPTP which was my preconceived illusion before we met.
Quote "The extra 24 rows probably aren't worth it, and may be interpolated by the canon firmware anyway."
Can you explain this a bit more? I don't know how relevant this is to what you are saying, but one thing I am certain is the 360x270 liveview in VIDEO_STD is now "crisp" meaning that it can be used effectively to focus my instrument. In still mode, for the SX110 it would be at very difficult to focus effectively.
By the way, I simulated a few lua commands, eg AUTO, shoot_half, VIDEO_STD. The response of the camera is subjectively for now, "very good." Can you direct me to sample code that would allow me to do tests like this?
M
set ISO
set Tv (and Av)
shoot_half
shoot
VIDEO_STD
How long can a command line be? Can I invoke a code text file to run with CHDKPTP ? .... things of this nature.
Thanks.
-
Quote "Even though it could be useful in some cases, most of us don't have cameras to sacrifice, and even if we did, the odds of end users bricking theirs and blaming us would be too high for any sort of general distribution."
For those users perhaps it would be worthwhile to remind them in no uncertain terms what "H" stands for. Folks are human and might forget. In my case, I could send you a sacrificial lamb, already in a partially dismantled state, but electrically functional.
The thing is, no matter how many disclaimers you put on it people will still use it, and they will still blame us if it ends badly. People blame us when their camera dies for things completely unrelated to CHDK. That might be tolerable on it's own, but if people start bricking cameras, it could attract unwanted negative attention from Canon.
To me, this makes development that involves re-writing camera ROM a dead end (though I'd still like to play with it a bit at some point.) Picking up sacrificial cameras on amazon/ebay is not the issue, the real cost is development time.
If you want someone to devote many hours of highly technical effort to something that is only applicable to your project, you may find volunteers hard to come by.
Can you explain this a bit more? I don't know how relevant this is to what you are saying, but one thing I am certain is the 360x270 liveview in VIDEO_STD is now "crisp" meaning that it can be used effectively to focus my instrument. In still mode, for the SX110 it would be at very difficult to focus effectively.
Going to video mode on your camera appears to change the viewport to 720x240 rather than 720x216. I wouldn't expect a huge jump in quality from this, but I suppose the Canon pipeline maybe be doing something different that ends up giving you better detail.
I would suggest comparing the unscaled, full res output (aspect correct off, viewport 1:1 on) of each to figure out what the differences are.
IIRC you mentioned earlier that halfshoot also switched the viewport to 240 (? or was it the post shot review ?), so you could perhaps use that instead of video mode.
By the way, I simulated a few lua commands, eg AUTO, shoot_half, VIDEO_STD. The response of the camera is subjectively for now, "very good." Can you direct me to sample code that would allow me to do tests like this?
...
The Camera side Lua functions available though chdkptp are the same as used for on camera scripting. I suggest you consult the wiki for information about that. http://chdk.wikia.com/wiki/Script_commands (http://chdk.wikia.com/wiki/Script_commands) is a good starting point. http://chdk.wikia.com/wiki/Lua/PTP_Scripting (http://chdk.wikia.com/wiki/Lua/PTP_Scripting) has some of the PTP specifics. http://chdk.wikia.com/wiki/Lua (http://chdk.wikia.com/wiki/Lua) has information specific to Lua on CHDK (ubasic and lua share most of the same CHDK scripting functions, but ubasic is not available over PTP)
The official Lua manual http://www.lua.org/manual/5.1/ (http://www.lua.org/manual/5.1/) is also very useful.
You should be aware that the scripting system is a hack, like all the rest of CHDK. In many cases, the behavior has subtleties which vary from camera to camera, are not well documented, and and may not be well understood by anyone. When in doubt, the best source of definitive information is careful experiment. chdkptp is quite useful for this, since you can interactively try stuff and see the results.
How long can a command line be? Can I invoke a code text file to run with CHDKPTP ? .... things of this nature.
There is no specific limit on the length of a command.
There are several ways you can execute file:
1) Upload a file to the camera and then use something like lua loadfile('A/CHDK/SCRIPTS/testprgm.lua')()
2) Use PC side Lua scripting to load your file as a string and send it to the camera to run. The 'rlib' module in chdkptp is one example of this.
Using either of the above, you can also start a script on the camera which reads messages over PTP, so you can build your own custom protocol / command set. There are some examples of this in the chdkptp lua files.
The PC side of chdkptp is very scriptable. In fact, the entire GUI and CLI are written in Lua. You can find all this code in the lua directory, plus some additional examples (multicam.lua and the extras directory)
-
Quote "If you want someone to devote many hours of highly technical effort to something that is only applicable to your project, you may find volunteers hard to come by."
Very true and I'd be silly to think that. Please recall from an earlier post that I did say something to the effect that you have the final decision in reference to this subject line // naturally // you are in that special position // basically // it's your call. I only jumped in because you raised the topic earlier than I would have, and, I would have asked you differently. Please note I am still in the process of evaluating the technical possibilities for a CCD. Had it been me to raise the question which I would done after escalating CCD selection to acceptable level, I would have asked it in the context of community you are addressing. And, by example, that I am also dedicating time interacting with you, is because I am interested in your help, but also, I am fully aware my requests are not permitted to be in a vacuum. Nevertheless we've opened the topic for discussion and I don't feel it's a bad thing .. because I already see huge potential with CHDK+++. So since you have an overview of the community's needs and desires, that of course would have to square with the needs of my own project, and I fully understand it. Were you able to read the thread I linked you to? It is important that you do because the context I am referring to is the fundamental objectives of my project that I feel are likely parallel to those in amateur astronomy, like what Leonardo did in his thread: technically, primary focal plane architecture gives the ultimate optical performance of a CCD, whether it's a telescope or microimager. What he did is what I did with my S50 a few years earlier back of him, but of course I used a different topology. I did describe the process as "horrendous" when you asked, and you can get an appreciation of what I meant by looking at his site link at the top of the thread. And since I detected yesterday that you suggested an interest in your community for direct CCD access, I felt that a firmware solution would be far more elegant than butchering the hardware and attempting to fool error-free access that way. But you see, I don't how generic that could be made. Those are indicators only you could come up with after thinking it all through. The other thing is "bricking" ... this is the first time I hear this word and I assume it imeans "reverse engineering with the objective of modifying copyrighted code." Let's say I am on the right track, then, lest one not forget, if my technical objective fits the community, would overwriting by a RAM position that holds the lens error variables by CHDK code, in principle, fall or not fall under that preventive "jurisdiction" of catching Canon's attention etc. I think perhaps not. What is your opinion of all this?
Quote "Going to video mode on your camera appears to change the viewport to 720x240 rather than 720x216"
Yes, that is correct.
Quote "IIRC you mentioned earlier that halfshoot also switched the viewport to 240 (? or was it the post shot review ?), so you could perhaps use that instead of video mode."
Yes, I tried this now and it works too. Is there a lua command that will permanently keep shoot_half down, and stay in that state after a shot is taken? Could be esoteric anyway if I choose a camera with a higher definition live view.
Thanks for all the great tips .. I have homework to do.
-
Quote "If you want someone to devote many hours of highly technical effort to something that is only applicable to your project, you may find volunteers hard to come by."
Were you able to read the thread I linked you to?
I'm pretty sure I read it when it was first posted.
And since I detected yesterday that you suggested an interest in your community for direct CCD access, I felt that a firmware solution would be far more elegant than butchering the hardware and attempting to fool error-free access that way.
My comment about community interest was just that, a comment. A fair number of people see the possibilities of CHDK, and ask about operating without lenses. A full software only solution is likely to be very difficult if it's possible at all.
The other thing is "bricking" ... this is the first time I hear this word and I assume it imeans "reverse engineering with the objective of modifying copyrighted code."
No, brick (http://www.catb.org/jargon/html/B/brick.html), meaning transforming the camera from a picture taking device into an inert object (similar to a brick, but with inferior structural properties), by damaging the on-board firmware in an unrecoverable way.
What is your opinion of all this?
My opinion is that if a bunch of people try to RMA cameras they bricked with CHDK, or otherwise consume Canon support resources, that might change Canons current hands-off attitude.
Yes, I tried this now and it works too. Is there a lua command that will permanently keep shoot_half down, and stay in that state after a shot is taken? Could be esoteric anyway if I choose a camera with a higher definition live view.
If you hold it down all the time, you can't really do the normal shooting sequence. Certain things must be updated outside of half shoot, others must happen inside.
CHDK scripting does have the capability to send key presses and releases, using the press and release functions. However, the current scripting code releases any keys that were pressed when the script terminates. Since each lua command over ptp runs a separate script, this means a simple lua press('shoot_half') will not do what you want. That's why I had you use press('shoot_half');sleep(1000) earlier, it waits 1 second before allowing the script to end. You can also use a long running message based script if you need more control, multicam.lua has an example of this. Another alternative is to send key presses an releases using the firmwares own http://chdk.wikia.com/wiki/LogicalEvent (http://chdk.wikia.com/wiki/LogicalEvent) system.
-
Second photo down, ie, getting the impostors in is no big deal, it's figuring out where the signal lines are and what voltage/current levels to use. That's what I meant by horrendous ...
http://www.astrosanjusto.net63.net/taller/a60focoprimario/index.htm (http://www.astrosanjusto.net63.net/taller/a60focoprimario/index.htm)
The more recent cameras have more gadgets there, so even more so.
Quote "A full software only solution is likely to be very difficult if it's possible at all."
I had expected what you said, but, could you give it some more thought? I am taking a guess that Canon boot loader starts CHDK first, CHDK initializes itself, and then transfers control back to Canon finish its startup. If I'm correct, you could have a window of opportunity. Here is my train of thought ... Ideally from the user's perspective, one would disconnect the lens electromechanics (ie, unplug the assembly, usually a few tiny connectors), then on PUP, CHDK would be the first to detect the error condition, write the impostors into memory, and Canon would think the lens assembly is there. Alternatively, CHDK when booting could read a script that has the table entries identified and the user puts the "hard-coded" error condition override values himself and proceeds at his own risk. Even if there are dynamic issues during normal operation that could clear the error flags, so long as the user does not attempt to mobilize any actuators by accident (focus, IS, zoom, aperture, shutter etc) then the fooled state might hold. Error table(s) and activity are probably similar across models. What I'm saying is admittedly simplistic and unquestionably it is a difficult proposition, so honestly I am not expecting miracles.
QUESTIONS
Q1. Now that I understand chdkptp better, what is the frame buffer resolution of the liveview on the S90 and S95, in still mode? I am considering these as potentially good CCD candidates for my instrument that will be driven by your FW/SW. The CCD format in those is more appealing than the SX110. The S90 has simpler lens electromechanics than the G12 as well.
Q2. Events (I think the events you talked about are related to operating the camera, something different). General overview if you could please .... I am not sure but it seems FW/SW is not event-driven, in the textbook sense. For example, once the server completes a 16 second exposure in the camera and finishes writing the file(s) to SD, it fires off an event to the client so it knows the operation was completed and can proceed or vector to the next operation, without the need for the client to poll or count time. How does it work at present? Do you foresee something like this in future FW/SW?
-
I am taking a guess that Canon boot loader starts CHDK first, CHDK initializes itself, and then transfers control back to Canon finish its startup. If I'm correct, you could have a window of opportunity.
The CHDK boot process is a little more complicated, but that's not the main problem. To do any of the stuff you suggest, a large amount of the firmware needs to be reverse engineered. You talk about error tables and writing resetting variables in memory, but first you have to actually find all that stuff.
Most of the firmware is unknown. As a typical example, the firmware code on my D10 amounts to >2.5 MB of ARM machine code. Analyzing a function of a few thousand bytes can take hours, if it's relatively self contained. So even for those of us who have devoted many many hours to reverse engineering, the vast majority is terra incognita.
Now it's *possible* that there is a relatively limited number of things you would have to correct, or that you could cleverly hook or disable error handling somewhere, but just finding out would be a substantial effort. If I had lots of free time, I'd be game to try, but at the moment I don't. Maybe someone else does.
Q1. Now that I understand chdkptp better, what is the frame buffer resolution of the liveview on the S90 and S95, in still mode?
According to the CHDK code, it looks like the s90 and s95 should both have 480 line viewports.
For example, once the server completes a 16 second exposure in the camera and finishes writing the file(s) to SD, it fires off an event to the client so it knows the operation was completed and can proceed or vector to the next operation, without the need for the client to poll or count time. How does it work at present? Do you foresee something like this in future FW/SW?
I'm not really clear what level you are asking about.
If you are talking about the CHDK PTP protocol (including the chdkptp client) it operates by polling. We don't have control of USB events on the camera, nor any other way to initiate communication from the camera side. It might be possible to reverse engineer, but in practice polling isn't much of a handicap. The live view also operates by polling: It's just a timer that periodically asks for a frame.
The CLI is obviously more limited:
'lua' (or '.') starts a script on the camera without waiting for it to finish.
'luar' (or '=') starts a script, blocks until it ends, and then prints any return values.
If you start a script with 'lua', you can get it's return values with 'getm'. If your script runs in a loop an uses the message API, you can also interact with it while it's running using 'putm' and 'getm'.
If you need to do something more complicated, you can write lua code that runs on the client.
The camera firmware itself has a multitasking operating system. Older cameras use vxworks, which you can find plenty of information about online. Very little about DryOS is publicly documented, but it seems broadly comparable to vxworks.
-
Quote "Analyzing a function of a few thousand bytes can take hours, if it's relatively self contained... terra incognita"
I have no doubt, and can only marvel at your dedication and what you've already accomplished, on top of your day job! ... which hopefully is something a little different from coding to keep your sanity. Most of us are happy to write a few lines of assembler to speed up our C code, but how many folks are there on this planet that have the skills you have, and ARM to boot? ... not many, I am sure.
Quote "It might be possible to reverse engineer, but in practice polling isn't much of a handicap. "
You answered my question.
lua and luar
While studying the docs I noticed this. Thanks for volunteering the explanation.
QUESTIONS
Script execution speed. In principle, let us say I write a 200 line script and want to operate strictly from the client (from the user-perspective fully flexible remote), for example, modifying a few parameters and then re-issuing the code on the fly. How do you handle this? Do you execute each small chunk of a function on the client with handshaking? Do you transfer the entire file to camera RAM (or SD temp file) and transparently execute from there? How best should I handle this? Eg, will the operation execute faster if I write the file to SD and give CHDK the command to execute it, or is there really not much difference from sending the entire script file from CHDKPTP? How do I transmit a multi-line script to the camera from CHDKPTP? Is there another way outside of CHDKPTP to do it?
LUA
In your docs you have an example titled "Universal Tv Mode." Very basically, how would I have to simplify the code for a remote scenario where I give it a Tv parameter in a function I wrote, from the client? Where/how do I find/extract the translation table of Tv index versus shooting speed (property case name?), for example? I suppose get_prop() and set_prop() are at the core, but I have had some difficulty finding a detailed explanation of the syntax. The most important dynamic parameters for me at the moment to test performance are set ISO, set Tv and shoot. Could you please direct me, or give me example code I could start with? How would this look like in lua that I can try with CHDKPTP?
C-style Pseudocode ...
my_set_ISO(iso) // where iso could be integers 100, 200, 400 etc
{
// ?
}
my_set_Tv(Tv) // where Tv could be strings "64", "32", "16" ... "1/128" etc
{
// ?
}
HARDWARE
Please don't dig for an answer if you don't have it at hand. Question is related to image file transfer speed on a USB 2.0 port. For example, the S50 is very slow, ~10s for a big 4 MB JPG. The S70 which Canon promotes as sporting USB 2, is only marginally faster. The SX110 on the other hand is super fast ... a split second for a 6 MB JPG. Would you have an idea how the S90, S95, perform?
-
My instrument has been working with the S50 CCD for many years, but the job was horrendous, especially too when there is need to cool the CCD.
Many years ago I worked on a CCD line scan camera where we put a Pellitier effect cell between the back of the sensor IC and a huge heat sink. While this greatly lowered thermal noise, controlling humidity to avoid condensation on the sensor window was a huge problem.
Any hints on how you plan to handle humidity ?
-
@waterwingz
Quote "Any hints on how you plan to handle humidity ?"
That is a never ending issue. What is well known is that for CCD photodiodes thermal noise drops by a factor of 8-10 every 20C drop in temperature. What I do much depends what improvements that can be had for the final objective of my imaging. For example, the S50 right now is limited to 15s Tv, so I operate the CCD a few degrees above dew point which doesn't change much in my lab. Therefore on the S50 not much improvement in SNR can be expected by lowering it below +15C. With CHDK and the CCD enclosed in the SX110 camera I am reading +44C at RT. I tested dark performance but only at 15s Tv only so far at RT, and it's at least as good as the S50, despite 10Mpx vs 5Mpx. Some of that difference may be due to lower CCD temp in the original S50 in-camera test when I did many years ago. So this is a very qualitative assessment for now.
Also CCD edge-charge fringing (pink-violet glow near corners) has to be known to me at the long exposures you offer, and I still have to investigate that on the newer cameras I am evaluating, ie. whether or not Canon has corrected the problem. Perhaps you might offer some insight on this.
If all works well for the "upgrade" of my instrument and the selection of a camera, the best solution is to machine a small enclosure housed in the camera body (hence getting the lens out) for the CCD & TEC that will have a quartz window, then extract the air with a vacuum pump. This method has 2 benefits: a) CCD temp can be lowered below dew point, and b) counter-heating the CCD by air convection is eliminated, greatly reducing the current through the TEC needed to keep the sensor cold. Hamamatsu does this in some of their line-scan sensors used in spectrometers. This architecture also reduces the transfer requirements of the hot-side, ie heat sink size etc.
-
Humidity is a big problem, filling the empty space with silica gel partially helps, silica gel and a hermetically sealed housing filled with an inert gas like Argon gas might be a good solution, but there is still a problem of how to create a hermetically sealed housing. :haha
I know people who restated cameras for astrophotography and humidity problem was never resolved.
-
Humidity is a big problem, filling the empty space with silica gel partially helps, silica gel and a hermetically sealed housing filled with an inert gas like Argon gas might be a good solution, but there is still a problem of how to create a hermetically sealed housing.
That's basically what we did - except we used about 2 cu.in. of molecular sieves to dry the air trapped in the hermetically sealed housing and didn't worry about the inert gas part.
-
@waterwingz @blackhole
Yes what you did is effective and I have worked with these kinds of solutions too. Some other sub-assemblies in my instrument have these implemented where heatsink size and power are not a big issue, but, these solutions only mitigate the humidity problems. The convection problem, which translates into higher TEC power for the same cold-side temperature, remains.
For me all I can say is that these are the basics. How I tackle and how much elbow grease I will put into the selected CCD cavity (if useful // the charge-fringing problem doesn't start dominating), including the hermetic seals (the flat cable is a bigger problem than the window for example), all depends on the results I will be getting with camera models driven by CHDK with its extended Tv, that is .... will it be worth going that route. I am still trying to narrow down potential camera models before making a commitment to purchase some more to test. The SX110 is good but the S90 or S95 are better in several important respects specific to my instrument, but they also have to be speedy .... the old sweet-spot on the tennis racquet.
Can anyone help me with beginner's code that address my psuedocode request a couple of posts ago?
What could this be in lua? Perhaps I should hang-in for reyalp ... he may be gone for the weekend // that's OK.
my_set_ISO(iso) // where iso could be integers 100, 200, 400 etc
{
// ?
}
my_set_Tv(Tv) // where Tv could be strings "64", "32", "16" ... "1/128" etc
{
// ?
}
-
Script execution speed. In principle, let us say I write a 200 line script and want to operate strictly from the client (from the user-perspective fully flexible remote), for example, modifying a few parameters and then re-issuing the code on the fly.
200 lines is not much. Achieved bulk transfer rate with USB 2.0 is at least several megabytes a second. This varies depending on the camera, newer cameras are faster. In the live view, the "last xfer kb/s" is a pretty good indicator of bulk throughput. Each transaction has overhead, which is a few ms at the minimum. I see ~5ms average for d10 and ~10ms for a540, both vary by about +10/-3. Bottom line, in most cases, it shouldn't matter if your script is 10 bytes or 1kb. The number of transactions tends to matter more.
You can use
set cli_time=true
to see the execution time of a command. This is the total time for the command to complete on the client, so if you do something that waits (e.g. =return 1) all the waiting is included too.
How do you handle this? Do you execute each small chunk of a function on the client with handshaking? Do you transfer the entire file to camera RAM (or SD temp file) and transparently execute from there?
The chdkptp CLI and GUI commands work by building the script on the fly and sending the entire thing over. This is done for simplicity. If you look at chdku.lua con_methods:exec you can see how it's done. rlibs.lua contains a bunch of utility code which is combined with the script to execute. So if you do something like
=return 1
It's actually loading prepending the 'serialize_msgs' code from rlib (76 lines, 1.6 kb) to whatever you enter, sending it to the camera, polling until script execution ends, fetching the return message. Many of the chdkptp cli commands load a lot more code.
How best should I handle this?
"Best" depends on your application. Does a few milliseconds matter ? How much code do you want to write ?
Eg, will the operation execute faster if I write the file to SD and give CHDK the command to execute it, or is there really not much difference from sending the entire script file from CHDKPTP?
I haven't benchmarked loading from the card (would depend a lot on card speed), but I would guess with a modern camera and USB 2.0, there is very little reason not to send it over USB.
If you want the absolute minimum latency, then you should write a script that runs continuously on the camera and uses the message interface. The multicam.lua file is an example of this.
How do I transmit a multi-line script to the camera from CHDKPTP? Is there another way outside of CHDKPTP to do it?
I answer this before, see the end of this post http://chdk.setepontos.com/index.php?topic=8585.msg89772#msg89772 (http://chdk.setepontos.com/index.php?topic=8585.msg89772#msg89772)
In your docs you have an example titled "Universal Tv Mode." Very basically, how would I have to simplify the code for a remote scenario where I give it a Tv parameter in a function I wrote, from the client?
I haven't looked at this script, but to set the Tv value, you probably want to use set_tv96_direct. This must be called *before* you press half shoot to take the shot. Most of the exposure related functions and propcases work in APEX*96 values. The 96 in the name tells you this one does too. So to set a 1 second exposure, you would use
.set_tv96_direct(0)
Where/how do I find/extract the translation table of Tv index versus shooting speed (property case name?), for example?
I suggest using the APEX values rather than any of the functions that take indexes. APEX is your friend. On the off chance you aren't familiar with it, http://dougkerr.net/Pumpkin/#APEX (http://dougkerr.net/Pumpkin/#APEX) is an excellent reference. The camera uses APEX*96 so it can work in all integer math.
If you want to use traditional (e.g. "1/250th") values, I'd suggest doing the conversion on the PC side, since PC side Lua has floating point math. Alternately, you could just make a lua table mapping the strings to the APEX*96 values.
A further wrinkle when dealing with ISO is the "real" vs. "market" values, discussed here: http://chdk.wikia.com/wiki/CHDK_User_Manual#Override_ISO_value (http://chdk.wikia.com/wiki/CHDK_User_Manual#Override_ISO_value) - CHDK generally works with the "real" values.
Please don't dig for an answer if you don't have it at hand. Question is related to image file transfer speed on a USB 2.0 port. For example, the S50 is very slow, ~10s for a big 4 MB JPG. The S70 which Canon promotes as sporting USB 2, is only marginally faster. The SX110 on the other hand is super fast ... a split second for a 6 MB JPG. Would you have an idea how the S90, S95, perform?
I would expect any new cameras to be as good or better than the sx110. My d10 (digic 4, newer than sx110) shows 10-20mb/sec for live view. My a540 (Digic 2, older than sx110) shows 5-10. The S90 and S95 should be as good or better than the D10.
-
For those folks wishing to use silica gel for CCD cavities, Alfa Aesar -6+18 mesh granules is an excellent compromise all-around ... also has intrinsic indicator that is blue when dry, pink when soaked. Desiccation time is 8 hrs at +120C.
-
... he may be gone for the weekend
That would be a first .... :o
-
Quote "The number of transactions tends to matter more."
Is a transaction the equivalent of a camera operation? If not, what is it?
@reyalp
Could you please give this noob (my son always called me that so I know I am) a kick-start? A lua example of these would be really appreciated.
my_set_ISO(iso) // where iso could be integers or strings 100, 200, 400 etc
{
// ?
}
my_set_Tv(Tv) // where Tv could be strings "64", "32", "16" ... "1/128" etc
{
// ?
}
I do continue to study your references and do read your posts at least 3 times, but, I feel that a bit of grade-school hand-holding will go a long way.
Thanks.
-
Is a transaction the equivalent of a camera operation? If not, what is it?
PTP transaction, meaning an operation that sends or receives data from the camera. Executing a script is one transaction. Checking the status of the script (running or not, are there messages waiting) is one. Reading messages/results is one transaction. In practice a
lua ...
command in chdkptp results in several transactions, because it first checks if there is a script already running or messages waiting, and then it runs the code. If you wait for results (luar command) then it polls waiting for the script to end or messages to appear.
But again, this probably doesn't matter unless you are worried about tens of milliseconds, and you can't do much about it unless you understand the details and are willing to write lua code for your specific application.
I feel that a bit of grade-school hand-holding will go a long way.
If you are going to successfully use chdkptp for camera control, you are going to have to understand the CHDK functions on the camera, and write some lua code.
Quick and dirty example for Tv follows
!log2=function(n) return math.log(n)/0.69314718055995 end
!function my_set_tv(n) local tv96=string.format('%d',96*-log2(n)) ; printf("set tv96 %s\n",tv96); con:exec('set_tv96_direct('..tv96..')') end
Now to set the tv, you could do something like
!my_set_tv(1/1000)
or
!my_set_tv(10)
Explanation:
! runs lua code locally in chdkptp. Any global variables you make in an ! command are available later.
con:exec... executes code on the currently connected camera, much like the 'lua' command.
1/1000 works because this is lua code, so the division happens right there. It's not a string.
The log expression may not exactly match what the Canon firmware does, but it should be reasonably close.
tv96 is truncated using string.format('%d') because the camera only works in integers. Rounding might be slightly more accurate.
You can put the above ! commands in the same file you put contextplus option in. However, if you are writing more complicated code, you probably don't want to put everything in one line. In that case, you can put it in a .lua file (in the same directory as the other chdkptp .lua files) and then put something like
!require('myfile')
in the settings file. 'myfile' is the name of the file without the .lua extension.
ISO is, as they say, left as an exercise to the reader ;)
-
First of all, please accept my sincere gratitude for this. I like the way you wrote the sample because I can "map" it to C code easily in my head, get a basis, and expand from there if needed. That was very thoughtful. Also that you volunteered extra relevant info will be very helpful. You have to realize too, that you're getting me hooked on CHDK and I've only done some very basic things. But right now I am very curious about hardware operation performance I will get when running the SX110 under chdkptp and comparing that with my existing instrument that uses an S50 with RemoteCapture, simulating an operative instance of my instrument. Therefore I need to be as expeditious as possible, because if I'll be looking for more candidate cameras I have to turn my attention to that objective. Recall that CHDKPTP has opened the field very wide for candidate selection and I am very content about that, even though it will cost me much more time. As such I prefer to defer concentrating on learning lua and creating elegant code for my application till later. That will come once the final bricked hardware will be installed and working on the instrument.
My questions are very simplistic (as opposed to simple) for now. Kindly remember what noob stands for.
So please don't laugh. I want to put your sample code into one text file, and I want to send that text file to the camera while CHDKPTP is running, preferably from the CHDFPTP command line, or something equivalent. How do I do it?
-
That will come once the final bricked hardware will be installed and working on the instrument.
I guess you did not understand (or didn't read) the part where reyalp explained what "bricked" means ?
-
So please don't laugh. I want to put your sample code into one text file, and I want to send that text file to the camera while CHDKPTP is running, preferably from the CHDFPTP command line, or something equivalent. How do I do it?
Umm
You can put the above ! commands in the same file you put contextplus option in
Although if you aren't using the gui, you should make a different file and run that with the -r option, since the contextplus option won't be recognized.
The first two commands only have to be run once per chdkptp session, since they just define functions to use later.
To use the function, you'd just enter
!my_set_tv(1/100)
or whatever at the prompt.
However, if you are writing more complicated code, you probably don't want to put everything in one line. In that case, you can put it in a .lua file (in the same directory as the other chdkptp .lua files) and then put something like
!require('myfile')
in the settings file. 'myfile' is the name of the file without the .lua extension.
FWIW, chdkptp looks for files called _chdkptpguirc or _chdkptprc in your home directory (or the directory given by the environment variable CHDKPTP_HOME, if set) and runs those automatically when starting with or without the gui respectively. This and much more is described in USAGE.TXT in the chdkptp package.
-
@waterwingz
My interpretation of "bricked" by reyalp .... in short: "(irrecoverably modified not to standard) partially-functional hardware." Not correct?
@reyalp
My apologies that you had to re-quote, but I can see this more clearly in my present state of mind given some reading and thought. The bottom line is for what I want to do:
a) While I am watching the live image in CHDKPTP with contextplus enabled,
b) From command line, I call a .lua script that I (you) wrote with function parameters (Tv, ISO) then shoot that executes on the camera. If I can do that, I can start with my comparison testing.
I think you've given me the answer. I will go on this and see what I can do. Thanks.
-
@waterwingz
My interpretation of "bricked" by reyalp .... in short: "(irrecoverably modified not to standard) partially-functional hardware." Not correct?
I have a 4" thick dictionary in the basement somewhere that I believe probably contains all the adjectives in your interpretation. For someone simple like me, "bricked" means that your device is dead & broken - suitable only to hold a door open (aka using a brick as a door stop). Hence my amusement at your statement "That will come once the final bricked hardware will be installed and working on the instrument." Picture a standard 3"x2"x8" red brick bolted to your system.
(https://chdk.setepontos.com/proxy.php?request=http%3A%2F%2Fwww.riazali.com%2Fwp-content%2Fuploads%2F2012%2F07%2Fbrick.jpg&hash=ac17771661168948ec5dbde67295a7c0)
Update : I just realized what this discussion most reminded me of : Wake up Polly Parrot (http://en.wikipedia.org/wiki/Dead_Parrot_sketch)
-
@waterwingz .. cute thanks .. nothing like a good pic // I see why you laughed!
@everyone
Little problem ... I have not done anything new, no new code yet. I just noticed this ...
Mode: VIDEO_STD on PUP. If I press shoot_half it works, but only once after cold start. That is, subsequent shoot_halfs, or any other button after that 1st shoot_half, I get ERROR: a script is already running. Even shutdown doesn't work. I have to manually restart the camera.
In CHDKPTP if I change mode to AUTO after PUP, then all's fine .. multiple shoot_halfs, buttons, etc. If I change mode AUTO to VIDEO_STD, then I am only allowed one shoot_half, and it falls into the same problem. I restart camera and client to get back in.
If I am in VIDEO_STD and I have not pressed shoot_half, the other buttons like zoom work as they should.
Thus one shoot_half in VIDEO_STD locks out the GUI. I absolutely necessary to restart the GUI to clear the problem.
Because I am depending on VIDEO_STD for the SX110 ... can we resolve this problem?
-
Mode: VIDEO_STD on PUP. If I press shoot_half it works, but only once after cold start. That is, subsequent shoot_halfs, or any other button after that 1st shoot_half, I get ERROR: a script is already running. Even shutdown doesn't work. I have to manually restart the camera.
I guess you are using the shoot half button in the gui ? This actually does more than shoot_half, it waits for the camera to be ready to shoot a still, so AF, exposure etc are done. This doesn't work in video mode, the variables it waits for never change, so it just gets stuck.
I should fix this so it detects video mode, but in the meantime, you can send something like
.press('shoot_half');sleep(1000);release('shoot_half') in the console.
to just press shoot_half for a second.
Because I am depending on VIDEO_STD for the SX110 ... can we resolve this problem?
I'm not sure I understand why you are using shoot_half in video mode. My understanding was that in still mode, when you pressed shoot_half, it would switch to the same viewport resolution as video mode normally has, so you would use that as an alternative to doing your focusing or whatever in video mode. Maybe I misunderstood your posts.
edit:
The shoot half button in the gui is fixed in chdkptp revision 284. I haven't made a new build, but you can replace your existing gui.lua with the one here http://trac.assembla.com/chdkptp/browser/trunk/lua/gui.lua?rev=284 (http://trac.assembla.com/chdkptp/browser/trunk/lua/gui.lua?rev=284) (download link is at the bottom)
-
OK thanks, understanding what the gremlin is is good enough. The story of the shoot_half was that you mentioned that someone said a shoot_half would increase the resolution. I tried it and confirmed it back to you, in fact using the same snippet you sent here because the button push was not long enough to see it on the live view. As a reminder, I did ask also if I could hold shoot_half pressed in to keep resolution and shoot etc, but if I recall correctly you explained that operations can't be done that way.
My instrument works somewhat differently from usual camera where focus is done by the electronics driving the lens in a camera in a negative feedback loop. Here the feedback is human, so in order to focus, it's very simple ... all I need is the 240 line resolution // not even shoot_half. Therefore I am relying on VIDEO_STD, which gives me a constant 240. Hence the operation would look like this, hopefully by tomorrow, simulated with the SX110 on the bench:
PUP -> VIDEO_STD
observe LIVE and manually focus the instrument (simulated by moving the camera back and forth)
acquire_image(Tv_1, ISO_1)
observe LIVE and manually focus the instrument
acquire_image(Tv_2, ISO_2)
etc
function acquire_image(Tv, ISO)
Mode AUTO
my_set_Tv(Tv)
my_set_ISO(ISO)
shoot
(the wish here is to behave like RemoteCapture and send the image(s) directly to PC, ie not write to SD card)
Mode VIDEO_STD
end
Thus from what you say and my requirement, I won't press it. But if I press it by accident, then I have to restart // not a big issue ... just a flag. If you happen to fix it, let me know. All's fine for now.
-
As a reminder, I did ask also if I could hold shoot_half pressed in to keep resolution and shoot etc, but if I recall correctly you explained that operations can't be done that way.
If switching back and forth to movie mode works for you, that fine, but just for completeness sake:
You need to release the button between shots to set the exposure values, at least if you use the normal exposure control functions like set_tv96_direct etc. However, there's no reason you couldn't have shoot_half held down for however long you need to focus. The fact that normal CHDK script key presses will be released when the script ends makes it more complicated, but there are several possible workarounds
1) Use a message script. Then you can just send it a press and release command when needed. You can also define messages to set exposure values, shoot etc.
2) Use http://chdk.wikia.com/wiki/LogicalEvent (http://chdk.wikia.com/wiki/LogicalEvent) functions, e.g
press halfshoot with lua post_levent_to_ui('PressSwOne') and release it with lua post_levent_to_ui('UnpressSwOne')
Things may get confusing in you mix levent keypresses with chdk press()/release() key presses.
-
Correction I meant Mode M, not Mode AUTO.
Quote "However, there's no reason you couldn't have shoot_half held down for however long you need to focus."
That flexibility is great no question. So .......
Initialize:
PUP -> Mode M
lua post_levent_to_ui('PressSwOne')
observe LIVE and manually focus the instrument (simulated by moving the camera back and forth)
acquire_image(Tv_1, ISO_1)
observe LIVE and manually focus the instrument
acquire_image(Tv_2, ISO_2)
etc
function acquire_image(Tv, ISO)
lua post_levent_to_ui('UnPressSwOne')
my_set_Tv(Tv)
my_set_ISO(ISO)
lua press("shoot")
(the wish here is to behave like RemoteCapture and send the image(s) directly to PC, ie not write to SD card)
lua post_levent_to_ui('PressSwOne').
end
On the right track? Would anything conflict? If so, it's more elegant I think because it keeps the same mode.
-
Quote "You can put the above ! commands in the same file you put contextplus option in. However, if you are writing more complicated code, you probably don't want to put everything in one line. In that case, you can put it in a .lua file (in the same directory as the other chdkptp .lua files) and then put something like
!require('myfile') in the settings file."
This is interesting. Thus in summary this forces chdkptp to give me access to functions I wrote in myfile.lua, correct? If that is the case, it's exactly the solution I'm looking for.
FYI ... I am now searching for an S90 on ebay.
-
This is interesting. Thus in summary this forces chdkptp to give me access to functions I wrote in myfile.lua, correct?
Yes, require is roughly analogous to #include in C. See http://www.lua.org/manual/5.1/manual.html#5.3 (http://www.lua.org/manual/5.1/manual.html#5.3)
In chdkptp revision 286, I added a module with conversions from traditional shutter, ISO and aperture values to/from APEX and APEX 96. You can use this with
exp=require('exposure')
and then you can set the shutter and iso using
!con:exec('set_tv96_direct('..exp.shutter_to_tv96(1/100)..')')
!con:exec('set_sv96('..exp.iso_to_sv96(80)..')')
Obviously, you could also use those in in your my_set_tv functions.
I also added the log2 function to util.lua
If you want to use these, you will need to get the most recent util.lua and exposure.lua. I will probably put up a new snapshot build today.
-
Very kind of you to do this, as I have been dreading what to do with ISO on my own // please advise me when you have the new build up. For folks needing Av, perhaps it might be a good time too, if applicable.
With the code you offered earlier I put into a myfile.lua, I cleared out all the error messages during chdkptp load, so that looks OK, and, an execute also runs my functions, it seems.
But I have a little snag ....
The camera is set to MANUAL and Tv to Canon 15sec on PUP. So if I say !shoot(), it works, ie it takes a 15-second exposure.
Next if I put this code in myfile.lua ............
function test_shoot()
con:exec('shoot()')
end
log2=function(n)
return math.log(n)/0.69314718055995
end
function shootat_Tv(n)
local tv96=string.format('%d',96*-log2(n))
printf("set tv96 %s\n",tv96)
con:exec('set_tv96_direct('..tv96..')')
con:exec('shoot()')
end
... and say !test_shoot(), that works too, running the 15s Canon exposure as expected.
However, if I say !shootat_Tv(1) in an attempt to override the Canon 15s with a CHDK 1s and shoot, the camera LCD blinks with your message window, but nothing more happens.
Console is:
> !shootat_TV(1)
set tv96 0
Also if I remove the con:exec('shoot()') statement, behavior is the same. There are no other messages on the console.
>> I found an S90, with luck it should be here by next weekend. In the meantime I will be learning lua.
-
However, if I say !shootat_Tv(1) in an attempt to override the Canon 15s with a CHDK 1s and shoot, the camera LCD blinks with your message window, but nothing more happens.
This is probably because you are using the two exec commands without waiting. Exec by default works like the 'lua' command, i.e. returns control as soon as the command is sent, rather than when it completes. This probably means your second exec is hitting a "script already running" error, which you don't see because you aren't doing any error checking.
If you replace exec with execwait, it should only return after the command finished.
Note that you could also combine the two commands, e.g.
con:exec('set_tv96_direct('..tv96..');shoot()')
As an aside, if you are using manual mode, you might want to use set_user_tv96 etc instead of set_tv96_direct. The *user commands set the same values that can be entered manually in the firmware. _direct should work, but _user may make it more likely exif etc are correct.
Another aside, for real code you probably want some error checking. Most chdkptp lua functions return
status, [error message | result ]
in other words, if status if false, the second return value is the error message, otherwise it's whatever value would be returned by the function.
If you just want to print the status, you can use cli:print_status, like this
cli:print_status(con:exec('set_tv96_direct('..tv96..');shoot()'))
If you want to do something with the results then you'd use something like
local status, msg=con:exec('set_tv96_direct('..tv96..');shoot()')
if not status then
printf("error %s",msg)
else
...
If you use execwait, it defaults to acting like remote procedure call to the camera, returning status followed by any values returned by the camera code. So for example you can do
status,bv96=con:execwait('return get_bv96()')
to get the "brightness value" from the camera to chdkptp lua code.
You can customize the behavior of exec/execwait extensively by passing a table of options after the code. These are documented in chdku.lua. execwait is actually just a wrapper that sets the wait option to exec
-
Yup, I replaced the line ...
con:exec('set_tv96_direct('..tv96..')')
with
con:execwait('set_tv96_direct('..tv96..')')
... and said !shootat_Tv(1) That took the shot this time // but // the exposure was Canon's 15 sec, not CHDK 1s. So there is an improvement but we're not there yet.
Thanks for the detailed informative reply. I will read it closely and try the ideas you propose.
-
... and said !shootat_Tv(1) That took the shot this time // but // the exposure was Canon's 15 sec, not CHDK 1s. So there is an improvement but we're not there yet.
Try using set_user_tv96
-
Very very nice // that worked. In fact, the Canon display changed to 1 s after the shot. And as you mentioned .. exif data ... that too, is necessary.
Next, ISO, then we can officially start RemoteCapture<->CHDKPTP comparison testing // cool. I'm out for a dinner bite but will be back on this afterwards.
Talk to you later.
-
Minor gremlin we ought to look at ...
!shootat_Tv(1) results in a 1s shot with PUP preset of 15s // OK. OSD shows 1" afterwards. Looks normal.
!shootat_Tv(2 | 4 | 8 etc) results in the present OSD Tv instead of the CHDK Tv. Hmmm.
!shootat_Tv(values less than 1s) seems to shoot the camera faster than 1s (I cannot be certain of speed accuracy), but the OSD does not change, remaining at the same value it was prior to shot.
Console
for 2s result= -95
for 4s result= -191
for 1/2s result= 95
for 1/8s result= 287
Ideas?
-
Update ... the way it looks after several more attempts at isolating the problem is !shootat_Tv(1) is the only value that overrides the current OSD Tv, and shoots at 1s. If I try any other value, it shoots at the Tv currently displayed on the OSD.
-
Minor gremlin we ought to look at ...
"We" ? You are responsible for debugging your own code, thanks ;)
Console
for 2s result= -95
for 4s result= -191
for 1/2s result= 95
for 1/8s result= 287
Without testing, I suspect the Canon OSD isn't updating when the values don't exactly match a value known to the Canon UI. E.g. 2s should be -96. The difference between -95 and -96 is very small (2 vs 1.985...), but the Canon firmware doesn't have a display for that value.
If you use the code I added in revision 286, the results will be slightly more accurate. They still may not exactly match the firmware values.
edit:
It's also possible set_user_tv96 only accepts values known to the camera. That should be something you can check by experiment.
edit:
OK, I verified in the source that _user only accepts values in the shutter speed table.
edit:
if you set the propcase directly, that should work for non standard values, e.g.
con:execwait("props=require('propcase');set_prop(props.USER_TV,<your tv96 value>)")
-
I did not know I was doing this in isolation ... after all, it would have been impossible for me alone to debug the original A/R caveat // just being friendly :)
Quote "I suspect the Canon OSD isn't updating when the values don't exactly match a value known to the Canon UI. E.g. 2s should be -96"
For now, my "update" is more accurate and stands. I will hardcode -92 and let you know what happens.
Is r286 available?
-
Yes, -96 shoots 2s, and leave the OSD at 2".
-
Yes, -96 shoots 2s, and leave the OSD at 2".
See my edits in the previous post.
edit:
On my camera at least, set_tv96_direct does work fine in manual mode. It doesn't update the OSD (except in half shoot), but the exposure time is changed correctly. This works with values that are not in the shutter speed list.
-
Does that mean set_tv96_direct() is the only function that I can use to get 32s and 64s? EXIF is not updated with this function correct? Give-n-take.
-
Does that mean set_tv96_direct() is the only function that I can use to get 32s and 64s? EXIF is not updated with this function correct? Give-n-take.
exif may or may not be updated. Takes some pictures using obvious overrides (e.g. ui set to 1 sec, override to 1/100th) and see what happens.
If you use the propcase to set the user TV value as I suggested earlier, that will also probably work.
-
Another issue you may encounter: set_tv96_direct will not set the exposure correctly if it's exactly 1 second (APEX 0). You can get around this by using a apex96 value of 1 or -1
-
Can you show me an example of what deteriorates?
-
Can you show me an example of what deteriorates?
if your code ends up doing
set_tv96_direct(0)
then the shutter speed override value will be ignored, rather than doing a 1 second exposure. This is a long standing CHDK bug which I thought was fixed but actually never was. It applies to all of the set_tv* functions.
I'll try to fix this in CHDK later.
-
Ok, moving along .... here is the code that works for standard Canon values, updates the UI and supposed to update EXIF. I tried values from 1/250 to 15s on the SX110.
round=function(f)
return math.floor(f + 0.5)
end
log2=function(n)
return round(math.log(n)/0.69314718055995)
end
function shootat_Tv(n)
local tv96=string.format('%d',96*-log2(n))
printf("set tv96 %s\n",tv96)
con:execwait('set_user_tv96('..tv96..')')
con:execwait('shoot()')
end
I like the wait-on-shoot because it prevents access to the client (mouse pointer disappears) until the last file write to SD is complete. Good feature // prevents finger trouble.
-
I've uploaded a new build, details here http://chdk.setepontos.com/index.php?topic=6231.msg89893#msg89893 (http://chdk.setepontos.com/index.php?topic=6231.msg89893#msg89893)
The "shoot" command may or may not be useful for you, but the underlying functions are available as
exp.shutter_to_tv96 and exp.iso_to_tv96
A few notes on the code you posted above:
shoot() can hang indefinitely if the camera isn't in a still shooting mode (playback or video)
If you ever up inputting a Tv value that doesn't convert exactly to one of the valid values, you will take a picture with whatever the current setting are, without any warning to the user.
Under the current CHDK stable builds, if your tv is exactly 1 second, it will be ignored. I have put a fix for this bug in the unstable development branch. Once I'm convinced it doesn't break anything, I will back port it to the stable branch.
-
Quote "If you ever up inputting a Tv value that doesn't convert exactly to one of the valid values, you will take a picture with whatever the current setting are, without any warning to the user."
Does that mean this new function does acts according to the semantics of set_user_tv96() instead of set_tv96_direct()? .... i.e. .. not allow the extended CHDK Tvs?
... because ... this circumvents the temporary bug you mentioned and it seems to work....
function shootat_CHDK_Tv(tv)
local tv96_i = 96 * -log2(tv)
local tv96 = string.format('%d',tv96_i)
printf("set tv96 %s\n",tv96)
if tv96_i == 0 then
con:execwait('set_user_tv96('..tv96..')') -- workaround for CHDK bug
else
con:execwait('set_tv96_direct('..tv96..')')
end
con:execwait('shoot()')
end
Of course the UI undergoes its predictable behavior for the exceptional case of 1s.
-
... because ... this circumvents the temporary bug you mentioned and it seems to work....
function shootat_CHDK_Tv(tv)
local tv96_i = 96 * -log2(tv)
local tv96 = string.format('%d',tv96_i)
printf("set tv96 %s\n",tv96)
if tv96_i == 0 then
con:execwait('set_user_tv96('..tv96..')') -- workaround for CHDK bug
else
con:execwait('set_tv96_direct('..tv96..')')
end
con:execwait('shoot()')
end
Of course the UI undergoes its predictable behavior for the exceptional case of 1s.
Yes, that should probably work around the problem. Note, as I said earlier, you could also just set the user tv propcase directly, which should allow you to use arbitrary values, and would still update the UI etc.
An alternative is just to send the value 1 instead of 0. This is what I used in the shoot command. The result is when you request a 1 second exposure, you get a 0.992... second exposure instead. The difference is not likely to be detectable...
-
It's curious because I always feel you're ahead of me by three or four (major) posts! So today is catch-up day // I am downloading the new stuff, rereading all you wrote from the last couple of days, and will be trying your work. That doesn't guarantee my pea brain won't overload however.
Quote "you could also just set the user tv propcase directly, which should allow you to use arbitrary values"
I will definitely be looking into this.
Quote "An alternative is just to send the value 1 instead of 0 ... The result is when you request a 1 second exposure, you get a 0.992..second exposure instead"
In my case absolute accuracy doesn't matter too much. The ability of the imaging system to acquire a consistent Tv exposure image-to-image is more important, but it is definitely a nice trick to know.
-
Quote "I've uploaded a new build, details here http://chdk.setepontos.com/index.php?topic=6231.msg89893#msg89893 (http://chdk.setepontos.com/index.php?topic=6231.msg89893#msg89893)"
I am deeply moved by your gracious comment // you did stop me in my tracks I must admit. You may have some idea of the objectives of this project by now, but you probably don't realize fully yet how far your help may take it. More later, once I have presented the final comparison.
-
QUESTION ... where do I find the code/file/files/exes that implement your new shoot function that supports Tv, Av and ISO ?
-
QUESTION ... where do I find the code/file/files/exes that implement your new shoot function that supports Tv, Av and ISO ?
The build r291 in the chdkptp downloads area https://www.assembla.com/spaces/chdkptp/documents (https://www.assembla.com/spaces/chdkptp/documents)
The actual code for the shoot command is in cli.lua (the actual command implementation), rlib.lua (camera side component) and exposure.lua (APEX conversions)
-
Improvement tip: RemoteCapture closes the camera when I press [X] to close the app. CHDKPTP leaves the camera running on exit, and I've noticed the leftover state is often usable, requiring manual OFF. My suggestion is to issue [shutdown] on [X] to streamline things and make operation smoother.
-
New shoot -- basic test and EXIF.
The shoot command, so far, appears to work fine for controlling the camera.
-tv: Tested were 1/16s to 32s are are performed on the camera and update EXIF correctly.
-av: Tested were f2.8, f5.6 and f8 were tested and perform OK on the camera and update EXIF correctly.
-sv: ISO *seems* to do what is expected on the camera by photo assessment, but, EXIF does not get updated properly. Sometimes the values change, but don't the change to the value I gave. The strange values in my case (SX110) are 164, 265 and 1600. There may be more of those.
-
-sv: ISO *seems* to do what is expected on the camera by photo assessment, but, EXIF does not get updated properly. Sometimes the values change, but don't the change to the value I gave. The strange values in my case (SX110) are 164, 265 and 1600. There may be more of those.
As mentioned in the documentation, ISO is Canon "real" ISO, which is not the same value as displayed on the screen / exif ("market" ISO). See http://chdk.wikia.com/wiki/CHDK_User_Manual#Override_ISO_value (http://chdk.wikia.com/wiki/CHDK_User_Manual#Override_ISO_value)
You may now express your undying gratitude to the Canon marketing department ;)
On my a540 I noticed if the ISO isn't a standard value (available in the Canon menu) then it's not displayed in the Canon firmware review info. I haven't looked at the exif.
-
Improvement tip: RemoteCapture closes the camera when I press [X] to close the app. CHDKPTP leaves the camera running on exit, and I've noticed the leftover state is often usable, requiring manual OFF.
Usable or unusable ?
I have noticed some weirdness if you have switched from playback to record mode. Switching to play before you exit the app may leave things in a better state.
Another thing you might encounter is described here: http://chdk.wikia.com/wiki/Lua/PTP_Scripting#General_information_about_executing_Lua_code_over_PTP (http://chdk.wikia.com/wiki/Lua/PTP_Scripting#General_information_about_executing_Lua_code_over_PTP)
"Lua script execution currently leaves CHDK alt mode in a confused state. You may need to press the alt button a couple times to get the alt menu and buttons to respond as expected."
My suggestion is to issue [shutdown] on [X] to streamline things and make operation smoother.
I don't think I'll do this, I frequently restart the app without wanting to restart the camera. A "shutdown on exit" option could be added at some point.
-
Therefore ... if ask shoot for -sv=100 if see an EXIF value X. But if I ask for 200, I still see X. It appears under some conditions to be struck, for example, I had the value of 164 stuck for I think it was 100, 800 and 800. ??
Yes correct, unusable.
I have not switched modes away from still for several days. Not a big deal // it's just a convenience and at the very basic level it would perform like RemoteCapture.
-
Therefore ... if ask shoot for -sv=100 if see an EXIF value X. But if I ask for 200, I still see X. It appears under some conditions to be struck, for example, I had the value of 164 stuck for I think it was 100, 800 and 800. ??
The effect of CHDK overrides on EXIF values isn't very reliable, and varies between cameras and probably camera settings. To determine whether the override is working, you need to compare the light levels in the the photos, if you take pictures at e.g 100,200,400,800 there should be 1 stop of exposure difference visible in a histogram. Alternately, you could reduce the shutter speed by 1 stop and they should all be exposed about the same (histogram will vary slightly due to increased noise)
Once you've done that, you can examine the exif and determine whether the values are being recorded or not.
In some cases, CHDK overrides do not update the regular EXIF, but do update the Canon "maker note" fields.
You should also know that on many Canon cameras, the last normal ISO value is done in software rather than sensor gain, and CHDK overrides may not turn this on. E.g. on my a540, the factory max ISO is 800, but CHDK ISO override only works to 400. If you examine raws taken at 400 and 800, the pixel values are the same, meaning the "ISO" adjustment is really just cranking up the brightness in the jpeg.
On cameras with a separate "High ISO" or similar mode, this is something different again.
Yes correct, unusable.
If you can elaborate on how it becomes "unusable" it might be possible to do something about it. Note that the alt mode issue I mentioned can make it appear like the keys have no effect, but to clear this, you just have to press the button until the <alt> indicator appears, and press it again to get to non-alt mode.
-
I like the RAW ON|OFF option. Something useful too would be DARK SUBTRACT ON|OFF.
You may have a solution for this already, nonetheless, please refer to the figures. In my case I only set WHITE BALANCE from the RemoteCapture right panel, very occasionally. Since these are in general not very dynamic properties during typical photographic shooting (the general audience I am guessing), I am thinking of a separate function that could be called something like "set_camera" in the same style as shoot, which would handle the more universal (static) settings, like Marco ON|OFF, AF stuff, (left panel), JPG features, camera tilt (top), and those in the right panel (which includes my pet). Is something like this doable within reason?
====
Quote "To determine whether the override is working, you need to compare the light levels in the the photos"
Yes I did this and mentioned earlier, the answer is in-camera exposure brightness follows the logical progression.
Quote "Once you've done that, you can examine the exif and determine whether the values are being recorded or not"
I did that too, but the EXIF value recording behavior is inconsistent, even if the values are not what the are supposed to be.
Quote "You should also know that on many Canon cameras, the last normal ISO value is done in software rather than sensor gain, and CHDK overrides may not turn this on."
This is *very good* to know thanks.
But unusable for me, was restarting the cli and getting no response.
-
... in other words it seems there are short ranges of ISO values that don't the EXIF ISO variable.
-
I like the RAW ON|OFF option. Something useful too would be DARK SUBTRACT ON|OFF.
You can control this with the camera side lua function http://chdk.wikia.com/wiki/CHDK_scripting#set_raw_nr (http://chdk.wikia.com/wiki/CHDK_scripting#set_raw_nr) (note, the name is misleading, it isn't actually related to raw)
[/quote]
You may have a solution for this already, nonetheless, please refer to the figures. In my case I only set WHITE Is something like this doable within reason?
[/quote]
You mentioned a lot of different features, and I don't specifically plan to make a gui or cli command for any of them right now. Some things controlled either by chdk lua functions, propertycases, or simulated key presses. If you are using the live view, you can just set them using the regular on camera interface, using the buttons in chdkptp.
But unusable for me, was restarting the cli and getting no response.
So you try to reconnect with chdkptp, and you get no response ? no error message or anything ?
-
Quote "So you try to reconnect with chdkptp, and you get no response ? no error message or anything ?"
This is not major as I mentioned ... but .. Power ON. chdkptp. Connect, live, viewfinder rec. [X]. chdkptp. Connect, live, viewfinder (should still be live, camera LCD black) rec (no live). Play (still image). Rec (now live OK). shoot. [X]. chdkptp. Connect, live, viewfinder (live view OK). [X]. chdkptp. Connect, live, viewfinder (live view OK). shoot. etc OK. Can't recreate systematically // perhaps I could have done something else wrong that gave me trouble before // not sure.
Quote "If you are using the live view, you can just set them using the regular on camera interface, using the buttons in chdkptp."
Once the camera is mounted on the instrument I can't touch it. Many weeks from now anyway // revisit then. So cli functions like set_raw_nr will be useful. Eventually White Balance will be needed. For now, you have opened the door to functionality required to proceed with camera selection testing // that is fabulous.
====
EXIF ISO, is there anything you can do with it? Getting that info (even if not the Canon values) is very useful to me.
-
Once the camera is mounted on the instrument I can't touch it.
I meant the "up/down/left/right/set" etc buttons in the GUI. You can navigate the regular firmware options using those, right ?
EXIF ISO, is there anything you can do with it? Getting that info (even if not the Canon values) is very useful to me.
Did you check the maker notes?
This is not major as I mentioned ... but .. Power ON. chdkptp. Connect, live, viewfinder rec. [X]. chdkptp. Connect, live, viewfinder (should still be live, camera LCD black) rec (no live). Play (still image). Rec (now live OK). shoot. [X]. chdkptp. Connect, live, viewfinder (live view OK). [X]. chdkptp. Connect, live, viewfinder (live view OK). shoot. etc OK.
I don't really understand this, but depending on your USB driver and windows configuration, other software can put the camera in some mode where the screen goes black and chdkptp can't access it. It's also possible the camera can go into some kind of sleep mode on it's own, although the CHDK disable LCD off option should block that (and usually sending a simulated keypress will wake it up).
-
Sideshow snag, perhaps you can help ... I have been using Camera A on the laptop since the installation of CHDKPTP. Now I moved the CHDK SD card to Camera B. With no USB, CHDK comes up fine, ie, Camera B works OK untethered. However, if I plug USB, it comes up in Device Manager as:
Imaging Devices
Canon Power Shot
and the Canon download menu comes up instead of windows being silent. Of course CHDKPTP does not recognize it.
Moving the SD card back to Camera A, all is fine. In the Device Manager shows:
libusb-win32 devices
Canon Power Shot
and CHDKPTP works fu=ine.
It seems as though the installation (libusb??) memorized Camera A??
Help.
-
You may have to install the inf driver for each actual camera.
I have also had to install it for plugging into different ports, but this does not seem consistent.
edit:
said filter, should have said inf
edit:
I think on cameras that report a serial number, windows recognizes individual cameras. My a540 does not report a serial number, D10 does. I don't know where the cutoff is, possibly digic III or dryos.
-
Does this mean I have to install/uninstall the libusb driver each time I switch cameras?
-
Does this mean I have to install/uninstall the libusb driver each time I switch cameras?
You shouldn't need to uninstall it. You may need to install it for new cameras. You definitely need to install it for different models, don't know about different cameras of the same model (may depend on the serial number thing).
-
Thank you // first performance comparison hopefully tomorrow, without my silly typos I hope.
-
This is for your community. I have camera in M mode and testing client GUI controls.
1) With UI overlay ON on startup, the UI is responsive in the liveview. For example, WHEEL L and R operate Tv and I see it. However, after I press SHOOT the Canon UI disappears but the CHDK UI remains up and functional, eg. clock is ticking. Unchecking/checking UI overlay does not bring the Canon UI back. Pressing SHOOT HALF does bring back the Canon UI and I can proceed to set the Tv.
2) Your GUI is missing the "+/-" and "face" buttons. To be able to set focus on Av so that WHEEL can change it, the "+/-" has to be pressed first. That is, "+/-" toggles between Tv and Av.
-
1) With UI overlay ON on startup, the UI is responsive in the liveview. For example, WHEEL L and R operate Tv and I see it. However, after I press SHOOT the Canon UI disappears but the CHDK UI remains up and functional, eg. clock is ticking. Unchecking/checking UI overlay does not bring the Canon UI back. Pressing SHOOT HALF does bring back the Canon UI and I can proceed to set the Tv.
This is probably because vid_get_bitmap_active_buffer is not implemented yet in the sx110 port. I'll look into adding it.
2) Your GUI is missing the "+/-" and "face" buttons. To be able to set focus on Av so that WHEEL can change it, the "+/-" has to be pressed first. That is, "+/-" toggles between Tv and Av.
Based on the picture here http://chdk.wikia.com/wiki/SX110, (http://chdk.wikia.com/wiki/SX110,) I'd guess +/- is erase, which has a button in the gui.
If you need to send face, you can probably use click('face') in lua.
-
Yes "erase" functions as "+/-".
-
Here's a test build with vid_get_bitmap_active_buffer implemented.
I notice this port also doesn't have the palette functions. Do the colors in the UI overlay match what appears on the camera fairly well ?
-
Thanks .. I will check it out tomorrow as it's 1am here and I'm getting sleepy.
Quote "Do the colors in the UI overlay match what appears on the camera fairly well ?"
In fact not ... palette problem yes ... I was to bring this to your attention, but it was not on my high priority list. If you'd like a dump let me know and I will send one tomorrow morning.
-
COMPARISON #1 Canon's RemoteCapture vs CHDKPTP: performance and usability, at the date of this post.
GENERAL OVERVIEW
I begin by extolling the fundamental virtue of CHDKPTP: in addition to support for PowerShot cameras that have RemoteCapture capability, CHDKPTP can enable liveview and PC remote camera operation in cameras that Canon had removed the RemoteCapture feature from, allowing photographers PC control of the most recent PowerShots while enjoying the other benefits the newer cameras offer. To my knowledge, the 9 Mpx SX110 is the last PowerShot camera that supported Canon's RemoteCapture and is being used for comparison here.
This means that the CHDKPTP client (working with on-camera CHDK server) accesses the camera LCD display buffer to do the job. By giving the photographer a live copy of the camera LCD on the PC, Canon icons and CHDK UI information are visible on the PC as it appears on the camera LCD. Canon RemoteCapture cannot do that. Hence this UI feedback gives the user enhanced performance for long distance remote photography.
Canon has two versions of their RemoteCapture application program: RemoteCaptureTask for lower-end PowerShots and RemoteCapture DC higher-end PowerShots. RemoteCaptureTask transfers the shot file from the camera, increments the file number, and saves it to PC updating ZoomBrowser automatically on the PC while RemoteCapture DC moves the shot file from the camera to a user-specified folder without updating ZoomBrowser display. For the DC version the user has to press F5 in ZoomBrowser to show the new photo. Both clients increment and name the new files. The default naming convention is Capture_NNNNN.* Canon does not write any files to the camera SD card during remote operation.
CHDK has one version of CHDKPTP for all cameras. A shot file (or 2 files if RAW is enabled) is saved to the SD card in the camera. Files are named in the same convention as under manual camera usage. Files can be downloaded from the SD card to the PC and then erased using the Files tab in CHDKPTP.
To enable CHDKPTP operation, needed are:
1) CHDK installed on camera,
2) libusb device driver (easy installation, replaces Canon driver),
3) CHDKPTP installed on PC,
4) USB 2.0 + Canon cable (with a "cable bubble" EMI suppressor)
5) Preferably an AC adapter.
At the time of this writing, here is how the two stack up.
RemoteCapture CHDKPTP
Live View Yes Yes
Play View No Yes
UI (ie OSD icons) No Yes *
* Switching UI ON|OFF does not disturb the OSD on the camera LCD. OFF gives the same appearance as RemoteCapture.
The only push-buttons common to both RemoteCapture and CHDKPTP are "Shoot" and "Viewfinder ON|OFF." RemoteCapture replicates other camera settings dropdown-select controls whereas CHDKPTP has buttons that mimic the buttons on the camera. CHDKPTP thus can access most or all of the camera's operations and settings whereas RemoteCapture accesses only the minimum subset required for shooting such as white balance, exposure compensation and A/F lock. However, accessing those settings and operations are easier and faster with RemoteCapture. RemoteCapture controls zoom by a vertical slider. CHDKPTP does not use sliders. The two common controls are MODE selection which are dropdowns in CHDKPTP and RemoteCapture.
PERFORMANCE DIFFERENCES
The tests here were done with camera set to M (MANUAL)
RemoteCapture: The application I am targeting requires changing Tv and ISO for every shot. To achieve this the actions I have to perform are:
a) Turn OFF viewfinder (to increase response)
b) Click ISO dropdown
c) Scroll to select desired ISO
d) Scroll to the desired Tv
e) Click Tv dropdown
f) Click SHOOT
CHDKPTP: The actions I have to perform are:
a)
Comments: Dropdown in RemoteCapture is direct access and is fast when changing Tv by a large amount. To change Tv by a large amount requires a click of the WHEEL buttons for every
RemoteCapture: can save one filetype per shot.
CHDKPTP: can save JPG and/or RAW per shot, naturally with CHDK settings applied to the RAW.
RemoteCapture: No SHOOT-HALF
CHDKPTP: SHOOT-HALF capable button.
RemoteCapture: crisp live image in 320x240
CHDKPTP: crisp live image in 360x270 (clearest image is obtained with ContexIsPlus enabled)
RemoteCapture: EXIF data Tv Av ISO are recorded
CHDKPTP: EXIF data Tv Av are recorded, ISO is not working yet.
PROGRAMMATIC CONTROL
RemoteCapture
RemoteCapture does not have programmatic control. My application requires very fast processing of a) set ISO b) set Tv and c) SHOOT. To simulate programmatic control on RemoteCapture I use a keystroke/mouse macro app that plays a series of recorded user-interface actions by a single call to a macro. Hence I have several dozen macros that I call to shoot the required ISO+Tv conditions on the camera. A macro call is simply what I would do with the keyboard and mouse on RemoteCapture but in fast time-compressed sequence. Both mouse and keyboard have to be used for programming the selections and unfortunately, not all dropdown choices can be selected this way. The worst case response before the camera reacts is approximately 3 seconds.
CHDKPTP
CHDKPTP has programmatic control, directly from the PC. This is done by issuing a lua command or built-in library function to the camera from the CHDKPTP command line. A splendid new function has been added by the CHDK team recently that addresses the setup-and-shoot requirement in one call:
shoot -tv=<v> -sv=<v> -av=<v> (see cli.lua for complete syntax)
-tv=<v> Shutter speed. In standard units (seconds) both decimal and X/Y accepted
-sv=<v> ISO value. In standard units, Canon "real" ISO
-av=<v> Aperture value. In standard units, f number
An example is:
shoot -tv=1/128 -av=5.6 -sv=200
The functionality is complete in that it supports all three basic shooting parameters in a single action. If a parameter is omitted, the currently-set parameter in the camera is used for the shot. If the parameter matches an in-camera Canon parameter, the Canon UI is updated after the shot and the update is immediately visible on the live view. The performance of this function is amazing: pressing the EXECUTE button on CHDKPTP to start the SHOOT operation on the camera is almost instantaneous, as opposed to RemoteCapture which is 3 seconds for the camera to respond.
CHDKPTP's use CHDK's extended Tv range is transparent. For example:
shoot -tv=32 -av=2 -sv=1600
does what one expects.
This performance of setting parameters in a shoot command gives CHDKPTP a "professional feel" and here I would say CHDKPTP+CHDK smokes RemoteCapture.
-
WEAKNESS
There is one area where CHDKPTP is weak compared to RemoteCapture and could use some consideration from the CHDK team. The rationale is that when I am remote, my prime objective is shoot-and-view immediately on PC, without concerning myself with. RemoteCapture does that automatically by transferring the photo to the PC after shoot without saving to SD as explained above. At present CHDKPTP saves the files to SD. A single shot can exceed 20 MB, requiring several seconds for the save to SD. I have to open Files tab, copy the file to PC, and then erase it from SD, doubling or even tripling the time for the transfer and wearing out the SD card. Most important, the immediate transfer RemoteCapture has helps keep my focus on the imaging task rather than being distracted to do file handling. I thus propose a file transfer mechanism and naming convention that is like to RemoteCapture, where CHDKPTP+CHDK handle moving file data directly from the camera to the PC, and CHDKPTP looks at the last file in the destination directory, for example Capture_00034.DNG and saves the new one as Capture_00035.DNG. Useful is that RemoteCapture has an Options tab that allows me to:
a) set the target directory,
b) give a file prefix such as "My_Capture_NNNNN, and
c) give an arbitrary starting number for the first file saved.
-
In fact not ... palette problem yes ... I was to bring this to your attention, but it was not on my high priority list. If you'd like a dump let me know and I will send one tomorrow morning.
No, the palette isn't defined, so there's would be no palette in the dump. In this case, chdkptp uses a hard coded copy of the a540 palette.
I'll look into finding that.
edit:
Regarding the weakness above:
You can script downloading and deleting in chdkptp lua, just like all the other functions. However, there isn't currently a really convenient way to get the most recently shot image. There are ways to do it, for example, You could list all the files in the DCIM/NNNCANON directory and download the last one. If you delete after each download, then it will always be the only file in the list. Adding a more convenient way is on my todo list.
As I mentioned earlier, srsa_4c is working on capturing direct over USB. There is no ETA, but for raw/dng at least it should be supported at some point.
edit:
Attached a build with the palette set. It also includes the change in the previous build. The colors may be wrong, but if you make a dump it should give me something to work with.
-
Oops! I hadn't realized that the pen fell out of my hand half way through ... so for completeness:
COMPARISON #1 - revisited - 2/3 down:
CHDKPTP: The actions I have to perform are:
a) Enable UI overlay if not enabled,
b) Get focus on the Tv parameter if not enabled using the CHDKPTP camera navigation buttons,
c) Use WHEEL L and R to set the new Tv,
d) Get focus on the ISO parameter (for SX110 it's the UP button),
e) Use WHEEL L and R to set the new ISO
f) Press SHOOT
Comments: Dropdown in RemoteCapture is direct access and is fast when changing Tv by a large amount. To change Tv by a large amount in CHDKPTP requires a click of the WHEEL buttons for every Tv value. Overall, manual setting up the shot takes longer in CHDKPTP than in RemoteCapture.
WEAKNESS - revisited - 2nd sentence:
... without concerning myself with file transfer operations.
-
// re: palette // Quote "I'll look into finding that."
I can test it on the SX110 when you're ready. At present, gray icons appear blue and if I enter into Canon menu mode, the colors are completely different.
// re: file transfer weakness // Quote "You can script downloading and deleting in chdkptp lua, just like all the other functions. "
Yes I am slowly appreciating the power you offer with your implementation. For example I have envisioned something like:
shoot -tv=1/128 -av=5.6 -sv=20; transfer_file
One of the main reasons apart from enhanced CCD low-light technology to upgrade my instrument is to improve file transfer time. Of course doing it according to the example does automate the process, however, because two transfers are needed (save to SD, then transfer and save to PC), the fast USB 2.0 file transfer capability of the newer PowerShots is being taken full advantage of. Because RemoteCapture direct transfer is blazingly fast, I have a sense that writing to SD is notably slower than transferring the image data directly to PC. So if implemented (especially for the case of those large RAW files), CHDKPTP file transfer performance can be at least as good as RemoteCapture.
In my case, 90% of shots are JPGs, while the remainder are RAW. I would expect too, that the majority of photographers desiring the extensibility CHDKPTP+CHDK offers likely use JPGs. Thus the problematic issue would be how to handle the Canon JPGs w/o an SD write. I suppose because Canon firmware detects RemoteCapture it puts itself in the direct file transfer mode, and that may be inaccessible to CHDK. I think on account of that it might very difficult or impossible to do. Your thoughts?
Quote "As I mentioned earlier, srsa_4c is working on capturing direct over USB. "
That would be great, beginning with RAW files I understand. I know it's not trivial and I am very willing help in the test scenario.
I will be testing your new build today. Please be patient.
-
I can test it on the SX110 when you're ready. At present, gray icons appear blue and if I enter into Canon menu mode, the colors are completely different.
If you can make a with the last build I posted, that would be helpful. The most useful would be to use CHDK to display the palette (chdk misc -> draw palette), and get a picture of the actual LCD screen along with a chdkptp dump.
I suppose because Canon firmware detects RemoteCapture it puts itself in the direct file transfer mode, and that may be inaccessible to CHDK
Given that many cameras don't support Canon remote capture, it's possible the code doesn't even exist in the firmware (it might though, Canon diagnostic could have similar functionality). Even if it does, re-using it could be difficult. srsa_4c has shown it can be done in principle in our own code, but for jpeg it involves a significant amount of platform specific code on each camera. Given that we have well over 100 platforms, it's unlikely to be supported across all cameras quickly. Raw/dng can be done in a platform independent way.
-
for jpeg it involves a significant amount of platform specific code
I could probably make some ports (luckily there's only one supported revision of the SX110), but I'm concerned about the chdkptp-side implementation. Especially now that there's a brand new shoot command, I'm not sure how mine should be implemented.
-
I just got in so it will take some time to get up to speed.
Quote "I could probably make some ports (luckily there's only one supported revision of the SX110), but I'm concerned about the chdkptp-side implementation."
The S90 is on its way // here by weekend if lucky // and I don't know what impact that could have on all your work. Now that I know I can access its liveview and remote control, it would most likely be the CCD of choice. Certainly at present it has much higher precedence than SX110. Could you easily port the new CHDK stuff to the S90? If you are considering trying something platform-specific for direct JPGs, could you wait until I have the S90 up and it has passed its hardware acceptance tests?
Quote "Especially now that there's a brand new shoot command"
This command is really quite something. An idea I have is to make the command more flexible such as:
shoot {existing parameters} {filesave=<PC|SDcard>}
... for backward compatibility, but default is PC. A separate function that sets up the file manipulation state would very good too.
Quote " (chdk misc -> draw palette), and get a picture of the actual LCD screen along with a chdkptp dump."
Coming soon.
Thanks for the clear explanations.
-
QUESTION ... I seem to recall a comment or reading that I can't access the CHDK menus from CHDKPTP. Is that true?.. is there a way?
-
QUESTION ... I seem to recall a comment or reading that I can't access the CHDK menus from CHDKPTP. Is that true?
Yes, it's true,
.. is there a way?
No, there is currently no way to do it. You can set most configuration option with set_config_value http://chdk.wikia.com/wiki/Script_commands#get_config_value (http://chdk.wikia.com/wiki/Script_commands#get_config_value) but you can't currently send keypresses to the CHDK system, so you can't navigate CHDK menus or use CHDK shortcuts. This is something that should be supported, but it would require pretty major changes.
I suppose you might be able to poke physw_status from Lua... but that would be non-portable to say the least.
Especially now that there's a brand new shoot command, I'm not sure how mine should be implemented.
Yes, this needs consideration. I'll follow up in the ptp protocol thread.
-
So there no simple way to show the palette on the liveview (and dump) , correct?
With the new CHDK build up, I see no difference in the UI color mix-up problem. I will wait for your answer and if no, I will use what the setup can give now (ie camera LCD photo with dump). For photographers wishing to use CHDKPTP is would be nice to have the same colors.
-
I don't I can access it so here you go anyway.
PALETTE #1 - photo color are not very accurate.
dump
http://www.sendspace.com/file/xsxroo (http://www.sendspace.com/file/xsxroo)
-
So there no simple way to show the palette on the liveview (and dump) , correct?
No, use the camera directly. Go into the ALT mode and call the CHDK menu and choose the palette function. The liveview displays all actions. It's an easy way make screenshots.
msl
-
PALETTE #2 - photo colors are not very accurate. Background is a neutral gray, item fields are darker gray and the bright background around the camera os pink.
dump
http://www.sendspace.com/file/jlhajw (http://www.sendspace.com/file/jlhajw)
@msl
Ho that's good - I tried that before and it did not respond. I got it to work so PALETTE is coming after dinner. I can see now it's a royal mess // but do have a look at what I sent anyway in the meantime.
-
PALETTE #3
dump
http://www.sendspace.com/file/7p1kd2 (http://www.sendspace.com/file/7p1kd2)
-
Are you sure you are using the build posted in http://chdk.setepontos.com/index.php?topic=8585.msg90010#msg90010 (http://chdk.setepontos.com/index.php?topic=8585.msg90010#msg90010) (CHDK-sx110is-100b-1.1.0.2105-reyalp-test-2.zip built around 8/28 22:55) ?
The dump you posted PALETTE #3 does not contain palette data. It shouldn't be possible with this build
FWIW, I only need a dump with the palette showing, I don't need all the other ones. Since I already have the screenshot, you don't need to make another one, as long as the camera is in the same mode when you make the new dump.
PALETTE #1 - photo color are not very accurate.
If you are talking about the live view color, this is not a palette issue, since it's a YUV not paletted.
-
I was using test-1 in error. In test-2 UI Overlay doesn't work // the liveview in rec mode is the same as in play mode.
dump
http://www.sendspace.com/file/nme110 (http://www.sendspace.com/file/nme110)
-
The S90 is in and am beginning testing. To contribute my community service I remain at your disposal to continue tests with SX110 to at least resolve the UI color problem, EXIF ISO update gremlin, and give it a thorough user-exercise.
QUESTION: Do I use the stock version of CHDK for the S90? If you feel the changes you did for the SX110 will be useful for the S90 and you have a stable build, please give a link.
-
I was using test-1 in error. In test-2 UI Overlay doesn't work //
Can you be more specific what "doesn't work" means ?
the liveview in rec mode is the same as in play mode.
I don't understand what this is referring to.
QUESTION: Do I use the stock version of CHDK for the S90? If you feel the changes you did for the SX110 will be useful for the S90 and you have a stable build, please give a link.
All the changes I've made for sx110 are specific 110.
S90 is also missing the various live view related functions (viewport buffer size, palette, bmp buffer) so all those will have to be implemented. I'll try to put up a test build this evening. Live view should still work to some extent, just like it did in your initial tests with sx110.
edit:
OK, I see the overlay doesn't show up because with the current palette settings most of the ui is 100% transparent. This will take a little digging.
-
Quote "OK, I see the overlay doesn't show up because with the current palette settings most of the ui is 100% transparent."
Yes I can confirm that if I cover the lens for the liveview to go dark, then a few UI icons do show a bit. But color is still off, eg the set direction arrows on each side of Tv which are supposed to be a vivid green, are a light violet. There is not much else I can see easily so that's all I can help you with for now.
The S90 ..........
Yes an update CHDK build would be very very interesting. In CHDKPTP, what is the default liveview size in the normal 4:3 A/R? 640x480? Larger? If it is larger than 360x270 say ... could you please consider adding a 1/2-size checkbox, ie the option to show liveview at 320x240 for example? The reason is to able to optimize screen real-estate. I have this good feeling that the glorious CHDKPTP+CHDK will produce a razor sharp liveview for this camera.
Quote "I'll try to put up a test build this evening. Live view should still work to some extent, just like it did in your initial tests with sx110."
Are you talking about the test build or the stock build? I am trying to determine if it's worth trying the stock or wait for the test.
-
Yes an update CHDK build would be very very interesting. In CHDKPTP, what is the default liveview size in the normal 4:3 A/R? 640x480? Larger? If it is larger than 360x270 say ... could you please consider adding a 1/2-size checkbox, ie the option to show liveview at 320x240 for example
I'm not sure what it will be, I don't have the camera. chdkptp currently just takes the actual viewport width (or 1/2) and scales the Y to get the correct aspect ratio. I probably add other options at some point, but it's not a priority to me.
Are you talking about the test build or the stock build? I am trying to determine if it's worth trying the stock or wait for the test.
Live view should work in the stock build. It will have a fixed, possibly incorrect viewport buffer size. the UI overlay palette will be wrong, and the the overlay will also have the same update issue that sx110 had.
A dump or screenshot of the current stock build could be helpful for me.
-
Quote "A dump or screenshot of the current stock build could be helpful for me."
Sure. I'm just finishing up a dark imaging report I will post. Afterwards I will apply the stock installation to the S90. It's an 8GB card so I have to partition it etc, and send you the data as soon as I can // maybe an hour - 2 hr range.
-
LOW LIGHT IMAGING COMPARISON: CAMERA S70, S90 and SX110
Test subject was a low-profile wooden dolly hidden in room corner partially obscured by two large tables at a right angle to each other. Room nighttime dark. White light source: darkened computer screen pointing away from the equipment corner. To the eye individual pieces of equipment in the photographed zone on the bottom of the dolly are barely visible and not discernible.
The S50, which is the existing camera mounted on the instrument, and the S70 have the same dark and SNR performance so the S70 can be used as a faithful reference.
S70: reference exposure was set 15sec at f2.8 and ISO 400: dark exposure, expected.
SNR: poor
SX110: 15sec at f2.8 and ISO 400: dark exposure, approximately same brightness as S70.
SNR: good.
15sec at f2.8 and ISO 1600: 1-1/2 f-stops brighter than SX110 @ ISO 400
SNR: visible but good.
S90: 15sec at f2.8 and ISO 400: dark exposure +1 f-stop brighter than S70
SNR: excellent
15sec at f2.8 and ISO 1600: Same brightness as SX110 @ ISO 1600, good image purity
SNR: excellent, not visually perceptible.
15sec at f2.8 and ISO 3200: +1 f-stop brighter than S90 ISO 1600, as advertised.
SNR: no visually perceptible noise. RGB tone curve brightness increase by +2 f-stops results in some noise that barely noticeable.
CONCLUSION
Low-light performance for the SX110 over the S70 is 3-4 f-stops. Low light performance of the S90 over S70 is about 6 f-stops. Subjectively, color purity of the high ISO S90 images probably extends that performance of 7 f-stops over the S70. Hence the S90 passes the 1st phase of dark imaging, and has tentatively replaced the SX110 as the candidate for the instrument.
NEXT ...
The next set of exposure tests will require CHDK to examine dead pixels and dark noise w/o dark-subtract, and 64second Tv to determine CCD "amp glow" performance (corner-edge electroluminescence/NIR chip surface tunneling) .
-
HELP -- I can't get a FAT16 onto an 8GB SD already formatted to FAT32 in WinXP. Partition Magic does not see the card too. Do you know if the S90 will boot from the 8GB FAT32?
-
HELP -- I can't get a FAT16 onto an 8GB SD already formatted to FAT32 in WinXP. Partition Magic does not see the card too. Do you know if the S90 will boot from the 8GB FAT32?
It won't.
And you can't make a single FAT16 partition - that's the limitation of FAT16.
Full instructions here : http://chdk.wikia.com/wiki/Prepare_your_SD_card#For_cameras_released_before_2011_: (http://chdk.wikia.com/wiki/Prepare_your_SD_card#For_cameras_released_before_2011_:)
-
The partition is created as a 2M FAT12 ... and 8G FAT32. Any problem with a FAT12?
-
The partition is created as a 2M FAT12 ... and 8G FAT32. Any problem with a FAT12?
FAT12 should work.
To make test builds, I need to know which Canon firmware version your s90 has.
-
GM1.01A
-
Here's a test build for s90 101a. It has the viewport size, active bitmap and palette functions implemented.
Note that to update a multi-parition card, you need to get the diskboot.bin onto the small partition.
edit:
I don't have a full firmware dump from this camera. If you can make a fresh dump, that would be helpful.
You can use the canon basic dumper http://chdk.wikia.com/wiki/Canon_Basic/Scripts/Dumper (http://chdk.wikia.com/wiki/Canon_Basic/Scripts/Dumper) or the following lua script (using a build with native calls enabled, like the one I posted.)
--[[
@title Write memory to file
--]]
function printf(...)
print(string.format(...))
end
function errf(...)
error(string.format(...),2)
end
if (type(call_event_proc) ~= "function" ) then
error("your CHDK does not support native calls")
end
if (call_event_proc("System.Create") == -1) then
error("System.Create failed")
end
DUMPFILE="A/PRIMARY.BIN"
addr=0xFF810000
size=0x007EFFFC
f=call_event_proc("Fopen_Fut",DUMPFILE,"wb")
if (f==0 or f==-1) then
errf("Fopen_Fut %s failed %d",DUMPFILE,f)
end
r=call_event_proc("Fwrite_Fut",addr,size,1,f)
if (r ~= 1) then
printf("Fwrite_Fut return %d expect 1",size)
end
call_event_proc("Fclose_Fut",f)
-
S90 Untethered starts CHDK fine. I only tried bringing up menu and OK.
This is the stock build // thanks mi amigo ==>> I will test your new build tomorrow -- you beat me again :(.
---------------------------
When I first connected to chdkptp, could not get rec to work // froze camera with blue light ON. Had to remove battery to reset camera.
2nd time connected // it's working for now.
The problem .......
It seems to me like you're "zooming-in" something like cropping off the bottom part of the image // look at the camera LCD and SS of liveview.
Hopefully not a big job to fix // because you have 360x270 it seems with Scale A/R checked (but I didn't pixel-measure).
dump
http://www.sendspace.com/file/w0l1r4 (http://www.sendspace.com/file/w0l1r4)
It's very late and this is all I can do until tomorrow // I hope you can work with it ... in the meantime I will do the special CCD tests possible w CHDK.
-
When I first connected to chdkptp, could not get rec to work // froze camera with blue light ON. Had to remove battery to reset camera.
2nd time connected // it's working for now.
That is odd.
It seems to me like you're "zooming-in" something like cropping off the bottom part of the image // look at the camera LCD and SS of liveview.
As I said, viewport dimensions were probably wrong in stock build. From the dump, chdkptp is seeing a 240 line viewport, when it should probably be 480. Test build should fix this.
-
Yes I can confirm that if I cover the lens for the liveview to go dark, then a few UI icons do show a bit. But color is still off, eg the set direction arrows on each side of Tv which are supposed to be a vivid green, are a light violet. There is not much else I can see easily so that's all I can help you with for now.
If you can give me a dump of this build with the CHDK palette displayed, I can probably fix this. It won't show correctly in chdkptp, since this is a new palette encoding (VUYA with some kind of 2 bit? lookup on the A component), but that doesn't matter, the dump still includes the palette and bitmap pixels
I have it mostly working from you chdk_3192_20120830_044601.lvdump dump but the interpretation of the alpha component is unclear. You can see this in the spots around the edges of some icons in the screencap below.
-
SX110 PALETTE.ZIP -- I also gave you a hi-res view of the LCD screen.
http://www.sendspace.com/file/661no0 (http://www.sendspace.com/file/661no0)
-
Wrong thread.