What is the difference between -raw and -dng in CHDK PTP??? - General Discussion and Assistance - CHDK Forum supplierdeeply
« previous next »
Pages: [1] 2 3   Go Down

What is the difference between -raw and -dng in CHDK PTP???

  • 24 Replies
  • 3209 Views
*

Offline blenderbach

  • **
  • 69
What is the difference between -raw and -dng in CHDK PTP???
« on: 02 / June / 2023, 17:39:58 »
Advertisements
I took two pictures, one using rs -raw and the other with rs -dng commands in CHDK PTP. The DNG opens up in all softwares with a black border around the image plus the entire image area of the sensor is visible. The RAW, on the other hand, opens as RAW image data in other software, and produces a black and white version of the image with the black border, with a lot of noise. However, RawTherapee can open this RAW normally, but for some reason, the black border is completely gone and it even slightly crops the image, even though, from what I can tell, all cropping and processing is turned off. Any ideas? Thanks!
Logged
*

Offline reyalp

  • ******
  • 14080
Re: What is the difference between -raw and -dng in CHDK PTP???
« Reply #1 on: 02 / June / 2023, 19:58:40 »
Quote from: blenderbach on 02 / June / 2023, 17:39:58
I took two pictures, one using rs -raw and the other with rs -dng commands in CHDK PTP. The DNG opens up in all softwares with a black border around the image plus the entire image area of the sensor is visible. The RAW, on the other hand, opens as RAW image data in other software, and produces a black and white version of the image with the black border, with a lot of noise. However, RawTherapee can open this RAW normally, but for some reason, the black border is completely gone and it even slightly crops the image, even though, from what I can tell, all cropping and processing is turned off. Any ideas? Thanks!
With -dng, you get a DNG format image, which includes metadata and should be loadable by any software that correctly implements DNG support. With -raw, you get "CHDK raw" which is just a framebuffer dump, and is only supported by software that implements CHDK raw support for the specific camera. The underlying raw data is effectively the same in either case, but software may render them differently.

With DNG, how much of the sensor area is rendered may vary depending on the CHDK "default crop" setting, and whether the particular software renders the default crop or active area.

edit:
The difference in brightness and apparent noise is just due to the software using different defaults for the conversion from raw to displayable colors for the different formats.
« Last Edit: 02 / June / 2023, 20:01:33 by reyalp »
Logged
Don't forget what the H stands for.
*

Offline blenderbach

  • **
  • 69
Re: What is the difference between -raw and -dng in CHDK PTP???
« Reply #2 on: 02 / June / 2023, 20:51:21 »
Quote from: reyalp on 02 / June / 2023, 19:58:40

With -dng, you get a DNG format image, which includes metadata and should be loadable by any software that correctly implements DNG support. With -raw, you get "CHDK raw" which is just a framebuffer dump, and is only supported by software that implements CHDK raw support for the specific camera. The underlying raw data is effectively the same in either case, but software may render them differently.

Thanks for the information, but I have selected the entire sensor for the DNG, not a crop. When I do DNGs, they come out as a full sensor image, along with the black non-working part of the sensor border. For some weird reason, when I do the plain RAW, it crops the image. I thought it's supposed to be the RAW framebuffer, so technically it shouldn't cropping either. It should be the full sensor image, and yet, it's cropping. Is it an issue with CHDK PTP, or is there a reason for this? Thanks.
Logged
*

Offline reyalp

  • ******
  • 14080
Re: What is the difference between -raw and -dng in CHDK PTP???
« Reply #3 on: 02 / June / 2023, 23:06:08 »
Quote from: blenderbach on 02 / June / 2023, 20:51:21
For some weird reason, when I do the plain RAW, it crops the image. I thought it's supposed to be the RAW framebuffer, so technically it shouldn't cropping either. It should be the full sensor image, and yet, it's cropping. Is it an issue with CHDK PTP, or is there a reason for this? Thanks.
No. It's almost certainly a default crop hard-coded in whatever program you are using view the CHDK raw.

When you use -dng in chdkptp, it gets the raw data in exactly the same way it would with -raw, and then combines it with a DNG header (and swaps the byte order as required for DNG).
Logged
Don't forget what the H stands for.
*

Offline blenderbach

  • **
  • 69
Re: What is the difference between -raw and -dng in CHDK PTP???
« Reply #4 on: 03 / June / 2023, 03:03:07 »
Perhaps maybe you know of a software that I could use to edit/view the RAW raws? Because I was using RawTherapee and RawHide. Both softwares had this crop. Those were RAWs from my S3IS. When it came to my SX40HS, they couldn't even open the RAW. I have no clue what to do. Since you've been doing this for many years, you probably have some expertise in this area, and perhaps know of a software that can do this.

The reason I want the RAW raws, is to do some sensor testing. I'm interested to see how many dead pixels I have and also I want to see if I can unlock that black border of dead sensor to perhaps make it work, bringing my 12.1MP to a full 13MP.

Afterwards, I have a certain idea to test. I want to see if I can use CHDK PTP while recording a video, and seeing if CHDK PTP can offload each RAW movie frame as soon as it hits the movie buffer. After all, the frames have to be captured first, meaning they are RAW before they hit the conversion. If I can grab a hold of them, through the CHDK PTP, I can get my laptop to do all the heavy lifting, while the camera just relays the data over USB. Since upon some research, it seems that even if I could make it happen in-camera, the SD Card can't take such high speed RAW offloading, but USB might.

The thing is, it seems that video research for CHDK is being looked over and pretty much ignored. Last talk about video was like 2008-2009. The thing is, video is also a major part of these cameras. CHDK runs on a bunch of cameras, and yet, only works with photos, and the only thing with video is the compression. Seems to me that if I don't do it, no one will.
Logged
*

Offline reyalp

  • ******
  • 14080
Re: What is the difference between -raw and -dng in CHDK PTP???
« Reply #5 on: 03 / June / 2023, 21:10:36 »
Quote from: blenderbach on 03 / June / 2023, 03:03:07
Perhaps maybe you know of a software that I could use to edit/view the RAW raws? Because I was using RawTherapee and RawHide. Both softwares had this crop. Those were RAWs from my S3IS. When it came to my SX40HS, they couldn't even open the RAW. I have no clue what to do. Since you've been doing this for many years, you probably have some expertise in this area, and perhaps know of a software that can do this.
I'd suggest using DNG. As I described before, the actual image data is the same. The difference is that for CHDK "raw", each program needs special support for every camera. As for software, it depends what you actually want to do with it. For processing raw into pretty pictures, raw therapee is fine.

For analyzing raw data, you probably want something else. If you want to inspect unprocessed raw data from a CHDK DNG, you can use the chdkptp dngdump command to extract it in a more common format, for example
Code: [Select]
dngdump -rfmt=16pgm -raw
gives you a 16 bit, non-debayered greyscale PGM, which you could load into gimp or convert further with common tools.

Quote
The reason I want the RAW raws, is to do some sensor testing. I'm interested to see how many dead pixels I have
If you use DNG, you can use the chdkptp dnglistpixels command to identify pixels with values in a particular range. See https://app.assembla.com/spaces/chdkptp/wiki/DNG_Processing

Quote
and also I want to see if I can unlock that black border of dead sensor to perhaps make it work, bringing my 12.1MP to a full 13MP.
This is extremely unlikely, IMO. At least some of the black borders are physically masked on the sensor, so the are already read out, but contain only noise and dark current.

Quote
Afterwards, I have a certain idea to test. I want to see if I can use CHDK PTP while recording a video, and seeing if CHDK PTP can offload each RAW movie frame as soon as it hits the movie buffer.
IMO, there is no raw data visible to the main CPU during movie recording. Whatever YUV data eventually gets turned into compressed video is generated directly during readout, similar to how live view works, and indeed, in some cases the live view buffer is use directly for video. So at best, you will find uncompressed YUV at the same resolution as the native video file.

It has to be this way, since the system does not have the bandwidth to read out and process full resolution images at video frame rates, and again, we know the live view works this way.

Quote
After all, the frames have to be captured first, meaning they are RAW before they hit the conversion. If I can grab a hold of them, through the CHDK PTP, I can get my laptop to do all the heavy lifting, while the camera just relays the data over USB. Since upon some research, it seems that even if I could make it happen in-camera, the SD Card can't take such high speed RAW offloading, but USB might.
These cameras only support USB 2. In practice, the fastest CHDK supported cams top out around 33 mbyte/s of user data, and older ones are significantly less.

FWIW, what you describe is essentially how the chdkptp live view works.
Logged
Don't forget what the H stands for.
*

Offline blenderbach

  • **
  • 69
Re: What is the difference between -raw and -dng in CHDK PTP???
« Reply #6 on: 04 / June / 2023, 00:19:45 »
Alright, thanks for the information!

Quote from: reyalp
IMO, there is no raw data visible to the main CPU during movie recording. Whatever YUV data eventually gets turned into compressed video is generated directly during readout, similar to how live view works, and indeed, in some cases the live view buffer is use directly for video. So at best, you will find uncompressed YUV at the same resolution as the native video file.

It has to be this way, since the system does not have the bandwidth to read out and process full resolution images at video frame rates, and again, we know the live view works this way.

So, you're saying it's possible to get raw 640x480 video? Since you mentioned I can find uncompressed YUV at the same resolution.

Also, you talked about bandwidth and read-out limitations, so what if we cheat a bit? Instead of filming at full-resolution, why not film at 720i? The camera is capturing a handful of lines, instead of progressive. Shouldn't that lighten the load and allow for 720 resolution in video?

Although, if I recall correctly, no one has even figured out how the video is created by these cameras, so I don't know... Not to mention Canon Camcorders like the Vixia HF S20 aren't even supported by CHDK, yet they also have a photo mode, that getting a RAW image from would be nice, but yeah... I have no idea...  ???
Logged
*

Offline reyalp

  • ******
  • 14080
Re: What is the difference between -raw and -dng in CHDK PTP???
« Reply #7 on: 04 / June / 2023, 17:16:42 »
Quote from: blenderbach on 04 / June / 2023, 00:19:45
So, you're saying it's possible to get raw 640x480 video? Since you mentioned I can find uncompressed YUV at the same resolution.
Uncompressed 8 bit YUV411 is not "raw" in the normal sense of the word, and I said at best, you might find it.

That said, depending on the camera and shooting mode, you can already get resolutions something like that with PTP live view. Whether the image quality "better" than the standard video with high quality compression is debatable, and of course PTP live view suffers some other drawbacks like low frame rate and not being synchronized with camera frames.

Quote
Also, you talked about bandwidth and read-out limitations, so what if we cheat a bit? Instead of filming at full-resolution, why not film at 720i? The camera is capturing a handful of lines, instead of progressive.
Well, if you reverse engineer the subsystems that handle sensor readout, altering the parameters would certainly be a thing you could explore.
Logged
Don't forget what the H stands for.
*

Offline blenderbach

  • **
  • 69
Re: What is the difference between -raw and -dng in CHDK PTP???
« Reply #8 on: 04 / June / 2023, 19:57:59 »
Hmmm, alright.

But I am still kind of confused. Videos are made by the sensor, thus the sensor must be capturing 30 frames per second, and then the camera has to convert the frames to the AVI. The frames must be getting stored somewhere before being saved, at least I think. I mean, the sensor always takes in a RAW image frame, and each frame goes somewhere. Does anyone know yet where the frame gets unloaded to? Or is it some extra part of the camera that is inaccessible to anyone?

Regarding dumping live view data, it seems that sometimes we get frame-skips, and the resolution is completely off. When I dumped some of the buffer stuff using "rmem 0x1900 10485760 -f=10megs.bin.raw", and opened it in TiledGGD, I got the overlay data being 360x240pixels. Apparently at 8bits/pixel, but for some reason, live view either stretches everything, or warps everything, and everything is all over the place. I mean I'm getting 720x540pixels. Upon some further inspection, a heavily "liney" and gritty version of the Canon Splash Screen appeared, also at 360x240. A bit lower down, was a very grainy and "liney" image of the live view, and all colors are completely off, but still 360x240. And lower down was a much larger version of this live view image, also very grainy and "liney", and everything is black and white or weirdly colored stuff. I feel that I might have found where the live high-res frame is stored or ARE stored, but I'm not sure. All I know is that I turned on the camera, hit the "record video" button, and used the "rmem 0x1900 10485760 -f=10megs.bin.raw" in CHDK PTP, then using TiledGGD, I opened the RAW data and managed to pull these images out, and one of them (the big one) seems to be cut off as I only pulled 10mb, there might be more data if I pull pull some more megabytes from the memory. But yeah, do you know what this data is, that I managed to extract? Because it seems that I found some stuff.



Logged
*

Offline reyalp

  • ******
  • 14080
Re: What is the difference between -raw and -dng in CHDK PTP???
« Reply #9 on: 04 / June / 2023, 21:34:07 »
Quote from: blenderbach on 04 / June / 2023, 19:57:59
Hmmm, alright.

But I am still kind of confused. Videos are made by the sensor, thus the sensor must be capturing 30 frames per second, and then the camera has to convert the frames to the AVI. The frames must be getting stored somewhere before being saved, at least I think. I mean, the sensor always takes in a RAW image frame, and each frame goes somewhere.
No, the values read out can be combined before they hit system memory, for example by binning. Pixels / rows can also be discarded. See https://www.allaboutcircuits.com/technical-articles/ccd-binning-imaging-systems-increasing-signal-to-noise-ratio-frame-rate/

Video and live view modes almost certainly use binning and other processing (such as converting to YUV) before the data is transferred to main memory.

Consumer imaging systems generally have low resolution, high frame rate readout modes specifically to support features like video and live view. Some time spent reading about how these systems actually work might help you get a better understanding of the sort of things that are likely to be possible on these cameras.

Quote
Regarding dumping live view data, it seems that sometimes we get frame-skips, and the resolution is completely off. When I dumped some of the buffer stuff using "rmem 0x1900 10485760 -f=10megs.bin.raw", and opened it in TiledGGD, I got the overlay data being 360x240pixels. Apparently at 8bits/pixel, but for some reason, live view either stretches everything, or warps everything, and everything is all over the place. I mean I'm getting 720x540pixels. Upon some further inspection, a heavily "liney" and gritty version of the Canon Splash Screen appeared, also at 360x240. A bit lower down, was a very grainy and "liney" image of the live view, and all colors are completely off, but still 360x240. And lower down was a much larger version of this live view image, also very grainy and "liney", and everything is black and white or weirdly colored stuff.
There can be a variety of different resolution versions of the live view present in memory, with multiple copies of each. 720x540 in particular is quite common when recording video on older cameras (I suppose this makes my earlier comment that you'd only the same resolution as the video file not *quite* true, but the practical difference is small)

Regarding the color, on cameras before digic 6, as far as we know, the live view is always YUV 411 in  UYVYYY order.

Not if you set gui_verbose=2 while running the chdkptp GUI, it will report the resolution in the console when live view is connected or when it changes.
Logged
Don't forget what the H stands for.
Pages: [1] 2 3   Go Up
« previous next »
 

Related Topics