Is this thing on? 🤣

In trunk rev 6354 I've changed cam_CalibrationIlluminant1 from 17 (Standard light A) to 1 (Daylight) on the following cams
sx220hs, sx230hs, sx240hs, sx260hs, sx280hs. I'll update stable later unless this post triggers a huge flame war.

Since this might seem like an odd thing to do for ports that have had the same DNG parameters for well over a decade, a long, excessively detailed rambling explanation follows.

A user in discord reported that it was impossible to get correct white balance for CHDK DNGs from SX280 in darktable with the illuminant set to 17.

All affected cams share the same color matrix and illuminant with SX280, so it seems certain if the illuminant is wrong on one, it's wrong for all. The sensors might be different, but it's extremely unlikely they'd be different in a way that they each end up using the same calibration matrix when using different light sources. Additionally the following cams use the same matrix, but use (and have always used) illuminant 1
ixus115_elph100hs, ixus125_elph110hs, ixus220_elph300hs, ixus230_elph310hs, ixus240_elph320hs, n and n_facebook

Using DNG samples for sx220, sx230 and sx260 from https://raw.pixls.us, I confirmed that in darktable (5.6.0, Windows), the colors like quite "off" with illuminant 17, and look more natural with 1. See attached, illuminant 17 on the left, 1 on the right. All exported with default settings in darktable.

Illuminant modified in chdkptp using:

dngload orig.dng
!ifd=dngcli.selected:get_ifd{0}
!ifd.byname.CalibrationIlluminant1:setel(1)
dngsave mod.dng

Strangely, both Raw Therapee (5.12, windows) and Adobe dng_validate.exe (1.4) produce visually identical output regardless of the illuminant. The output looks more or less like what darktable produces with illuminant 1.

dng_validate command:

dng_validate.exe -v -16 -tif file.tif file.dng

I confirmed the dng_validate output is pixel identical by loading the resulting images in gimp, exporting without metadata, and comparing hashes.

This surprises me. As far as I understand the DNG spec, it doesn't seem like ignoring the illuminant should be an option, and a calibration based on 17 (approximating an incandescent) should be wildly different from one using 1 (daylight). I had assumed that dng_validate would conform very well to the spec. Skimming the code, I do not obviously see it using the illuminant value, though I easily could have missed it.

In any case, I'm fairly convinced that 1 is more correct in darktable, and users of programs that behave like RT or dng_validate will see no change, and so will be no worse off.

The fact that only some raw converters are affected probably helps explain why this went unnoticed for 14 years.

This leaves several questions:

Could this be a darktable bug?
There could be bugs in the darktable implementation, but it seems pretty clear 17 and 1 shouldn't both be correct, and 17 looks wrong.

How did we up with the matrix copied but different illuminant values?
Digging through the history, the first camera with the matrix was sx220, added in r1230 2011/07/09, originally with illuminant 1, and subsequently copied to the other cameras that use this matrix with illuminant 1. In r1652 (2012/02/10) the illuminant was changed to 17 as part of the CHDKDE merge. I was unable to find the corresponding CHDKDE source (the assembla repo was created in 2016 and only has a few commits), so it's unclear whether this as mistake in the merge, or came from CHDKDE, and if so whether the change in CHDKDE was intentional. The matrix and illuminant 17 was then used in the sx240 and sx280 ports.

Additionally sx40 and ixus310_elph500hs use the "same" matrix with illuminant 1, except the divisor value is 100,000 rather than 1,000,000. This is strange and seems likely to be a mistake, but I haven't investigated the impact on image rendering. The ixus310 was added in r1399 on 2011/09/14, sx40 in r1414 on 2011/11/12 so both came after the first use of this matrix.

What about other cameras using other illuminants?
Numerous other cameras use illuminants other than 1. Even if they were correctly calibrated with a different illuminant, they'd still be wrong in programs that apparently ignore it. Skimming the source, there a couple general patterns
Two matrix cams with DNG calibration derived from Canon raw generally use 17 for the first matrix, and 21 (D65) for the second. I haven't tested any of these in darktable yet, but the ones I have seem fine in Raw Therapee.
One matrix cams that use 21 (D65). This is probably fine, since D65 should be quite similar to daylight.