My first thought, looking at the webpage: "Huh, that's neat. I didn't know that painting software didn't even attempt to do color mixing beyond naive interpolation, though I guess it figures; the physics behind all the light stuff must be fairly gnarly, and there's a lot of information lost in RGB that probably can't be just reconstructed."
Scrolling down a bit: "Huh, there's some snippets for using it as a library. Wait, it does operations in RGB? What's going on here?"
Finally, clicking the paper link, I found the interesting bit: "We achieve this by establishing a latent color space, where RGB colors are represented as mixtures of primary pigments together with additive residuals. The latents can be manipulated with linear operations, leading to expected, plausible results."
That's very clever, and seems like a great use for modern machine learning techniques outside the fashionable realm of language models. It uses perceptual color spaces internally too, and physics based priors. All around very technically impressive and beautiful piece of work.
It rhymes with an idea that's been floating in my head for a bit - would generative image models, or image encoder models, work better if rather than rgb, we fed them with wavelength data, or at least a perceptually uniform color space? Seems it'd be closer to truth than arbitrarily using the wavelengths our cone cells happen to respond to (and roughly, at that).
Consider that just after the cone cells, there are other cells doing some computation / aggregate of cone signals. Don't forget that color is a brain construct.
For those reasons (and others), there's often a strong disconnect between stimuli and perception, which means there's no such thing as a perceptual uniform color space.
Sidestepping what is defined as an "edge", quite a lot of work is done in the retina, including differential computation across cones - some "aggregator" cells will fire when it detect lines, movement, etc.
You can read on ganglion cells, bipolar cells and amacrine cells and see that a lot of preprocessing is done before even reaching the brain!
AI and machine learning aren't necessary at all. You 'just' have to empirically measure a few constants that describe and bound the various nonlinearities of different real pigments, and then plug them into a relatively straightforward paint mixing equation.
Paints have predictable mathematical properties in terms of what color they produce when mixed; they just mix nonlinearly, which is counterintuitive for people who have not practiced mixing paint a lot.
Photoshop and the other comparison programs on the page illustrate the linear mixing that most people intuitively expect.
As far as I know, most painting apps mix color in sRGB space instead of the linear RGB space. Which means they're even "worse" than naive interpolation.
My first thought, looking at the webpage: "Huh, that's neat. I didn't know that painting software didn't even attempt to do color mixing beyond naive interpolation, though I guess it figures; the physics behind all the light stuff must be fairly gnarly, and there's a lot of information lost in RGB that probably can't be just reconstructed."
Scrolling down a bit: "Huh, there's some snippets for using it as a library. Wait, it does operations in RGB? What's going on here?"
Finally, clicking the paper link, I found the interesting bit: "We achieve this by establishing a latent color space, where RGB colors are represented as mixtures of primary pigments together with additive residuals. The latents can be manipulated with linear operations, leading to expected, plausible results."
That's very clever, and seems like a great use for modern machine learning techniques outside the fashionable realm of language models. It uses perceptual color spaces internally too, and physics based priors. All around very technically impressive and beautiful piece of work.
It rhymes with an idea that's been floating in my head for a bit - would generative image models, or image encoder models, work better if rather than rgb, we fed them with wavelength data, or at least a perceptually uniform color space? Seems it'd be closer to truth than arbitrarily using the wavelengths our cone cells happen to respond to (and roughly, at that).