Ein Beitrag zur Optik der Farbanstriche

November 9, 2012 6:25 AM

For years now, the primary way of representing and storing color on a computer display has been to define it as existing in three dimensions: Red, Green, and Blue. What if that's wrong? “While the appearance of a color on a screen can be described in three dimensions, the blending of color actually is happening in a six dimensional space,” How Fifty-Three, developers of the iPad painting app Paper, used a theory of paint optics from 1931 to develop a better color mixer.
posted by gauche (28 comments total)

Are colors actually stored with six parameters on iPads or is the "magical tech" that these guys simply made a color calculator with the German guys' algorithms plugged in? The article seems to go out of its way to avoid mentioning that Adobe products aren't the only place you would work on mixing colors and myriad stand-alone color calculators with different approaches exist, mostly free ones.
posted by XMLicious at 6:53 AM on November 9, 2012 [1 favorite]

If a monitor has a limited and discrete gamut, and the RGB system allows one to reach every point on that gamut, then I don't see how that could be significantly improved upon. Perhaps you could make a more intuitive representation, but a combination of RBG and HSV has always gotten me what I wanted fairly easily. What problem does this solve?
posted by Scientist at 6:59 AM on November 9, 2012

The problem it solves is that if you draw a line between two points in RGB space representing a gradient between colors, some of the resulting colors in the middle are going to be unexpected and uninteresting to someone who is using that gradient as a model for paint blending. It sounds like Paper is going for a more physically-oriented, intuitive approach than your standard image editing software, so it's not much of a stretch to see how that would be a stumbling block for them.
posted by invitapriore at 7:15 AM on November 9, 2012 [3 favorites]

I couldn't quite tell from the article whether they actually kept the 6-dimensional representation under the hood, or whether that just helped give them new ideas.

It sounds like the major feature is that they've encoded some human perceptual expectations about paint blending into the app, so the blends move through a warped "human colour space" rather than RGB or any other more standardised space.

I'd be interested to see more blend curves between their 100 test pairs, especially to find out if green stands out as a commonly warped line, because I think human eyes are most sensitive to green light. Maybe the trick to blending between colours is to keep the perceptual brightness the same by pushing the green channel.
posted by lucidium at 7:16 AM on November 9, 2012 [1 favorite]

You can get each color, and think of each color located on a 3 dimensional surface, or manifold.

The problems come in if you want to find a smooth continuous path from one color to another, and hope to do so by smoothly interpolating the values for two colors. That will work fairly well if the path goes around a color wheel at a fixed luminescence and saturation. Other paths will need to follow very jagged and noncontinuous trajectories in the R,G,B values in order for the color sequence to appear smooth. Stepping through dark red on to medium blue values, there may be a yellowish zone in the middle that does not seem to lie between the two colors in a perceptual space. The Munsell system makes a first order correction by skewing the manifold, but in terms of perception, there are a lot of spikes and dents in the surface required.
posted by StickyCarpet at 7:17 AM on November 9, 2012 [2 favorites]

Great to see such a mathematically rich exposition in a general-interest magazine like Fast Company. The app sounds great too.

If a monitor has a limited and discrete gamut, and the RGB system allows one to reach every point on that gamut, then I don't see how that could be significantly improved upon.

If my reverse-engineer of what they actually did based on the article is correct, here's the idea. I'm going to express it in a fairly mathy way but I can also unmath it a bit if you want.

There's a space of colors. We can think of it as 3-dimensional Euclidean space via the RGB parametrization, which works well in many contexts. But the shortest path from yellow to blue in RGB space is a straight line, and (apparently) when you show people the corresponding varying family of colors, they say "those are not the intermediate colors between blue and yellow," In other words, the RGB parametrization is NOT reflecting the geometry of whatever perceptual space these colors live in. Essentially, these guys are trying to determine the geometry of perceptual space experimentally by choosing 100 landmark points and getting humans to make judgments about which paths are shortest -- geometrically, you would say you are experimentally estimating the _geodesics_ in perceptual space. And if you have a reasonably dense set of points in hand (I have no idea whether to think 100 is actually enough) and approximate geodesics between them, you have a pretty decent map of the space, good enough, I guess, to generate approximate geodesics between points that you didn't use as landmarks.

It sounds pretty cool! I don't really care whether it makes a nice painting app, I just think it's a great math problem.
posted by escabeche at 7:17 AM on November 9, 2012 [11 favorites]

RGB is just fine for computer programs; it's how the hardware produces color, so it makes sense for the software to work in RGB. RGB is like the assembly language of colors.

RGB is absolutely the wrong color space for people to think in, or mix colors in. It's way easier to work in HSL (Hue, Saturation, Lightness); the numbers have more intuitive visual meaning. And are easily mapped to RGB for the hardware; CSS3 even supports HSL natively. Even better is a fancy perceptual color space like Lab, but that's complex enough that not many people really understand it. HSL has the advantage of being relatively straightforward, based on the color wheel we know as kids. It's also built in to enough color pickers to be commonly available.

My understanding from the Fine Article here is that the team created a new sort of perceptual color space that is particularly good for mixing colors in an iPad app. That's awesome, good work, but it's not some fundamental breakthrough that overturns all color everywhere. I haven't used the app but it sounds like an improvement over HSL or Lab for ordinary humans, at least if you're used to thinking in terms of mixing paints.

While I'm here, a shout out to Color Brewer for its easy to use tool for choosing good sets of colors for visualization. There's a lot of fancy science behind it, but you don't need to understand it to use some excellent groups of colors. Also Color Oracle, nice free software for simulating color blindness so you can see if your designs are useful to the 7%+ of men (and some women) who have limited color perception.
posted by Nelson at 7:21 AM on November 9, 2012 [8 favorites]

Furthermore, paths through color space should not only be smooth, but each step of the same width should move the same amount in the colors. Places where this breaks down are, for instance in a gradient passing through dark yellow.
posted by StickyCarpet at 7:29 AM on November 9, 2012

It's also built in to enough color pickers to be commonly available

It's NOT built into the color picker in Adobe Photoshop or Corel Painter. This is totally unacceptable and a scandal in my opinion. I can't figure out how Adobe, after throwing so much development work at their brush engine solely for the digital painting community, has still overlooked the crappiness of the color system.

Amazingly, only Paint Tool SAI (a light Windows-only program which is a favorite of manga artists) allows you to switch into HSL colorpicking mode.

The closest you can get in Photoshop is to use Lab, but it's still not properly representing the way you go in and out of gamut. It ought to show the gaps in the colorspace as you change parameters, but it doesn't.
posted by overeducated_alligator at 7:31 AM on November 9, 2012

Here's a way to think about adding an extra dimension: Suppose you want to map a 2D chessboard. You take uniform measured steps over the surface, using a ruler, and mark points on the grid. But when you stand back and look at the board it is not an even grid, it's spread out and bunched up in places. So you introduce a new, imaginary, dimension of height that varies across the board. With the height accounted for, the short steps are made longer by raising the height gradient for that step. You find the heights by starting in one place and adding in the required differential fudge factors as needed for each step. In the end you will have determined an integral surface of heights, that don't change where you are in x,y, just how far it is to neighbors with the new invisible z added in.

The height field of required z values could form a round hump, or rolling hills, or spiky outcroppings. Turns out with color distance its spikier and more irregular than you'd expect. You could find the terrain height and store a sparse map, and interpolate points on that to match any x,y. But since its a spiky surface, you'd rather have an equation that solves for the height at any x,y. To describe that surface could require numerous factors in the equation, and these factors become extra dimensions, not of color but of the model needed to find uniform distances.
posted by StickyCarpet at 7:51 AM on November 9, 2012 [5 favorites]

The commenters on the site correctly point out that what they've done is overlay a subtractive color system on top of the additive colors that displays use. For more on subtractive vs additive colors click here but only if you have time to waste.
posted by euphorb at 8:00 AM on November 9, 2012

And if you have even more time to waste you can go Lovecraft-style insane on this topic over at huevaluechroma.com
posted by overeducated_alligator at 8:05 AM on November 9, 2012 [3 favorites]

Color Appearance Models by Mark D. Fairchild is a good place to find out how much you don't know about color display. Have an ice pack handy, because your brain will overheat.

For one thing, all of the fundamental constants like monitor gamma and other color perception metrics are based on tests done with a small to medium sized illuminated monitor in a darkened room. This is because television was presumed to be watched in a darkened room. Now that almost no one views monitors that way, the basic assumptions are wrong. But of course few have cared about color fidelity for a long time. Color is now representational, it depicts in a jpeg thumbnail something about what colors there may have been when an image was taken. Xerox quality is good enough to see that Saddam is dead, and that his shirt was dark blue.
posted by StickyCarpet at 8:26 AM on November 9, 2012

It's NOT built into the color picker in Adobe Photoshop or Corel Painter.

What? HSL is definitely in the Photoshop color picker. I'm pretty sure it's even the default one.

The radio buttons in the color picker let you sort by any of the HSL values, or by RGB values, or by whatever the LAB color axes are (can someone explain to me how LAB works?). You can plug in CMYK values too, but it doesn't seem to have a CMYK viewer.
posted by cirrostratus at 8:44 AM on November 9, 2012

No it isn't -- it's HSB, which is completely different and based on the brightness of phosphors, not on perceptual lightness. Lab works to a point, but doesn't correctly show how the gamut is changing -- for an example of how it's supposed to look (with transparent regions for out-of-gamut), see http://vis4.net/blog/posts/avoid-equidistant-hsv-colors/
posted by overeducated_alligator at 8:56 AM on November 9, 2012

Just for the record, the "Theory of Color" as a scientific discipline probably starts with Goethe's Theory of Colors published in 1810 as a reaction to Newton's Opticks.

( Newton's Opticks is one of those books which is fit rather severely into the procrustean bed of "science." In many respects it represents an "alchemical" approach to light and color[pdf] rather than recognizable modern science... which puts a lot of the hurf durf artist critique of Goethe's theory in a different context, especially in hindsight.)
posted by ennui.bz at 9:12 AM on November 9, 2012

What about L*A*B color? It takes about five minutes to get used to, and then BAM. After reading Dan Margulis's "The Canyon Conundrum", this mildly colorblind guy can do color correction - by the numbers - and get great results.
posted by notsnot at 9:17 AM on November 9, 2012 [1 favorite]

Great to see such a mathematically rich exposition in a general-interest magazine like Fast Company.

This is not a mathematically rich exposition. There are two really simple equations in it, one that is simple arithmetic and doesn't even contain any variables, which are examples of "very simple color blending in RGB space." There is no mathematical exposition at all of the "magical tech" subject of the article and it isn't even really described very thoroughly in general terminology.

It's following an advertising trend I've been noticing in the last few years which I call "we have crossed oceans of design to reach you!1!!" that presents an extremely embroidered narrative of the product's design process while refraining from saying anything specific about the product itself, in the hopes that the consumer will imagine all sorts of wonderful characteristics the product must have by extrapolating the supposed course the design team took.
posted by XMLicious at 9:24 AM on November 9, 2012 [5 favorites]

What about L*A*B color?
The simplicity is obviously advantageous, but for many applications, "black, yellow, or chocolate" is just too limiting.
posted by Wolfdog at 10:18 AM on November 9, 2012 [7 favorites]

This is not a mathematically rich exposition.

Well, it's a sort of gesture toward a mathematically rich exposition. It's an interesting math problem and they talk about that in sort of general terms, enough to let one put together what's going on, though the author doesn't fully connect the dots. I think escabeche's explanation above is clearer about what's going on in the math.
posted by LobsterMitten at 10:58 AM on November 9, 2012

enough to let one put together what's going on

Could someone do us all the favor of describing what's going on, then? How does this magical new color picker/blending gamut generator actually work? The article really does seem like a non-explanation — it describes the problem fairly well, but then effectively just handwaves about the amazing new solution that's allegedly been discovered (and which it actually makes sound a bit kludgy anyhow).
posted by RogerB at 11:14 AM on November 9, 2012

As near as I can tell from the article, they abandoned the idea of using straight-line interpolation between points in any parametrization of color-space, whether 3-variable or 6-variable, or whatever. Instead, they generated a lot of pairs of points, and a lot of different curves between each pair, and used human preference to select the curve that gave the (subjectively) best-looking gradient. And then they hardcoded those preferences, and (in a way which is not made clear) they the curves chosen for those pairs to guess a good curve for any given pair of colors.

If that's what they've done, it's a sensible approach to the problem and it sounds like they're happy with the results. What they discovered - that straight-line interpolation in various parametrizations doesn't work well for this problem - is interesting, and there are probably interesting articles to be written yet about exactly why that's the case. The data they generated in the form of those human-preferred curves could conceivably be basis for a new parametrization, in which straight-line interpolation does yield satisfactory results. Again, there's probably a lot of mathematical fun and work to be done in figuring out to turn that into a model with a good conceptual explanation and a sensible set of parameters. But that hasn't been done yet, as far as I can see.
posted by Wolfdog at 12:04 PM on November 9, 2012

And then they hardcoded those preferences, and (in a way which is not made clear) they [use] the curves chosen for those pairs to guess a good curve for any given pair of colors.

Right, this is my reading of the article as well — in which case the gist of it is:
1. Representing color perception mathematically is hard.
2. There is a whole field of research about this problem, in which some guys who make an iPad app just discovered a single really old published result that they for some reason liked better than newer research.
3. That published result in no way actually informed the solution they came up with, because they didn't like how it worked in practice.
4. So they did a seat-of-the-pants experiment in color perception and then kludged their experimental data (which they won't give us) into some new model of perceptual color space (which they aren't going to tell us anything about but would be glad to sell us for $1.99).

The content of #4 would make a really interesting tech note — if we knew anything about it — but the article as presented really reads like the ad-fluff version, trying to impress us with all the hard work that went into the product without actually telling us anything about it.
posted by RogerB at 12:42 PM on November 9, 2012 [1 favorite]

I like the solution employed by David Briggs, the superhuman behind the aforelinked huevaluechroma.com

He painted his own transitional swatches, then charted the curve which the intermediate colors describe in the colorspace. If you want to work from subtractive color (which is what everyone means when they talk about "color mixing"), just use traditional pigments as your model and forget math.

And here's the thing he explains which really blew my mind:
You know how yellow objects like lemons are extremely "bright"? It's not because they're absorbing all wavelengths except yellow, and only reflecting yellow. If they did, they'd reflect so little light that the resulting color would be a very low value (like a dull yellowish brown). No, in reality, bright yellow paint and bright yellow objects like lemons both reflect yellow light, and also reflect red and green waves which form yellow through additive mixing!

The combination of the additive and subtractive mixing means a lot of light is being reflected, much more than the waves in the yellow part of the spectrum. That's why yellow things sort of "pop" and look bright.

I'm a little too excited about color theory, I have to go sit down.
posted by overeducated_alligator at 1:29 PM on November 9, 2012 [6 favorites]

I like the solution employed by David Briggs, the superhuman behind the aforelinked huevaluechroma.com

fixed link
posted by thegears at 4:14 PM on November 9, 2012

I think escabeche's explanation above is clearer about what's going on in the math.

But let me emphasize that I'm just inferring what I think must be going on based on the description in the article -- I have no first-hand knowledge of what they actually did!
posted by escabeche at 7:45 PM on November 9, 2012

Sure, I meant your explanation seemed to put the pieces together into a clearer picture of what they (might be) doing.
posted by LobsterMitten at 8:08 PM on November 9, 2012

I don't know that much about color theory and I definitely don't know anything about the technology behind creating colors digitally, but I did coincidentally just download the new color mixer for the Paper app today, and it is really awesome to play with. Reading a little bit about how they built it is neat.
posted by padraigin at 8:40 PM on November 9, 2012