weee eigenvectors

October 6, 2012 11:25 AM

Eigenfaces for facial recognition. (This post assumes familiarity with the terminology and notation of linear algebra, particularly inner product spaces.)
posted by Evernix (18 comments total)

Several thoughts:

1. This post assumes familiarity with the terminology and notation of linear algebra, particularly inner product spaces.

Way to sell your FPP.

2. For whatever reason (perhaps some faces are in slightly more agreeable positions), the lighter faces are more “unique” in this sample of faces.

I have heard this kind of thing somewhere before.

3. While Euclidean distance is fine and dandy, there are a whole host of other methods for classifying points in Euclidean space.

I wish more scholarly papers read like this.

5. "Facespace" sounds like a very derivative social networking startup.

6. This was a much more enjoyable read than I expected.

7. Even if you can't follow the math, the general outline and the graphics are pretty interesting.

8. I did not know about this blog; damn it, I have work to do!
posted by GenjiandProust at 11:45 AM on October 6, 2012

Way to sell your FPP.

No problem, I'll be sure to be less descriptive in the future so that my links are more ambiguous. Sound good?
posted by Evernix at 11:53 AM on October 6, 2012 [8 favorites]

Apparently, from my reading so far of Mr. Penumbra's 24-Hour Bookstore, I'm thinking eigenvectors are also involved in OCR. I wish I knew more about them!
posted by limeonaire at 11:56 AM on October 6, 2012

Can I use 76 arbitrary coefficients to generate an artificial face? Because that sounds great.
posted by squinty at 11:59 AM on October 6, 2012 [1 favorite]

I see your Eigenfaces, and raise you an Eigenworm.
posted by James Scott-Brown at 12:09 PM on October 6, 2012 [2 favorites]

Can I use 76 arbitrary coefficients to generate an artificial face? Because that sounds great.

No, if you pick random coefficients and reconstruct an image, you'll get something face-like but probably not a well-formed face. To give a concrete example, there are probably a set of eigenfaces that represent different eye separations, and arbitrary coefficients could produce faces with multiple sets of eyes. In a more formal sense, if all sets of coefficients produced proper faces, that would imply that any linear combination of faces is also a face*, which is obviously false**, and therefore there must be coefficients that do not produce faces.

* If face A has coefficients x_a and face B has coefficients x_b, then since everything is linear, the combination face alpha*A + beta*B will have coefficients x_c = alpha*x_a + beta*x_b.

** A simple average is a linear combination, and it is pretty evident that just averaging two faces together rarely produces a well-formed new face
posted by Pyry at 12:26 PM on October 6, 2012 [1 favorite]

For more face-related goodness: image compression with SVD (PDF) often uses faces as examples. The PDF includes a confidence-inspiring disclaimer: "We assume that the reader has a basic knowledge of eigenvalues, eigenvectors, and matrix operations."
posted by Nomyte at 1:01 PM on October 6, 2012

Can I use 76 arbitrary coefficients to generate an artificial face?

Well, kinda.
posted by RobotVoodooPower at 1:28 PM on October 6, 2012 [1 favorite]

Hmm, I always assumed facial recognition software worked by detecting common facial features (eyes, nose, mouth) and then using measurements on those things to form the basic representation vector.

Using the entire matrix of pixels as a vector seems kind of crazy. I mean, different exposures, lighting, tilt, angles and even hair styles would produce images, and thus pictures of the same person would have completely different vectors, that were nowhere near eachother in vector space.

In fact, later on in the paper he says:

The reader is invited to look at the results, and see what sorts of changes in appearance mess up recognition the most. As a sneak preview, it seems that head-tilting, eye-closing, and smiling account for a lot of variability. If the reader is a fugitive on the run (and somehow find’s time to read this blog; I’m so honored!), and wishes to have his photograph not recognized, he should smile, tilt his head, and blink vigorously while the photograph is being taken

Which would only be true if all facial recognition algorithms worked the same way his did, which I don't think they do. This is an interesting post about how eigenvalues work but I don't think it's a very good way to go about actually trying to recognize photos of the same person at all. He said with a sample of 30 faces, he got about 90% right (two false positives and 1 false negative), but probably a lot of those photos were taken in the same studio with the same lighting, etc.

Way to sell your FPP.

Some people actually like linear algebra.
posted by delmoi at 1:30 PM on October 6, 2012 [4 favorites]

Well, kinda.

That's actually a very different process. It's using a genetic algorithm to generate a random image, then feeding that into a facial detection algorithm. If you were to just generate a face from random eigenvectors, the process would be much, much faster.
posted by delmoi at 1:32 PM on October 6, 2012

For more face-related goodness: image compression with SVD (PDF) often uses faces as examples.
posted by Nomyte

Wow, Lenna deserves to be awarded an an honorary Coon's prize. That 512 x 512 image scanned from a 1973 Playboy, has only two runners up: Pam, and Tiffany.

Thousands of algorithms have been developed using that single image. It's pretty crap, too. Look at the color histogram sometime.

I wouldn't want to be her going through a TSA checkpoint.
posted by StickyCarpet at 2:16 PM on October 6, 2012

Someone has been reading Hackaday.
posted by clvrmnky at 4:30 PM on October 6, 2012

See also, http://www.faceresearch.org/
posted by palbo at 7:22 PM on October 6, 2012

If the reader is a fugitive on the run (and somehow find’s time to read this blog; I’m so honored!), and wishes to have his photograph not recognized, he should smile, tilt his head, and blink vigorously while the photograph is being taken

Which would only be true if all facial recognition algorithms worked the same way his did, which I don't think they do.

Try smiling in your next passport photo and see if they accept it.
posted by srboisvert at 7:44 PM on October 6, 2012

Try smiling in your next passport photo and see if they accept it.

What does that have to do with anything? Those requirements have been in place long before any recognition algorithms existed.
posted by delmoi at 9:52 PM on October 6, 2012

What does that have to do with anything? Those requirements have been in place long before any recognition algorithms existed.

Visual inspection was the earliest recognition algorithm, and it's still in use.
posted by Nomyte at 10:43 PM on October 6, 2012

Really delmoi? I don't remember the more severe restrictions being in place until sometime in the 90's-00's, by which time machine face recognition was widespread, if not actually all that useful. The "eigenfaces" paper that TFA is talking about was published in 1991, for example, and it's simply a new approach to a well-studied existing problem.
posted by hattifattener at 10:49 PM on October 6, 2012

The point I was making was that this method wouldn't work well with those variations (smiling/blinking/tilting/turning and lighting) in photographs, because it's treating each pixel as an independent dimension in a vector space (then using the eigenvectors to define a new basis). And it would work a lot better to break the face up into facial components and search on the basis of their shape and relationship.

Whether or not you can smile/blink/tilt your head in passport photos is irrelevant to whether or not that's true. I assumed that the restrictions were in place in the past, but whether or not they were it's still kind of irrelevant. It could be because it makes it more difficult (though not impossible) for humans to distinguish the photos, it could be they're using an out of date algorithm, and it could be that they've updated their algorithms, but haven't changed their regulations.
posted by delmoi at 9:54 PM on October 7, 2012