The New York Times has a fascinating piece on some new software that automatically tweaks pictures of human faces to make them more attractive by reducing the concept of facial beauty to simple vector-based algorithms.
The image on the right is a ‘before and after’ picture of the software at work, and the researchers have a page for the project with many more examples and the full-text of the academic paper.
The researchers asked participants to rate the attractiveness of a series of faces and they then used software to calculate distances and directions between key facial landmarks.
By combining the attractiveness ratings and the landmark vectors they created a statistical model of which general facial attributes are most attractive. Their software allows new faces to be subtly altered to more closely approximate the general model of attractiveness.
I’m fascinated by the fact that software advances are increasingly taking advantage of the quirks of our mind and brain.
The MP3 format is perhaps the most well known, which allows audio files to be compressed because it takes advantage of a psychological effect called auditory masking where, when two sounds of certain frequencies are present, we can only perceive one.
The MP3 encoding algorithm simply scans sound files for times when auditory masking would eliminate the perception of one sound, and then actually eliminates the data from the file, thereby making it smaller.
Another wonderful idea is chroma subsampling used in jpg and digital video compression. It’s based on the finding that our visual system is less accurate at pinpointing colour differences than brightness differences.
Chroma subsampling takes advantage of this by storing colour information at a lower resolution than brightness information. For example, rather than storing separate colour information for every pixel, it will store it for every four. When we see the image, we often can’t tell the difference.
This is particularly true for moving images, and you’ll notice sometimes when you stop YouTube videos the colours seem to be fuzzy and bleed from where they’re supposed to be (have a look at this YouTube still I used on a recent post ) even though you hardly notice this when the video is playing.
These software advances wouldn’t have happened without the psychology research to find the bugs / features in human perception and it’s curious to think that these new developments build on both the digital and neural platforms.
What will be most interesting is if software starts to take advantage of cognitive features found only in certain members of the population (for example, some women have four types of colour receptor in the retina, rather than the usual three).
In other words, we might find that some important software advance will only work on some people (or rather, will be developed with only some people in mind), and so these people might be preferentially hired to work with certain applications.
If these applications become particularly high value (usually due to their use in the military or intelligence services), people might starting attempting to engineer themselves or others to have the uncommon attribute.
Sci-fi writers, start your engines.
Link to NYT piece ‘The Sum of Your Facial Parts’.
Link to researcher’s page with photos and full-text.
Mind Hacks 
It seems inevitable that software like this will end up on digital cameras, or maybe even social networking websites. Every image of a person could be tweaked. There’s even the possibility of digital mirrors that could do it in real time! You’re certainly right about the sci-fi writers being interested in this.
“In other words, we might find that some important software advance will only work on some people (or rather, will be developed with only some people…”
Quite some time ago I read a research paper that examined a correlation between programming language style proficiency and country of origin of the programmer. They concluded that programmers from cultures with an emphasis on form, such as the French as they argued, tended to be more proficient at object oriented style programming language abstractions when compared to other cultures. Unfortunately, I can’t recall the title of the paper.
As an aside, I find the ‘before’ photo to be the more attractive one 😛
Interesting. I’m fairly familiar with these techniques. I put together averages of celebrity faces a while back, the results of which are here:
I’m not the only on to have done work like this, though I haven’t seen anyone else do it with celebrity faces, rather with student or volunteer’s faces, and one famous one with hot-or-not faces
I lost the original files, unfortunately, but I’m sure I could, with a bit of time do something similar to what they did – it’s just a matter of taking the vectors of the facial “beautiful” average and applying them to an individual’s face. Sort of like the texture mapping used in 3D applications like video games. You take the texture of one person’s skin & apply it to a beauty wireframe that’s somewhere between that person & the “beautiful” average.
I think your analysis is a bit off, this isn’t about our perceptions on the level of auditory masking or other “illusions.” These aren’t tricks or illusions, they’re simply modifying (a-la “photoshopping”) images to bring them closer to a standard of beauty. It’s not tricking the brain, it’s actually showing a different thing to the brain.