caricatures and facial recognition

[gusl]

I would like to see an AI program that makes caricatures of human faces.

The idea is that the way we represent (perceive, remember) faces is by storing a "diff" from the baseline. We probably have different baselines for different categories of people: gender, age, race, etc.

The caricature program would select image features that people perceive (probably eyes, nose, lips, chin, etc.), amplify the deviation from the baseline in terms of location/size/shape, and reconstruct the image. This gets interesting when features perceived interact with each other, e.g. the distance between the eyes interacts with the size of the eyes, the size of the lips wrt to the nose interacts with the size of the nose. The point is that, while we would like to amplify the difference in all features by the same amount, this is impossible: some features need to be sacrificed for the sake of others, so we need to give priority to the more salient ones... just like when you project a globe into 2D, you must choose some properties you want to preserve, while losing others.

I think this means that the caricature function is information-lossy, i.e. irreversible. An algorithm that makes you more "average" again would not return your original face.

This face-morphing website lets you can change your gender / race / age.

---

A caricature of my body would probably include narrow shoulders, short legs. But how does my face deviate from the baseline? I suspect I am rather brachycephalic, and have a large forehead, but I couldn't say more. What is your caricature of me? Submit your entry today.

---

I want to get serious about biometrics. I would like to scan every millimeter of my body, every month or so. Who knows what the benefits will be? When medicine finally knows use all this information, my medical history will have a lot more data.

Comments

[madcaptenor.livejournal.com]

I don't think the caricature function is lossy. In particular, my instinct is that it's injective -- different people have different caricatures -- so in theory it's invertible, i. e. given a caricature there is a unique real face that it came from.

But you might be using "lossy" in more of a computer scientist's sense -- it may be true that there is a unique real face corresponding to a given caricature, but is there an algorithm to find it?

[gustavolacerda.livejournal.com]

I like this notion of "lossy", which you ascribe to computer scientists. In cryptography, there is a notion of "one-way function". Is this what you thought I meant?

The set of images of a given size (call it I) is finite. So it's easy to write a terminating algorithm to check whether any function f: I->I is injective. Likewise, given any f: I->I, it's easy to write a terminating algorithm to find original image (or set of possible original images). I am neglecting efficiency here.

I'm trying to think of an example of non-injectivity, i.e. two images that get mapped to the same image.

[brianfey.livejournal.com]

That seems like a cool do-able idea.
Perhaps basing it on existing code would make it super easy. There is a lot of face recognition stuff out there.. .

And you could go the other way as well... to take a photo of a person and let them adjust their attributes towards the baseline. (Likely increasing "attractiveness" since the baseline is often perceived as optimal.

[gustavolacerda.livejournal.com]

Yup! Baseline is pretty good. But you can do better. Attractive women are more feminine than average, and vice-versa for me.

[darius.livejournal.com]

There was an A.K. Dewdney column in Scientific American back in the 80s about automatic caricaturing like that. IIRC it needed a human to mark out reference points on the faces, though.