The Truest Portrait Is a Caricature

Small-data computer vision is the practice of training a model on a handful of well-chosen images rather than a warehouse of them. Like a caricature, it works by exaggerating the few features that distinguish your target, letting an open-vocabulary baseline and a little human review do the rest.

Somewhere in a drawer you have a passport photograph that is technically perfect and useless. Correct exposure, every pore present, an honest record of the light that fell on your face one grey morning at the post office. Nobody who loves you would choose it to remember you by. Hand those same people a cartoonist's ten-second scribble, two strokes for the eyebrows you raise when you are unconvinced, and they will say your name before the pen leaves the paper.

The accurate picture is the worse likeness. This turns out not to be a quirk of taste. It is how recognition works, and it is the most useful thing I know for building machines that see.

Loaded portraits

The people who first noticed were not engineers. Late in the sixteenth century, in a Bologna workshop, the painters Annibale and Agostino Carracci took to sketching one another between commissions. Not portraits. Ritrattini carichi — little loaded portraits. The verb is caricare: to load, to weigh down. You found the one feature most stubbornly a person's own, and you loaded it until it nearly buckled. From those idle afternoons we got the word caricature.

A caricaturist is not a worse photographer. A caricaturist is doing the opposite job. The camera's task is to keep everything. The cartoonist's task is to throw almost everything away and leave you holding the person.

Why does a caricature beat the photograph?

In 1987 three cognitive scientists, Gillian Rhodes, Susan Brennan and Susan Carey, sat the cartoonist's hunch down in a lab. They built caricatures by computer, exaggerating only the ways each face departed from the average face, then asked people to name them. The caricatures won. A deliberate exaggeration was recognised faster and more reliably than an accurate drawing of the same person. They called the result a superportrait: a likeness truer than the truth.

Why should a lie about a face beat the face? Because of how the face was filed in the first place. The mind does not keep a photograph. It keeps a departure. In the model the psychologist Valentine proposed in 1991, every face you know is held as the way it strays from the average of all the faces you have ever seen: the nose a shade longer than the mean nose, the eyes set wider than most. A caricature exaggerates exactly that departure. It does not fight your memory of a face. It speaks your memory's own language, louder.

A caricature is not a distortion of the memory. It is the memory, drawn out loud.

"Below the Mark" — Mark Twain by 'Spy' (Leslie Ward), Vanity Fair, 1908. The white suit, the mane, the moustache: Spy spent his exaggeration exactly where the man lived. (Yale Center for British Art, CC0.)

Why is a model a caricature, not a photograph?

This is where I keep coming back to our own corner of the problem. A model that has learned to see is far closer to a caricature than to a camera. It is not a faithful record of everything it was shown. It is a compressed opinion about what mattered in all of it.

The industry's reflex, when a model is not good enough, is to take a better photograph. More images. Higher resolution. Another million examples scraped from somewhere and poured in. More data is the higher-megapixel sensor, and it feels like progress because it is so easy to count. (Nothing flatters a roadmap like a number that only goes up.)

But scale is not representation. It is a way of deferring the decision about what matters. A billion photographs of a street corner will never tell a model that the thing you care about is the bus and not the weather. Somebody still has to load the distinctive feature and leave the rest on the floor. The question every vision system answers, on purpose or by accident, is never how much have you seen? It is what did you decide to exaggerate?

A model is not a photograph. It is a caricature. And a caricature is mostly the things its maker had the nerve to leave out.

One decisive line

This is the whole shape of the loop we build at VisionEngine. Point it at a folder of images and it will not ask you to annotate every object in every frame. You draw a single box on the first picture, one decisive line, and name the thing you came for. An open-vocabulary baseline labels the rest of the set with no training at all.

You do not check everything it proposes. You check the handful it is least sure of, accept or reject or fix them, and you trust the rest, because the model's own uncertainty is the only honest map of where your attention is worth spending. Every one of those judgements is written to a versioned labelset history: a standing record of what you chose to load and what you chose to ignore.

None of this is about volume. Every image the model keeps has earned its place; the model is good not because it has seen a great deal, but because someone decided what was worth seeing. That is a caricature, not a photograph.

Where does small-data computer vision earn its keep?

It is tempting to file all of this as a parlour trick that evaporates at scale: charming in a sandbox, useless in the real world. I think it runs the other way. The places where computer vision has to earn its keep are exactly the places where you cannot take a billion photographs. A hairline crack in a turbine blade. A tumour the size of a comma. The defect that appears once in ten thousand parts and must never be the one you miss.

There is no warehouse of neatly labelled examples for the rare and the dangerous, and there never will be. When data is scarce or expensive, which is to say whenever the problem is worth solving, the winning team is not the one with the most images. It is the one that knows what to leave out.

The Carracci brothers were not trying to found a science of recognition between commissions. They had stumbled on something the rest of us keep forgetting: that to capture a thing, you have to be willing to misrepresent almost all of it. The camera records everything and understands nothing. The cartoonist throws nearly everything away and hands you back the person.

Every model is a caricature. The only question that has ever mattered is whether it had the nerve to exaggerate the right things.

Draw one box on your own images and watch the rest fall into place →