"You look exactly like that guy from that show." "I really don't." This argument happens in every group chat, and both sides are usually sincere. Resemblance is not a fact about two faces; it's a judgment made by a particular brain with a particular history. Here is what face perception research says about why people disagree, and how an AI's notion of resemblance fits in.
Faces are processed as wholes
The brain treats faces differently from other objects. A region on the underside of the temporal lobe, the fusiform face area, responds strongly to faces, and the way we recognize them is holistic: we take in the configuration of features together rather than checking eyes, nose, and mouth one by one. Two classic demonstrations show this. Turn a photo upside down and it becomes much harder to recognize (the inversion effect), far more so than for a house or a car. And in the Thatcher illusion, a face with its eyes and mouth flipped upside down looks grotesque when the face is upright but nearly normal when the whole face is inverted, because inversion switches off holistic processing.
The practical consequence: when someone says two people look alike, they are usually reacting to the overall configuration, the gestalt, not to a checklist. That's why a resemblance can be obvious to one person and invisible to another who is focusing on a single feature that differs.
Face space
Psychologists model recognition with a "face space": imagine every face as a point in a many-dimensional space, positioned relative to an average face at the center. Typical faces cluster near the middle; distinctive faces sit far out. Two faces "look alike" when their points are close. Caricatures work by pushing a face further from the average along the direction it already leans, which makes it more recognizable, not less.
If that sounds like the embedding space in how AI face matching works, it should. A neural network's vector space is a mechanical version of the same idea: each face becomes a point, and similarity is distance. The difference is in which dimensions get built. The brain's dimensions are tuned to the faces it grew up with; the network's dimensions were learned from millions of photos of everything.
Familiarity sharpens discrimination
The dimensions of your personal face space depend on the faces you've seen most. This is the source of the other-race effect: people distinguish faces from their own ethnic group more easily than faces from groups they have less experience with, because their face space has finer resolution where they've had more practice. The same thing happens with fandom. Someone who has watched a series for years perceives its cast as distinct individuals; a newcomer sees a set of similar faces. So "you look like this character" carries different weight depending on who says it. The fan is comparing you to a sharply defined point; the non-fan is comparing you to a blur.
The mirror problem
You have a special relationship with your own face: you almost always see it mirrored. Photos show the un-mirrored version, which is what everyone else sees. Because faces are slightly asymmetric, the two versions are subtly different, and mere-exposure research shows that people prefer the version they see most often, the mirror image, while friends prefer the photo. This is one reason people flinch at photos of themselves and why a resemblance others notice in a photo can feel wrong from the inside. You are comparing the character to a face you rarely see.
Where humans and AI disagree
Humans weight the eyes and the mouth heavily and are strongly biased by hair and context: a similar haircut alone can trigger "they look alike". A general image network is also influenced by hair, because hair is a big high-contrast shape, but it gives more weight to overall contour, the spacing of features, and skin texture. That produces some matches that feel spot-on and some that feel like the AI looked at a different part of the face. Both are true. The AI is reporting a real similarity in the dimensions it can see; you are reporting one in the dimensions you can see. They overlap but aren't the same.
What to do with this
When the AI's top match surprises you, look at the still it chose: the model picked that frame because it was closest to your photo, and the resemblance is often clearer in that specific image than in your mental picture of the character. And when a friend insists you look like someone, remember they may simply have a sharper point for that face than you do.