How SynthStamp recognises a reposted image

A tag on an image is only useful if it follows the image. The AI-generated sunset tagged in r/pics on Monday will be back on Wednesday in another subreddit, resized, recompressed, maybe flipped. If SynthStamp only matched exact files, every copy would start from zero.

So SynthStamp doesn’t compare files. It compares fingerprints.

A fingerprint, not a file

A cryptographic hash like SHA-256 changes completely if a single pixel changes. That’s what you want for passwords and useless for pictures: every recompression makes a “new” image.

A perceptual hash does the opposite. It summarises what an image looks like, so that images that look alike get fingerprints that are alike. We use PDQ, the perceptual hash Meta published for exactly this job: finding copies of known images at scale.

PDQ shrinks the picture, turns it to greyscale, and looks at its broad patterns of light and dark. The result is 256 bits, one yes-or-no answer about each of 256 patterns. Two copies of the same picture, one sharp and one compressed, answer almost all of those questions the same way.

To compare two fingerprints you count the bits that differ. Zero means identical. A handful means the same picture after resizing or recompression. Around half the bits differing means two unrelated images. SynthStamp treats two fingerprints as the same image when no more than 31 of the 256 bits differ, the threshold PDQ’s authors recommend.

It happens in your browser

The fingerprint is computed by the add-on, on your computer. The image itself is never uploaded, only the 256 bits and the image’s address. That keeps our servers out of the business of storing other people’s pictures, and it means the fingerprint of a picture you look at is all we ever learn about it.

We ported PDQ to JavaScript for this, and before trusting it we checked it bit for bit against Meta’s reference implementation on two dozen test images, including all eight rotations and flips of each. Every fingerprint matched exactly.

Mirrored copies

Flipping an image is one of the oldest tricks for getting a repost past a duplicate check, and a flipped picture has a quite different fingerprint. So the add-on computes two: one of the image, and one of its mirror image. If either matches a known picture, it’s the same image.

The honest limit: crops

Here’s what PDQ doesn’t survive. Crop just five percent off each side of a picture and 86 of its 256 bits change, far past the 31-bit threshold. To PDQ, a cropped copy is a different image.

That matters, because cropping is exactly what a careful reposter does. We’re not going to pretend otherwise on the front page, and it’s the next thing on the list. The likely fix is to fingerprint several centred crops of each image as it comes in, so a cropped copy lands on one of them. A heavier fix is matching local features, the kind of keypoints that survive almost any edit.

Why fingerprints need a memory

A fingerprint can only match something the server has already seen. That’s why the add-on sends a fingerprint of every image on the pages you open, not only the ones you tag: the original has to be on record before the repost turns up. It’s the one part of SynthStamp that works better the more people use it, and it’s why we’re clear about it on what we collect.

If you want to see it working, install the add-on, tag an image, and wait for it to come round again. It will.