Real-World Deepfake Detector Evaluation Datasets: Building a Representative Test Set

Why a real-world evaluation dataset beats a public benchmark

Detectors learn the quirks of the data they are trained on: particular generators, compression levels, lighting, resolutions and face crops. When the test set shares those quirks, scores look excellent. When it does not, they fall. Deepfake-Eval-2024, first posted to arXiv in March 2025, collected deepfakes that circulated online in 2024 from 88 websites in 52 languages. Its first version held 44 hours of video, 56.5 hours of audio and 1,975 images. Open-source detectors that performed well on older academic benchmarks lost about half their AUC on it.

Your own channel has its own quirks. A mobile onboarding app compresses selfies, a contact centre narrows audio to telephone bandwidth, and an insurance portal receives photos of photos. A test set that skips these conditions measures a detector you will never run.

What are the standard deepfake detection benchmarks and datasets?

The widely cited public sets cover different eras of generation and different modalities. They are good for screening a shortlist and for reproducing a vendor's published claim. None of them stands in for your channel, and older face-swap sets say little about diffusion-generated images.

The ARCAS 1B paper from Neural Defend reports results on several newer image sets, including GenImage, WildFake, AIGC-Detect, AIGIBench, Community Forensics and OpenFake. VendorBench-100 takes a different approach: 100 deliberately hard still images, scored by MCC, so that differences between strong systems show up.

How to build a representative test set from your own channel

Start from the population you need to protect. Write down the devices, capture paths, file formats, languages and user groups that reach the decision, and sample authentic media across them in proportion to traffic, with extra samples for small groups you need to measure separately. ISO/IEC 5259-2:2024 defines data quality measures, such as completeness and representativeness, that can structure this step.

For the synthetic side, collect the attacks your channel has seen, then add the techniques you expect next: face swaps, fully generated faces, AI edits to genuine photos, lip-sync video and cloned voices where relevant. Generate some items yourself with current open tools so you know exactly how they were made. Then pass everything, real and fake, through your production pipeline so compression and resizing match.

Decide the class balance on purpose. Production traffic may be 99 percent authentic, but a test set with that ratio holds too few fakes to measure recall. A common approach is to over-sample synthetic items for the test and re-weight error rates to the expected production mix when estimating review volume and fraud loss. Write the re-weighting into the protocol before results come in, so nobody adjusts it to flatter a favourite vendor.

Prevent leakage. Split by source person, original clip and generator, not by file, so that near-duplicates cannot sit on both sides of a tuning and test split. Keep the final held-out set in your custody and release it to vendors only at test time.

How to label ground truth you can defend

A label states where an item came from and what was done to it. It should rest on evidence: the capture record for authentic items, and the tool, model, settings and operator for items you generated. Items collected from fraud cases need the case reference and the basis for calling them synthetic. A vendor's output is never ground truth, because using it to label the test set grades the detector against itself.

Have two people label independently where the evidence is indirect, record disagreements, and settle them with a named adjudicator. Keep a separate status for items whose origin cannot be established. Leave those out of headline metrics, and report how many there were.

Store the original file, every derivative you created from it, and the label record together under one identifier. When a result looks wrong later, that record lets you check whether the detector failed or the label did.

Where C2PA and metadata help, and where they do not

C2PA Content Credentials can record who created an asset and which edits were applied, signed so that tampering with the manifest is detectable. A valid manifest is useful supporting evidence for a label. It does not prove the depicted scene is real, and most media in the wild carries no manifest at all, because platforms often strip metadata on upload.

NIST AI 100-4 treats provenance, watermarking and content-based detection as separate techniques with separate limits. In a test set, record provenance status as its own field, and do not mark an item authentic only because its metadata looks clean.

Testing Neural Defend on your dataset

We ask buyers to test NeuroVerify on a sealed set built this way, because that is the result that predicts production. NeuroVerify returns six signals for each face image, video or audio file, with a unified risk score and signal-level evidence, so you can score each stratum and attack type separately.

For context, the ARCAS paper reports pooled accuracy for ARCAS 1B, the model behind our products, of 99.99 percent on GenImage, 99.82 percent on AIGC-Detect and 97.74 percent on Community Forensics. On the public VendorBench-100 benchmark, Neural Defend ranked first of 36 systems with an MCC of 0.876. One VendorBench author also co-authored the ARCAS paper, and both corpora are public, so you can check these figures as well as your own.