Deepfake Detection False Positives and False Negatives: Causes, Accuracy and MCC

What Are False Positives and False Negatives in Deepfake Detection?

A false positive is an authentic image, video or recording that a detector labels as manipulated or AI-generated. A false negative is a deepfake the detector labels as authentic. Every detector produces both, and the balance between them is set by the threshold the operator chooses.

The costs are different, which is why the balance matters. In customer onboarding, a false positive blocks or delays a genuine applicant and may push them to a competitor; a false negative opens an account for a fraudster. In a newsroom, a false positive can lead an editor to dismiss real footage of an event; a false negative can put a fabrication on air. Teams should price both errors for each workflow before choosing a threshold.

What Causes False Positives in Deepfake Detection?

False positives happen when something genuine shares features with the fakes a detector learned. The most common causes are listed below, roughly in the order fraud and trust and safety teams encounter them:

False negatives have their own causes: generators released after the detector was trained, fakes laundered through compression or screen recording, deliberate adversarial changes, and attacks that bypass the detector altogether, such as injecting a stream through a virtual camera into a check that only tests liveness.

• Heavy compression and repeated re-sharing, which add blocking and ringing artefacts that can resemble manipulation.
• Phone processing, beautification modes and filters, which smooth skin and reshape faces in ways close to what face-editing models do.
• Low light, motion blur, very low resolution and very short clips, which leave the detector little real signal.
• Screenshots and screen recordings, which replace camera noise with display rendering.
• Faces, skin tones, ages or accents under-represented in training data, which can raise error rates for some groups.
• Legitimate AI edits such as background replacement or upscaling, which are real manipulations even when the person is genuine.

How Accurate Is Deepfake Detection?

Deepfake detection can be very accurate on media close to its training conditions and much less accurate elsewhere. On the FaceForensics++ benchmark, published in 2019, XceptionNet reached 99.26% accuracy on raw face video, 95.73% at light compression and 81.00% at heavy compression. On Deepfake-Eval-2024, built from media that circulated in 2024, open-source detectors lost about 45% to 50% of their AUC compared with earlier benchmarks.

Commercial systems vary widely. On VendorBench-100, a public benchmark of 100 adversarial still images tested on 36 systems, MCC ranged from 0.876 for Neural Defend and 0.646 for Reality Defender down to 0.135 for OpenAI's GPT. Neural Defend's precision was 0.963, recall 0.987 and specificity 0.857. One VendorBench author also co-wrote Neural Defend's ARCAS 1B paper; the corpus and results are public, and the benchmark does not cover video, audio or injection.

The rates that matter for you are the ones at your threshold, on your media, at your base rate of fakes. The worked example below shows why base rate changes everything.

Why Is Accuracy a Misleading Metric for Deepfake Detection, and What Is MCC?

Accuracy is misleading because it rewards a detector for agreeing with the majority class. If 90% of a test set is fake, a detector that calls everything fake scores 90% accuracy while being useless on genuine files. If 99% of production traffic is genuine, a detector that calls everything genuine scores 99% while catching nothing. F1 has a related blind spot: it ignores true negatives, so it says nothing about how many genuine files were wrongly flagged.

The Matthews correlation coefficient, or MCC, uses all four cells of the confusion matrix (true and false positives, true and false negatives) and ranges from -1 to 1, where 0 is no better than chance. It is high only when a detector does well on both fakes and genuine files. Davide Chicco and Giuseppe Jurman argued in BMC Genomics in January 2020 that MCC is more reliable than accuracy and F1 for imbalanced binary classification. The VendorBench-100 authors make the same point for deepfake images, warning that accuracy and F1 can look good for a model that flags nearly everything as fake, and rank systems by MCC.

Report MCC together with specificity, recall and ROC-AUC. ROC-AUC tells you how well a detector ranks fakes above genuine files across all thresholds; MCC tells you how good its decisions are at the threshold you actually run.

How Can Teams Reduce False Positives and False Negatives?

Teams reduce both errors by controlling the input, setting thresholds deliberately and keeping humans in the loop for the uncertain middle. Capture media with a vendor SDK or upload path that avoids extra compression, and ask for a recapture when quality is below a set minimum. Choose thresholds per workflow from the cost of each error, with an uncertain band that goes to trained reviewers rather than an automatic decision.

Measure regularly. Run the detector in shadow mode on your own traffic before launch, sample both flagged and unflagged cases for review afterwards, and break error rates down by device type, capture channel and customer group to find where false positives concentrate. NIST's AI 100-4 report recommends that evaluations reflect the real distribution of content, including benign distortions and unseen generators, which is the standard your own measurements should meet.

Neural Defend's NeuroVerify returns signal-level evidence from six signals alongside its score, so reviewers can see whether a flag came from face manipulation, generative-model traces, pixel-level edits or presence, which makes false positives faster to resolve. NeuroDetect records the model version behind each analysis and states uncertainty in its forensic report.