Deepfake Detection Security Testing: Presentation, Injection and Evasion Attacks

Which attacks should deepfake detection security testing cover?

A deepfake control has several layers, and attackers pick the weakest. Generating a convincing face is one capability; getting it into your system is another. An attacker might hold a screen up to a phone camera, feed a synthetic stream through a virtual camera driver, replay a recording of a real customer, or alter a file just enough to slip past the model. Each route hits a different control, so each needs its own test.

NIST AI 100-2e2025, the March 2025 edition of NIST's adversarial machine learning taxonomy, gives shared terms for evasion, poisoning and privacy attacks on AI models. Use it to describe attacks on the detector itself, and use biometric and identity standards for the capture and delivery layers.

Writing the threat model before the first attack

Start with what an attacker wants and what they can reach. Name the asset (a new account, a payout, a password reset), the attacker's likely knowledge of your system, the interfaces they can touch, and the time and money they would spend. A fraud ring opening mule accounts at scale behaves differently from a targeted attacker impersonating one executive, and the test should reflect whichever you face.

Keep media generation separate from delivery in the model. An attacker who can make a convincing face swap but can only upload through your app exercises different controls from one who controls a rooted phone or an emulator. Map each attack to the trust boundary it crosses and the decision point it targets, and note which attacks are out of scope for this round so they are not mistaken for tested ones later.

Agree success criteria per attack class before testing starts, for example the share of attempts that reach approval, so results cannot be reinterpreted afterwards.

Why presentation attack results do not prove injection resistance

ISO/IEC 30107-3:2023 sets out how to test and report presentation attack detection at a biometric capture device, and it explicitly excludes other attack types and overall system security. A lab certificate against that standard is real evidence, for the presentation layer only.

Injection skips the camera. FIDO Biometrics Requirements v4.1 separates presentation attacks from image or video data injected to bypass capture, and ENISA's remote identity proofing good practices address both. FinCEN's alert FIN-2024-Alert004 lists a customer using a third-party webcam plugin during live verification as a red flag, along with repeated claimed technical glitches and requests to switch channels mid-check. Test capture integrity, device signals and channel binding directly, and test whether the content detector still flags a synthetic face when it arrives through an injected stream.

Testing evasion against the detection model

Evasion tests ask whether an attacker who knows a detector exists can change a fake enough to pass. Realistic attempts include re-compression, recapturing from a screen, compositing a generated face onto a real background, editing only part of an image, and adding small perturbations aimed at the model. Use a sealed attack set for the final comparison so no vendor has seen it.

Measure three outcomes for each attempt: what the detector scored, what the workflow decided, and how the system recovered. Keep a record of attempts that produced errors or inconclusive results, since a crash or timeout can be a bypass in itself. Do not publish working bypass recipes; report attack classes and outcomes.

How to run attack tests safely and legally

Get written authorization that names the systems, accounts, dates and attack classes in scope. Use isolated environments, test identities and non-production credentials, set rate limits and stop conditions, and agree a disclosure process with every vendor before the first attempt. Coordinate with legal and privacy owners when tests involve biometric data or third-party infrastructure.

Report the system version, configuration, tester knowledge and access, number of attempts, success criteria and results for each class. Retest after fixes, and check that a fix has not raised false positives or latency for genuine users.

• Never test people, accounts or systems outside the written scope.
• Separate confirmed vulnerabilities from theoretical concerns in the report.
• Track each finding to closure, with any compensating control recorded.

Reading attack test results without overclaiming

A good result applies to the attacks tried, at the tester's skill level, against the tested version. Say so in the approval record, and list the attack classes that were not tested. Attackers adapt, and a new generator or injection tool can appear within weeks, so set retest triggers for model updates, SDK changes and new attack reports in your sector.

Treat detection output as one signal in a layered control. Capture integrity, device intelligence, behavioural checks and human review each stop attacks that a content detector cannot see, and the strongest designs combine them.

What to test when evaluating Neural Defend

NeuroVerify includes Live Presence Verification alongside face manipulation detection, AI generative model analysis and AI pixel-level manipulation, returned together in one API call with signal-level evidence. That lets a tester see which signal reacted to each attack, which is useful when judging whether a defence is working for the right reason.

Our public benchmark evidence has a clear boundary. Neural Defend ranked first of 36 systems on VendorBench-100, with an MCC of 0.876, but that benchmark covers still-image detection only and does not measure injection attacks, and one of its authors also co-authored our ARCAS 1B paper. Test injection, replay and workflow bypass against NeuroVerify yourself. We support authorized testing during evaluation, and security findings can be reported through our responsible disclosure process.