What Are Audio Deepfakes? Types, Risks and Detection

What are audio deepfakes?

An audio deepfake is speech or sound generated or altered by machine learning so that it appears to come from a real person or a real event. Most cases involve a voice: a cloned executive approving a payment, a relative asking for bail money, a politician saying something they never said. The term also covers synthetic background sound and edits that change a few words inside a genuine recording.

Audio is the easiest deepfake to make and the hardest to check. It needs less source material than video, it travels over phone lines that strip away detail, and the listener usually has no second channel, such as a face, to compare it against. The FBI's December 2024 public service announcement on generative AI fraud lists cloned audio of family members, public figures and account holders among the main techniques criminals now use.

The four main types of audio deepfake

Audio deepfakes differ in how they are produced, and the difference matters because each type leaves different evidence and calls for different checks.

How criminals and propagandists use audio deepfakes

Fraud is the largest use. The FTC warned in March 2023 that scammers use voice cloning to make family emergency calls sound like a real relative, and asked people not to trust the voice but to call back on a known number. Starling Bank's survey of UK adults, published on 18 September 2024, found 28% said they had been targeted by an AI voice cloning scam in the past year.

Impersonation of officials is the second pattern. The FBI reported in May 2025 that since April 2025 attackers had sent AI-generated voice messages claiming to come from senior US officials to build rapport before taking over accounts, and warned that the clones can sound nearly identical to the real person.

Political robocalls are the third. In January 2024 AI-generated calls imitating President Biden's voice urged New Hampshire voters to skip the primary. The FCC's declaratory ruling, released on 8 February 2024, confirmed that the Telephone Consumer Protection Act's restrictions on artificial or prerecorded voices cover AI technologies that generate human voices, so such calls need the called party's prior express consent.

How audio deepfakes are detected

Human listening is a weak test. People can sometimes notice flat emotion, odd pauses or mispronounced names, but modern clones avoid most of these and phone compression hides the rest. The FBI suggests listening for unusual tone or word choice while warning that tone alone is not reliable.

Automated detection looks at evidence below the level of hearing: phase behaviour, fine spectral structure, the fingerprints of vocoders and audio codecs, and the statistics of breathing and silence. Research systems are benchmarked through the ASVspoof challenges. The 2024 edition, ASVspoof 5, used crowdsourced recordings from many more speakers than earlier rounds and included adversarial attacks built to fool detectors for the first time.

Detection works best on longer, cleaner audio and on generator families represented in training data. It weakens on narrowband calls, heavy background noise, very short clips and brand-new generators. For partial fakes, a system must score short segments rather than give one verdict per file, otherwise a few synthetic words inside a long real recording can pass unnoticed.

How to protect people and processes from audio deepfakes

Process controls do the most work. Do not accept a voice as proof of identity for payments, password resets or changes to account details; require a callback to a number already on file or a second factor. Families can agree a code word, as the FBI recommends. Organisations should give staff permission to pause any urgent voice request until it is verified.

Detection adds a signal where volume makes manual checks impossible, such as contact centres and fraud teams reviewing voice notes. Its score should feed a policy that decides whether to step up verification, route to a specialist or end the call.

If a deepfake call or voice note has already caused harm, keep the original file or voicemail, note the number and time, and report it. In the US, the FTC takes reports at ReportFraud.ftc.gov and the FBI at ic3.gov. Banks should be told immediately if money was sent, since recovery chances fall quickly. Preserved originals also matter if the audio later becomes evidence, because re-recorded or re-encoded copies lose detail that forensic analysis depends on.

How Neural Defend detects audio deepfakes

Neural Defend covers audio in three products, depending on where the audio arrives. NeuroCall, currently in Beta, analyses call audio for synthetic speech and returns a risk signal for call-handling rules. NeuroVerify accepts audio along with face images and video in one API request at onboarding or recovery, and returns one unified risk score with signal-level evidence. NeuroDetect is the forensic workspace for authorised recordings in a case, showing evidence down to audio segments and producing a downloadable report with uncertainty, provenance and model context recorded.

All three run on ARCAS 1B, Neural Defend's own deepfake detection model. Deployment options run from a managed cloud API to private cloud, on-premises and air-gapped installs for teams that cannot send recordings outside their network.