How to Evaluate Synthetic Personas and Fake Profiles Made With AI Faces

What counts as a synthetic persona?

A synthetic persona is an online identity that does not correspond to the person it claims to be. The face may be fully generated by a model such as StyleGAN or a diffusion system, or it may be a real person's stolen photo. The biography, work history and posting style are invented or copied, and the account is run by someone else, sometimes by a team or a script.

The term overlaps with two others that are worth keeping apart. Synthetic identity fraud is about attributes, such as a mix of real and fake personal data used to open a bank or credit account. Romance and investment scams are a use of personas to take money. A synthetic persona can exist without either: it can sell, recruit, spy, harass or push a political message.

Most personas are built in layers. A profile photo comes first, then a few months of ordinary posts, connections with real people who accept requests without checking, and a link to a website or company page that looks credible. CISA's Tactics of Disinformation guide lists fake personas and websites among the methods used to build inauthentic networks.

Why can't people spot AI-generated profile photos?

People cannot reliably spot AI-generated faces because current generators produce faces that are statistically average in the ways people read as trustworthy. In a study published in PNAS in 2022, Sophie Nightingale and Hany Farid asked 315 participants to classify faces made with StyleGAN2 and real faces. Average accuracy was 48.2%, close to chance. A second group given training and feedback reached 59.0%, and a third group rated the synthetic faces 7.7% more trustworthy than real ones.

The old tells are unreliable too. Earlier advice pointed to mismatched earrings, warped backgrounds and eyes centred in the same position in every image. Newer models fix many of those, and an attacker can simply discard bad outputs. A clear face with a blurred background describes millions of real photos as well.

How to evaluate a suspect profile: the signals that matter

A useful evaluation scores the account on several independent signals and asks what the persona is trying to achieve. A single odd photo proves little. A new account with a generated face, a biography copied from another profile, a burst of connection requests and a quick push to a private channel is a strong pattern.

The NPR report from March 2022 shows how this looks in practice. Researcher Renée DiResta received a LinkedIn message from a sales profile whose photo seemed slightly off. With a colleague at the Stanford Internet Observatory she went on to find more than 1,000 LinkedIn profiles that appeared to use AI-generated faces to make outreach messages for companies look personal.

How do platforms detect fake profiles made with AI faces?

Platforms detect fake profiles made with AI faces by combining image analysis at upload with account and network signals over time. The image check runs when a profile photo or verification selfie is submitted and flags fully generated faces, face swaps and edited regions. The account signals, such as device reuse, sign-up velocity and links between accounts, catch networks whose photos are stolen rather than generated.

Neural Defend's NeuroVerify fits the upload step. One API call checks a face image, video or audio and returns a unified risk score with signal-level evidence, drawn from six signals that include Face Manipulation Detection and AI Generative Model Analysis. Neural Defend states a latency of under 200 ms at p95 for eligible image requests under defined test conditions, which lets a platform score photos in the sign-up flow. The platform's own policy decides whether to block, challenge or send the account to review.

On the public VendorBench-100 benchmark of 100 difficult still images, Neural Defend ranked first of 36 systems with an MCC of 0.876 against 0.646 for the next-best vendor. One VendorBench author also co-wrote Neural Defend's ARCAS 1B paper; the corpus and results are public so anyone can rerun them.

Investigating a persona network before you act

Before blocking or reporting, capture the evidence: profile URLs, account IDs, messages, claimed affiliations, linked domains, payment details and related accounts. Platform data exports are better than screenshots where available. Preserve the earliest contact and every move to another channel, because the sequence often shows how the network operates.

For trust and safety, threat intelligence and fraud teams, NeuroDetect keeps the images, videos, voice notes and documents from one network in a single case. Analysts see heatmaps and attention views for each file, and can export a forensic report with observations, uncertainty and model context to support a takedown, a law enforcement referral or an internal decision.

Look for the shared pieces that tie accounts together: the same website registrar, the same payment wallet, identical phrasing in first messages, or photos generated with the same artifacts. Those links let a team remove a whole network at once instead of chasing single accounts that are quickly replaced.

Where image analysis stops and verification starts

Image analysis answers one question: was this face generated or manipulated. It does not say who runs the account, and it cannot catch a persona built on real photos taken from someone else's profile. For hiring, partnerships, journalism or investment decisions, verify the claims directly. Contact the named employer through its official directory, check professional registers, and keep any transaction inside protected systems.

When a real person's identity is being misused, tell them through a verified channel and report the account under the most accurate category, such as impersonation, fraud or coordinated inauthentic behaviour. Avoid publicly naming a suspected operator without evidence, which can redirect harassment to the wrong person.