Why does lighting expose synthetic faces?
Lighting exposes synthetic faces because light is a single physical system that touches everything in a scene at once. The direction, colour, softness and number of light sources determine shading on the cheeks, the shadow under the nose, highlights on the forehead, reflections in both eyes and the way the face relates to the hair, neck and background. A real camera records all of that consistently by default. A generator or a face swap has to reproduce it, and most models learn how lit faces look statistically rather than how light behaves.
That gap shows up in two ways. A face inserted into existing footage can be lit for a different scene than the one around it. A fully generated face can be internally inconsistent: soft frontal light on the skin but a hard side highlight in one eye, or a shadow that falls the wrong way for the visible light source.
Why does lighting still expose synthetic faces after years of progress?
Because the hardest lighting effects are the ones a model sees least and must reason about most. Hard cast shadows from the nose, glasses or a hat brim depend on 3D geometry. Several light sources of different colours create overlapping shading that is rare in training data. Specular highlights on skin and in the eyes depend on surface properties and on where the camera is. Hair and the boundary between face and background mix light from the subject and the scene.
Eyes are a well-documented example. In September 2020, Shu Hu, Yuezun Li and Siwei Lyu showed that GAN-generated faces often have inconsistent corneal specular highlights, the small reflections of light sources in each eye. In a real photo, both corneas reflect the same environment, so the highlights match in shape and position. The authors attribute the mismatch to generators lacking physical and physiological constraints, and report that the artifact is widespread in high-quality GAN faces.
How relighting research is closing the gap
Relighting models learn to change the light on an existing portrait, which helps both legitimate editing and better fakes. Face2Face in 2016 already estimated scene illumination so its re-rendered face would blend with the target video. Deep Single-Image Portrait Relighting, presented at ICCV 2019, relit a portrait from one image under a new target lighting. Google's Total Relighting, presented at SIGGRAPH 2021, trained on portraits captured in a light stage, used a per-pixel lighting representation that models diffuse and specular components separately, and combined alpha matting, relighting and compositing so a person could be placed into a new background under that background's light. Holo-Relighting, at CVPR 2024, offered controllable volumetric relighting from a single image.
Each step reduces the obvious mismatches. What remains hardest are hard shadows, consistent eye reflections, and agreement between relit face, hair edges and the new environment.
| Year | Work | Contribution |
|---|---|---|
| 2016 | Face2Face (CVPR) | Estimated scene illumination to re-render a reenacted face |
| 2019 | Deep Single-Image Portrait Relighting (ICCV) | Relit a portrait from one image under new lighting |
| 2020 | Corneal highlight analysis (arXiv) | Showed mismatched eye reflections in GAN faces |
| 2021 | Total Relighting (SIGGRAPH) | Light stage training, separate diffuse and specular lighting, background replacement |
| 2024 | Holo-Relighting (CVPR) | Controllable volumetric relighting from a single image |
How lighting gives away face swaps and composites
A face swap transplants a face learned from one set of footage onto a head in another. If the source footage was lit from the left and the target scene from the right, the swapped face carries the wrong shading unless the model corrects it. Colour transfer and blending can hide the difference in skin tone, but the direction of shadows and highlights is harder to fake. In video, lighting changes as a person moves through a room; a swapped face that does not brighten and darken with the rest of the scene stands out over a few seconds.
Examiners can make this concrete. Compare the brightest side of the face with the side of the neck, ears and nearby objects; check whether the nose shadow agrees with shadows elsewhere in the frame; and follow the face across a lighting change, such as a person walking past a window. Each check is quick, and each targets a relationship a face-only model is not forced to get right.
How to read lighting evidence without overreaching
Genuine images break the rules too. Phone cameras merge multiple exposures, beauty filters smooth shading, ring lights create unusual eye reflections, and mixed indoor lighting produces colour casts that look wrong but are real. Compression removes the fine highlights that lighting analysis depends on. A lighting inconsistency is a reason to look harder at a file, not a conclusion, and a lighting-consistent face is not proof of authenticity, because relighting models now handle many common cases.
For verification flows, some liveness checks use active illumination, changing the screen's colour or brightness during capture and checking that the face responds as real skin would. That turns lighting from passive evidence into a challenge an attacker has to answer in real time, though injected video can still try to simulate it.
How Neural Defend uses lighting and other evidence
Neural Defend does not rely on one hand-crafted cue such as eye reflections. NeuroVerify combines six signals in one API call, including Face Manipulation Detection, AI Generative Model Analysis, AI Pixel-Level Manipulation and Live Presence Verification, and returns one risk score with signal-level evidence in under 200 ms at p95 on eligible image requests under defined test conditions. A selfie whose lighting is perfect but whose pixels carry generator traces is still flagged for your policy to decide.
For investigations, NeuroDetect shows explainability heatmaps over the regions behind a finding, which often include eyes, hairlines and face boundaries where lighting breaks, and records observations and uncertainty in a downloadable forensic report. The ARCAS 1B model behind both reports pooled accuracy between 97.65% and 99.99% across six image benchmarks in the ARCAS paper.
Evidence boundary
Limitations
- Lighting cues such as eye reflections are being reduced by newer generators and relighting models.
- Genuine images processed by phone pipelines or filters can show lighting inconsistencies that resemble manipulation.
- Relighting milestones are summarised from project descriptions and are not compared quantitatively here.
Source record
Sources and references
- arXivExposing GAN-generated Faces Using Inconsistent Corneal Specular HighlightsPublished September 24, 2020. Verified October 19, 2026.
- CVF Open AccessDeep Single-Image Portrait Relighting (ICCV 2019)Verified August 7, 2026.
- ACM Transactions on GraphicsTotal Relighting: Learning to Relight Portraits for Background ReplacementPublished July 19, 2021. Verified October 19, 2026.
- CVF Open AccessHolo-Relighting: Controllable Volumetric Portrait Relighting from a Single Image (CVPR 2024)Verified August 7, 2026.
- arXivFace2Face: Real-time Face Capture and Reenactment of RGB Videos (CVPR 2016)Published July 29, 2020. Verified October 19, 2026.
