16 major AI image models found vulnerable to synthetic ID generation

A review of 16 major AI image-generation models found that most could generate realistic synthetic government identity documents, raising new concerns for KYC and digital identity systems.

The audit achieved a 92 percent success rate across 75 test attempts, with models including Google Gemini, ChatGPT, Grok and Imagen 4 Ultra producing fake passports, driver’s licenses and national identity cards that appeared visually authentic to human reviewers.

The study was conducted by AI detection platform AI or Not and spanned 14 vendors. The image generation models were tested both through consumer interfaces and developer APIs. The findings also highlighted an important gap between consumer-facing safeguards and developer APIs, which are more easily integrated into automated fraud operations.

Researchers say the findings demonstrate how quickly synthetic document fraud is becoming accessible through mainstream AI tools, lowering the barrier to creating convincing counterfeit identity documents with simple prompts.

No model in the study categorically refused all synthetic-ID requests across both interfaces. The documents were generated for 17 countries.

Five models, including Google Gemini, ChatGPT, Recraft v4, Grok and Imagen 4 Ultra, produced high-fidelity fake adult IDs whose layouts, typography, and apparent security features closely matched authentic documents.

Three of the models went further: Google Gemini, Grok, and Imagen 4 Ultra generated realistic identity documents depicting minors through standard consumer interfaces, with no technical workaround required. ChatGPT and Recraft v4 declined equivalent minor-ID requests via their consumer apps on stated safety grounds, but fulfilled the same requests when accessed through their developer APIs.

“We did not expect these findings,” says Anatoly Kvitnitsky, the company’s CEO. “We started this work assuming the major AI-image generators had built real safeguards against the most obvious abuse cases, like fraud, identity theft, and content depicting minors. What we found is that those protections are either missing or sitting in the wrong place.”

All 16 models were also found vulnerable to authority-framing: every tested model generated synthetic IDs when prompts were reframed as Know Your Customer (KYC) reviews, compliance evaluations, or security audits, including models that had initially declined direct requests.

The findings are likely to intensify pressure on identity verification providers to strengthen document authentication, liveness detection and injection attack defenses against synthetic identity fraud.

Article Topics

AI fraud  |  digital ID  |  generative AI  |  identity document  |  synthetic identity fraud