Deepfake wave breaking the digital dam; orgs are busy building defenses

Deepfakes are at the door. There are two options: wait until they’re inside to craft a response – or take active steps to keep them out with effective deepfake detection. Transitioning from “reactive fact-checking to a proactive trust architecture” is part of what organizations must do to defend against deepfake identity impersonation at scale through disinformation-as-a-service (DaaS), according to a new Gartner report.

The firm says forty percent of government organizations will establish dedicated TrustOps functions by 2028 to combat deepfakes, as generative AI technology gets better and easier to access, lowering the barrier of entry for deepfake fraud.

Deepfake threats “manifest as public-facing disinformation campaigns, such as impersonating leaders to issue misleading public statements, and in attacks on internal systems. The latter aims to compromise automated biometric authentication (voice or face) or use social engineering to manipulate employees into harmful actions, typically by rapidly establishing authority and urgency.”

Deepfakes can destabilize democracy: Gartner

Attacks can go all the way up the political chain, attacking the credibility of the state. In a release, Daniel Nieto, senior director analyst at Gartner, observes that “if citizens cannot distinguish a legitimate prime minister’s announcement or a secure tax agency portal from a replica, the foundational architecture of truth collapses.”

This is no mere IT problem. “The deepfake phenomenon threatens to induce digital regression; reversing the ROI of digital transformation by forcing a retreat to high-friction, paper-and-in-person interactions,” Nieto says. One “cannot outrun a deepfake once it is viral. Organizations must saturate the information space with the truth first.”

That means verifiable proof. Education will only go so far; ultimately, “the burden of verification must shift away from the end user to the institutional architecture through cryptographic provenance.”

Gartner recommends that organizations establish a trust council in consultation with primary stakeholders to manage digital identity issues and disinformation activity. They should “identify and then audit high-risk administrative workflows, such as financial disbursements,” and “implement security measures that require multiple approvers and application-level authentication to eliminate single-point-of-failure vulnerabilities exploitable by voice-cloned executives.”

Finally, businesses need standard operating procedures (SOPs) for deepfake defense tools. Nieto says “long-term, government organizations can consider implementing solutions such as the C2PA protocol.”

Haotian AI points to future of real-time deepfakes

Exactly what are these sinister deepfake tools that are so easy to access and so threatening to reality? One of them is Haotian AI, which 404 Media describes as “a sought-after piece of realtime video deepfake software that lets you turn into anyone else during Microsoft Teams, WhatsApp, or Zoom calls.” The outlet investigates Haotian in a new podcast, “The Chinese Deepfake Software Powering Scams.”

“Whereas video deep fakes used to be about superimposing somebody’s face onto a static video,” says host Matthew Gault. “This is something else, and that is promising to do a video deepfake in real time.” The example he shows has a man in a hotel room, morphing seamlessly in real time through various faces – none of them real.

The system still requires robust hardware like a gaming PC, and it comes at a cost – reportedly around $2000 for a yearly subscription. Moreover, Gault emphasizes that, in his own testing, effective results took significant configuration work: “it’s not like magic out of the box.” But the results, he says (and shows) are highly realistic – “a generational leap when it comes to real-time def.” And given the rate at which deepfake tools are evolving, it won’t be long before the same result can be achieved with a mobile app.

But right now, Gault says, Haotian AI was able to convincingly put his face on someone else’s body in real time. “It was very unsettling,” he says, “like there is something visceral about that. My face was on him and he pinched his cheek and he stroked his chin and he covered his eye and for the most part during those tests it didn’t break the illusion.”

New deepfake detection tools from iProov, GetReal

In response to the elevating deepfake threat, both iProov and GetReal Security have launched products to strengthen defenses against identity fraud.

A release says iProov Verified Meetings “enables organizations to authenticate the identity of participants in video calls without disrupting the user experience.” Video is vulnerable, as evidenced by the notorious deepfaked Zoom call that cost a Hong Kong firm $25 million. Injection attack techniques and tactics make any media stream or virtual camera a target.

“Video has become the standard way of communicating for business and consumers alike, from meeting with colleagues and suppliers to hiring, onboarding, and approving financial transactions,” says Andrew Bud, CEO of iProov. “But organizations still largely assume that seeing a person on screen means they’re real. That assumption no longer holds. Organizations must be confident about who they’re really engaging with. iProov Verified Meetings provides that capability, enabling organizations to authenticate that participants are real people using real cameras while protecting the integrity of critical interactions.”

Delivered as a native plugin for video conferencing platforms, iProov Verified Meetings is part of the iProov Workforce Solutions Suite.

GetReal Security has announced the general availability of continuous identity verification within its flagship product, GetReal Protect. A release says the platform is “now available to any organization that relies on mission-critical digital workflows and interactions to conduct business.”

The launch aims to meet Gartner’s call for proactive defenses in the face of the deepfake firehose. Data from GetReal’s recent Deepfake Readiness Benchmark Report shows that eight out of ten organizations encounter AI deepfakes or impersonation attempts at least occasionally, and 45 percent encounter them frequently. That kind of penetration demands an active response, incorporating deepfake and impersonation detection, continuous identity verification and global threat intelligence.

“Our digital lives are made up of sound waves and pixels, and until now we’ve been technologically limited in how we can protect a person’s likeness,” says Matt Moynahan, CEO of GetReal Security. “It’s time we reverse engineered and reimagine the concept of a background check and move to a pre-ground check. Identity should come first. That’s the kind of digital safety that businesses, governments, and individuals deserve in this new AI environment.”

GetReal recently published a blog on reverse engineering Haotian AI.

Article Topics

AI fraud  |  biometrics  |  deepfake detection  |  deepfakes  |  Gartner  |  GetReal Security  |  iProov