Microsoft Research, Peking University develop AI deepfake detector

Microsoft Research in partnership with Peking University has published two academic papers discussing a concept for face-swapping artificial intelligence and face forgery detection technology: FaceShifter and Face X-Ray “a framework for high-fidelity and occlusion-aware face swapping and a representation for detecting forged face images, respectively,” writes Venturebeat. Researchers claim that, compared to other approaches developed, the two apps don’t need as much data and performance is still optimal.

FaceShifter addresses the substitution of a person a target image with a different person in a source image, retaining head pose, facial expression, lighting, color, intensity and background. The researchers say that while AI applications such as Reflect and FaceSwap claim to be accurate in the process, they can be influenced by changes in posture and angle. On the other hand, to ensure face swap accuracy, FaceShifter leverages a generative adversarial network (GAN) dubbed Adaptive Embedding Integration Network (AEI-Net) that can collect features in multiple spatial resolutions. The generator is equipped with Attentional Denormalization (AAD) layers that trained how to integrate facial features, and with the Heuristic Error Acknowledging Refinement Network (HEAR-Net) that detects roadblocks by analyzing inconsistencies between reconstructed images and their inputs.

Following a qualitative test, researchers noticed FaceShifter accurately maintained face shapes, lighting and image resolution, while for images collected online it did not require human-annotated data to recover anomaly regions.

“The proposed framework shows superior performance in generating realistic face images given any face pairs without subject specific training. Extensive experiments demonstrate that the proposed framework significantly outperforms previous face swapping methods,” the team said.

Their second suggestion, Face X-Ray, detects fakes, a tool that researchers believe is critical today as deepfakes are increasing online. It doesn’t require human supervision or previous algorithm training based on fake images and manipulation methods. Face X-Ray creates grayscale images and detect if can image can be broken down into two different images that were merged. This is successful, researchers claim, because images have unique differentiation marks either from hardware or software components.

The researchers conducted a number of experiments where they trained Face X-Ray on FaceForensics++ which has more than 1,000 deepfake videos, on data from the Deepfake Detection Challenge and Celeb-DF. They concluded that the tool successfully identified unknown deepfakes and blending regions. Researchers point out that while their method is based on finding blending steps, it is not a silver bullet and it may not work for wholly synthetic images.

Article Topics

artificial intelligence  |  biometrics  |  deepfakes  |  face swap  |  facial recognition  |  research and development  |  spoof detection