New, easier way to make deepfakes emerges from Russia

Scientists in Russia are claiming advances in deepfake creation that could make it easier to produce startlingly believable avatars.

Their techniques appear to make it possible to create a high-resolution deepfake video of a portrait using almost anyone as a movement model, or driver, and a single image of the image to be faked.

The researchers are from the Samsung AI Center in Moscow; the Skolkovo Institute of Science and Technology, also in Moscow; and Yandex, a Russian Internet-services company.

They describe their innovation as “the one-shot creation of high-resolution human avatars, called megapixel portraits, or MegaPortraits.” It is a set of new neural architectures and training that put out high-resolution video.

It is eye-catching, though by no means undetectable.

In short videos, drivers bearing little obvious resemblance to the source image to be enlivened look up, down, left and right; mouth words; smile and tilt their heads.

The result is, for example, an animated Mona Lisa and U.S. actor Brad Pitt that are generated in real time. Movement of the deepfake is as fluid as that of the driver. Particularly notable are lifelike sparkles in the eyes and facial features like high cheekbones (in the Angelina Jolie source) that are subtle or non-existent in the source image.

In a video accompanying their research paper the team congratulates itself on the realism they create around eyes, delivering a high approximation of genuine emotion – and they are right to do so.

Clothing and shoulders, they admit, need work. The shoulders do look those seen on puppets and clothing flutters and waves unrealistically.

Also needing work are backgrounds, which is odd. Algorithms are creating angles and features not in the sources, so when an avatar’s head shifts, it should be simple enough to generate realistic features. Instead, a smudgy gray shadow forms and freezes in place.

Proportionality suffers, too. Pitt’s and Jolie’s faces are bulgy and rubbery around the chins and foreheads. And in a few deepfake portraits, white-white teeth do not move side to side with head movements of the drivers.

Each sample deepfake is a few seconds long and soundless, limitations that doubtless will not remain for long.

Article Topics

AI  |  biometrics  |  deepfakes  |  research and development  |  Russia  |  Samsung