New 3D face scanning captures the details but misses with the whole face

A team of British researchers say they have written what they say is the first algorithm capable of reconstructing photorealistic three-dimensional faces from a single low-resolution, in-the-wild image.

AvatarMe was created by scientists from Imperial College London and FaceSoft.io, a 2018 spinout from Imperial College. FaceSoft, which creates databases for three-dimensional face scanning, received $717,000 in February, its second round of seed funding.

According to the team’s open-access paper (PDF), which had not been peer reviewed as of its publication, a “large data set of facial shape and reflectance” was used with a three-dimensional texture and shape reconstruction method.

The paper also claims their results are the first example of 4k-by-6k-resolution busts that avoid the uncanny valley.

In training its software, the team used 168 polarized LEDs and nine digital SLR cameras in a sphere on 200 people using seven expressions. The dataset, called RealFaceDB, skewed to light-skinned subjects, limiting the algorithm’s ability to optimally render darker skin.

Indeed, among the examples of finished avatars published in the report is an attempt at Muhammed Ali that bears little resemblance to the famed boxer.

Ali’s image is joined in the paper with those of several other well-known personalities as well as some people who are not tabloid fodder.

AvatarMe conjured digital busts from color candids and black-and-white professional head shots of actors Orlando Bloom, Aaron Eckhart, Audrey Hepburn and others. There also is a reconstruction from what looks to be a Frida Kahlo painted self-portrait.

Facial recognition algorithms were used to judge the results. Where the images excel is in how realistically light is absorbed by and reflected off the avatars’ skin.

But the Ali rendering is not the only one that suffers in translation. One non-celebrity subject gains a third nostril. A second goes from having a relaxed smile to displaying annoyed chagrin.

Eckhart’s digital face is acceptable, if a bit dead-eyed, yet the full bust leaves him looking more like actor Matthew Fox. It is a comparison that both men likely would be happy with unless one of them had licensed his face to a game firm only to have his money maker rebranded.

In a side note possibly only of interest to would-be investors, London-based FaceSoft has not posted on its social media accounts in more than a year, and it seems its web site is no longer in search engine databases.

Article Topics

3D  |  biometrics  |  computer vision  |  facial recognition  |  machine learning  |  research and development