Minimalist’s dream or nightmare? Researchers apply ‘deepfake’ to sketching

Chinese and Hong Kongese researchers have developed software they call DeepFaceDrawing, and the name is a good description of what it does. Sketch a face freehand as sloppily as one wants, and the algorithm uses it to create a realistic synthetic face.

DeepFaceDrawing, a deep learning framework, analyzes black-and-white rasterized sketches –even incomplete scratchings — and produces color images based on what it has learned about human faces in training, according to a new research paper that has not been peer reviewed. The researchers work at the Chinese Academy of Sciences and the City University of Hong Kong.

Acceptable final images can be had with remarkably minimalist sketches, although more-detailed drawing produced images of greater detail, including more realistic skin tone.

“Our key idea is to implicitly learn a space of plausible face sketches from real face sketch images and find the closest point in this space to approximate an input sketch,” they wrote in their research paper.

They suggest that the most obvious role for their algorithm is as a tool for law enforcement sketch artists. Earlier attempts at producing realistic images from sketches exist, the authors write, but the older code requires either edge-mapped images or line drawings of the same quality.

DeepFaceDrawing takes its cues from input — front view only at this point — to generate features encountered in its training data set. Smiles, scowls and winks are inferred as is hair position, length and style.

Coloring was solely the decision of the algorithm in testing, something the authors said likely would need to be changed.

The algorithm chooses facial features independently of each other, which is a benefit when considering that natural faces are asymmetrical and can move separately, including eye brows and eyelids. But, at least at this point, the authors realized this ability to choose can result features that typically do have symmetry — such as irises — not always matching.

Article Topics

algorithms  |  deepfakes  |  facial recognition  |  research and development  |  training