Open source toolchain can generate ‘digital twin’ avatar

FaceChain, a program that generates “digital twin” lookalikes of a given user, has been introduced with potential use cases in healthcare, retail, and AI research. The software is a deep-learning toolchain that uses face biometrics and generative AI to create personalized portraits while preserving unique identity characteristics of individuals, and is explained by self-described IoT, AI, and ML enthusiast Devendra Bogati in a Medium post.

Developed by a team at Alibaba Group, the toolchain is a personalized portrait generation framework of pluggable components. The toolchain injects face models into the generation process of portraits, which improves label-tagging, data-processing, and post-processing, distinguishing FaceChain from DreamBooth and InstantBooth.

The software is trained by taking user-uploaded images and converts them into high-quality forward-facing training images through a series of face models. Then it uses face attribute and text annotation models in tandem with tag post-processing methods to generate labels for training images. The images and label data then fine tune the Stable Diffusion model to get the face LoRA model.

To generate images, the software fuses the weights of the face LoRA model obtained from training online and the style LoRA model trained offline into the Stable Diffusion model. It then uses text-to-image generation based on tag inputs and uses a series of face refinement models. Finally, facial recognition determines the similarity between the generated images and the original template before delivering the image with the highest similarity as the output.

The toolchain could be used to create lifelike avatars in gaming, digital avatars for social media and marketing campaigns, medical training, virtual try-ons for retail, and AI research.

FaceChain is open-source under the Apache-2.0 license and is available on GitHub. The repository has an installation guide that covers a variety of methods.

Article Topics

Alibaba Group  |  face biometrics  |  FaceChain  |  generative AI  |  open source  |  twin