DEEP LEARNING FOR 3D FACE RECONSTRUCTION FROM A SINGLE IMAGE

Abstract

Recovering 3D face models from 2D in-the-wild images has gained considerable attention in computer vision due to its wide range of potential applications, such as game development, virtual reality, and augmented reality. The core challenge lies in inferring 3D geometric and texture information from inherently incomplete 2D inputs, which lack explicit depth information. While learning-based methods for 3D face reconstruction have made notable progress, they often rely on large-scale labeled 3D facial datasets, which are expensive and impractical to collect. As a result, reconstructing 3D faces from single RGB images has emerged as a cost-effective alternative, though it remains a highly under-constrained and ill-posed problem. This thesis aims to enhance geometric and texture reconstruction accuracy for 3D face reconstruction from single images. The first method, Multi-level Attention Network (MLANet), employs a pre-trained hierarchical backbone network and introduces multi-level attention mechanisms at different stages to effectively extract and integrate multi-level features, focusing on disentangling complex facial attributes such as geometry, texture, pose, and illumination. A semi-supervised training strategy incorporating 3D Morphable Model (3DMM) priors and differentiable rendering enables the model to achieve robust predictions despite the lack of ground-truth 3D data. Experiments conduct on the AFLW2000-3D and MICC Florence datasets demonstrate marked improvements in face alignment and 3D reconstruction accuracy, on both quantitative metrics and qualitative evaluations. The second method, Multi-Scale Feature Fusion for Multi-Attribute (MSMA), centers on integrating multi-scale features to optimize task-specific learning for multi-attribute 3D face reconstruction. Through the use of a Multi-Scale Feature Fusion (MSF) module, the framework aligns and fuses hierarchical features at varying resolutions, ensuring the effective decomposition of attributes such as geometry, texture, pose, and illumination. This design ensures that each attribute is predicted using the most relevant resolution of features. Additionally, the integration of a Large Kernel Attention (LKA) mechanism enhances feature precision across scales, enabling the recovery of fine-grained facial details. Extensive evaluations on MICC Florence, FaceWarehouse, and custom-collected datasets validated the method’s robustness across diverse facial attributes and environmental conditions, achieving results that are either on par with or surpass the performance of current state-of-the-art methods under challenging conditions.

Date of Award	21 Mar 2025
Original language	English
Awarding Institution	The University of Manchester
Supervisor	Hujun Yin (Main Supervisor) & Wuliang Yin (Co Supervisor)