Robust Facial Landmark Localization Under Simultaneous Real-World Degradations

2018-12-11T22:44:52Z (GMT) by Keshav Thirumalai Seshadri
The automatic localization of facial landmarks, also referred to as facial landmarking or facial<br>alignment, is a key pre-processing step that is of vital importance to the carrying out of tasks such<br>as facial recognition, the generation of 3D facial models, expression analysis, gender and ethnicity<br>classification, age estimation, segmentation of facial features, accurate head pose estimation, and<br>a variety of other facial analytic tasks. Progress in all these areas of research has heightened<br>the need for developing accurate facial alignment algorithms that can generalize well to handle<br>simultaneous variations in pose, illumination, expression, and high levels of facial occlusion in<br>real-world images.<br>This thesis proposes a facial alignment algorithm that is not only tolerant to the joint presence<br>of facial occlusions, pose variation, and varying expressions, but also provides feedback (misalignment/<br>occlusion labels for the detected landmarks) that could be of use to subsequent stages in a<br>facial analysis pipeline. Our approach proceeds from sparse to dense landmarking steps using a set<br>of pose and expression specific models trained to best account for the variations in facial shape and<br>texture manifested in real-world images. We also propose the use of a novel shape regularization<br>approach that sets up this task as an `1-regularized least squares problem. This avoids the generation<br>of implausible facial shapes and results in higher landmark localization accuracies than those<br>obtained using prior shape models. Our approach is thoroughly evaluated on many challenging<br>real-world datasets and demonstrates higher landmark localization accuracies and more graceful<br>degradation than several state-of-the-art methods. We proceed to put the task of facial alignment<br>into better context by examining its role in two applications that require alignment results as input:<br>(1) a large-scale facial recognition scenario and (2) a project aimed at improving driver safety<br>by assessing facial cues. Finally, we also carry out a rigorous set of experiments to analyze the<br>performance of our approach when dealing with low-resolution images and provide some insights<br>gained from this study.