Reference Publications

Facial Pose Estimation and Normalization

Breitenstein, M. D., Kuettel, D., Weise, T., Van Gool, L., & Pfister, H. 2008. Real-time face pose estimation from single range images. In Computer Vision and Pattern Recognition, 2008. CVPR 2008. IEEE Conference on (pp. 1-8). IEEE.

  They present a real-time algorithm to estimate the 3D pose of a previously unseen face from a single range image. Based on a novel shape signature to identify noses in range images, we generate candidates for their positions, and then generate and evaluate many pose hypotheses in parallel using modern graphics processing units (GPUs).

Fanelli, G., Gall, J., & Van Gool, L. 2011. Real time head pose estimation with random regression forests. In Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on (pp. 617-624). IEEE.

  Fast and reliable algorithms for estimating the head pose are essential for many applications and higher-level face analysis tasks. We address the problem of head pose estimation from depth data, which can be captured using the ever more affordable 3D sensing technologies available today. To achieve robustness, we formulate pose estimation as a regression problem.

Ekenel, H. K., & Stiefelhagen, R. 2009. Why is facial occlusion a challenging problem?. In Advances in Biometrics (pp. 299-308). Springer Berlin Heidelberg.

  This paper investigates the main reason for the obtained low performance when the face recognition algorithms are tested on partially occluded face images.

Chai, X., Shan, S., Chen, X., & Gao, W. 2007. Locally linear regression for pose-invariant face recognition. Image Processing, IEEE Transactions on,16(7), 1716-1725.

  The variation of facial appearance due to the viewpoint (/pose) degrades face recognition systems considerably, which is one of the bottlenecks in face recognition. One of the possible solutions is generating virtual frontal view from any given nonfrontal view to obtain a virtual gallery/probe face. Following this idea, this paper proposes a simple, but efficient, novel locally linear regression (LLR) method, which generates the virtual frontal view from a given nonfrontal face image.

Asthana, A., Jones, M. J., Marks, T. K., Tieu, K. H., & Goecke, R. 2011. Pose Normalization via Learned 2D Warping for Fully Automatic Face Recognition. In BMVC (pp. 1-11).

  They present a novel approach to pose-invariant face recognition that handles continuous pose variations, is not database-specific, and achieves high accuracy without any manual intervention. Their method uses multi dimensional Gaussian process regression to learn a nonlinear mapping function from the 2D shapes of faces at any non-frontal pose to the corresponding 2D frontal face shapes.

Zhang, Xiaozheng, and Yongsheng Gao. "Face recognition across pose: A review." Pattern Recognition 42.11 (2009): 2876-2896

  This paper provides a critical survey of researches on image-based face recognition across pose. The existing techniques are comprehensively reviewed and discussed. They are classified into different categories according to their methodologies in handling pose variations.

Asthana, Akshay, et al. "Pose Normalization via Learned 2D Warping for Fully Automatic Face Recognition." BMVC. 2011.

  They present a novel approach to pose-invariant face recognition that handles continuous pose variations, is not database-specific, and achieves high accuracy without any manual intervention.

Berg, Thomas, and Peter N. Belhumeur. "Tom-vs-Pete Classifiers and Identity-Preserving Alignment for Face Verification." BMVC. Vol. 2. 2012.

  Berg, Thomas, and Peter N. Belhumeur. "Tom-vs-Pete Classifiers and Identity-Preserving Alignment for Face Verification." BMVC. Vol. 2. 2012.

Ashraf, Ahmed Bilal, Simon Lucey, and Tsuhan Chen. "Learning patch correspondences for improved viewpoint invariant face recognition." Computer Vision and Pattern Recognition, 2008. CVPR 2008. IEEE Conference on. IEEE, 2008.

  Variation due to viewpoint is one of the key challenges that stand in the way of a complete solution to the face recognition problem. It is easy to note that local regions of the face change differently in appearance as the viewpoint varies. Recently, patch-based approaches, such as those of Kanade and Yamada, have taken advantage of this effect resulting in improved viewpoint invariant face recognition. In this paper we propose a data-driven extension to their approach, in which we not only model how a face patch varies in appearance, but also how it deforms spatially as the viewpoint varies.

Beymer ve Poggio, 1995. Face recognition from one example review. Proc. IEEE Int. Conf. Comput. Vis., pp.500-507

  To create a pose-invariant face recognizer, one strategy is the view-based approach, which uses a set of real example views at different poses. But what if we only have one real view available, such as a scanned passport photo-can we still recognize faces under different poses? Given one real view at a known pose, it is still possible to use the view-based approach by exploiting prior knowledge of faces to generate virtual views, or views of the face as seen from different poses. To represent prior knowledge, we use 2D example views of prototype faces under different rotations.

Li, Shaoxin, et al. "Morphable displacement field based image matching for face recognition across pose." European Conference on Computer Vision. Springer Berlin Heidelberg, 2012.

  Fully automatic Face Recognition Across Pose (FRAP) is one of the most desirable techniques, however, also one of the most challenging tasks in face recognition field. Matching a pair of face images in different poses can be converted into matching their pixels corresponding to the same semantic facial point. Following this idea, given two images G and P in different poses, we propose a novel method, named Morphable Displacement Field (MDF), to match G with P’s virtual view under G’s pose.

Li, Shaoxin, et al. "Maximal likelihood correspondence estimation for face recognition across pose." IEEE Transactions on Image Processing 23.10 (2014): 4587-4600.

  Due to the misalignment of image features, the performance of many conventional face recognition methods degrades considerably in across pose scenario. To address this problem, many image matching-based methods are proposed to estimate semantic correspondence between faces in different poses.

Yin, Qi, Xiaoou Tang, and Jian Sun. "An associate-predict model for face recognition." Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on. IEEE, 2011.

  Handling intra-personal variation is a major challenge in face recognition. It is difficult how to appropriately measure the similarity between human faces under significantly different settings (e.g., pose, illumination, and expression). In this paper, we propose a new model, called “Associate-Predict” (AP) model, to address this issue.

Zhang, Yizhe, et al. "Random faces guided sparse many-to-one encoder for pose-invariant face recognition." Proceedings of the IEEE International Conference on Computer Vision. 2013.

  One of the most challenging task in face recognition is to identify people with varied poses. Namely, the test faces have significantly different poses compared with the registered faces. In this paper, we propose a high-level feature learning scheme to extract pose-invariant identity feature for face recognition.

Kan, Meina, et al. "Stacked progressive auto-encoders (spae) for face recognition across poses." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2014.

  Identifying subjects with variations caused by poses is one of the most challenging tasks in face recognition, since the difference in appearances caused by poses may be even larger than the difference due to identity. Inspired by the observation that pose variations change non-linearly but smoothly, we propose to learn pose-robust features by modeling the complex non-linear transform from the non-frontal face images to frontal ones through a deep network in a progressive way, termed as stacked progressive auto-encoders (SPAE).

Hassner, Tal. "Viewing real-world faces in 3D." Proceedings of the IEEE International Conference on Computer Vision. 2013.

  They present a data-driven method for estimating the 3D shapes of faces viewed in single, unconstrained photos (aka "in-the-wild"). Their method was designed with an emphasis on robustness and efficiency with the explicit goal of deployment in real-world applications which reconstruct and display faces in 3D.

Gao, Yongsheng, et al. "Fast face identification under varying pose from a single 2-D model view." IEE Proceedings-Vision, Image and Signal Processing 148.4 (2001): 248-253.

  One of the key remaining problems in face recognition is that of handling the variability in appearance due to changes in pose. The authors present a simple and computationally efficient 3-D pose recovery methodology.

Lee, Mun Wai, and Surendra Ranganath. "Pose-invariant face recognition using a 3D deformable model." Pattern recognition 36.8 (2003): 1835-1846.

  The paper proposes a novel, pose-invariant face recognition system based on a deformable, generic 3D face model, that is a composite of: (1) an edge model, (2) a color region model and (3) a wireframe model for jointly describing the shape and important features of the face.

Liu, Xiaoming, and Tsuhan Chen. "Pose-robust face recognition using geometry assisted probabilistic modeling." 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05). Vol. 1. IEEE, 2005.

  Researchers have been working on human face recognition for decades. Face recognition is hard due to different types of variations in face images, such as pose, illumination and expression, among which pose variation is the hardest one to deal with. To improve face recognition under pose variation, this paper presents a geometry assisted probabilistic approach.

Hassner, Tal, et al. "Effective face frontalization in unconstrained images." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015.

  "Frontalization" is the process of synthesizing frontal facing views of faces appearing in single unconstrained photos. Recent reports have suggested that this process may substantially boost the performance of face recognition systems. This, by transforming the challenging problem of recognizing faces viewed from unconstrained viewpoints to the easier problem of recognizing faces in constrained, forward facing poses.

Zhu, Xiangyu, et al. "High-fidelity pose and expression normalization for face recognition in the wild." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015.

  In this paper, they propose a High-fidelity Pose and Expression Normalization (HPEN) method with 3D Morphable Model (3DMM) which can automatically generate a natural face image in frontal pose and neutral expression.

Paysan, Pascal, et al. "A 3D face model for pose and illumination invariant face recognition." Advanced video and signal based surveillance, 2009. AVSS'09. Sixth IEEE International Conference on. IEEE, 2009.

  With this paper they publish a generative 3D shape and texture model, the Basel Face Model (BFM), and demonstrate its application to several face recognition task. They improve on previous models by offering higher shape and texture accuracy due to a better scanning device and less correspondence artifacts due to an improved registration algorithm.

Zhang, Xiaozheng, Yongsheng Gao, and Maylor KH Leung. "Recognizing rotated faces from frontal and side views: An approach toward effective use of mugshot databases." IEEE Transactions on Information Forensics and Security 3.4 (2008): 684-697.

  This paper presents a novel appearance-based approach using frontal and sideface images to handle pose variations in face recognition, which has great potential in forensic and security applications involving police mugshot databases.

Chen, Weiping, and Yongsheng Gao. "Recognizing partially occluded faces from a single sample per class using string-based matching." European Conference on Computer Vision. Springer Berlin Heidelberg, 2010.

  Automatically recognizing human faces with partial occlusions is one of the most challenging problems in face analysis community. This paper presents a novel string-based face recognition approach to address the partial occlusion problem in face recognition.

Sagonas, Christos, et al. "Robust Statistical Face Frontalization." Proceedings of the IEEE International Conference on Computer Vision. 2015.

  In this paper, we propose a novel method for joint frontal view reconstruction and landmark localization using a small set of frontal images only.

Rudovic, Ognjen, and Maja Pantic. "Shape-constrained Gaussian process regression for facial-point-based head-pose normalization." 2011 International Conference on Computer Vision. IEEE, 2011.

  Given the facial points extracted from an image of a face in an arbitrary pose, the goal of facial-point-based head-pose normalization is to obtain the corresponding facial points in a predefined pose (e.g., frontal). This involves inference of complex and high-dimensional mappings due to the large number of the facial points employed, and due to differences in head-pose and facial expression.

Yim, Junho, et al. "Rotating your face using multi-task deep neural network." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015.

  This paper proposes a new deep architecture based on a novel type of multitask learning, which can achieve superior performance in rotating to a target-pose face image from an arbitrary pose and illumination image while preserving identity.