LARNet: Lie Algebra Residual Network for Face Recognition

Xiaolong Yang 1,2    Xiaohong Jia 1,2    Dihong Gong 3    Dong-Ming Yan 4,2    Zhifeng Li 3    Wei Liu 3    1 Academy of Mathematics and Systems Science of the Chinese Academy of Sciences (AMSS, CAS) 2 University of Chinese Academy of Sciences 3 Tencent Data Platform 4 National Laboratory of Pattern Recognition, Institute of Automation Chinese Academy of Sciences (NLPR,CASIA)

Frontalization or rotation in the feature space. To show the equivalence, we reconstruct the images corresponding to the modified features (blue dot) and provide the visual results for the expected frontal faces.

Abstract

Face recognition is an important yet challenging problem in computer vision. A major challenge in practical face recognition applications lies in significant variations between profile and frontal faces. Traditional techniques address this challenge either by synthesizing frontal faces or by pose invariant learning. In this paper, we propose a novel method with Lie algebra theory to explore how face rotation in the 3D space affects the deep feature generation process of convolutional neural networks (CNNs). We prove that face rotation in the image space is equivalent to an additive residual component in the feature space of CNNs, which is determined solely by the rotation. Based on this theoretical finding, we further design a Lie Algebraic Residual Network (LARNet) for tackling pose robust face recognition. Our LARNet consists of a residual subnet for decoding rotation information from input face images, and a gating subnet to learn rotation magnitude for controlling the strength of the residual component contributing to the feature learning process. Comprehensive experimental evaluations on both frontal-profile face datasets and general face recognition datasets convincingly demonstrate that our method consistently outperforms the state-of-the-art ones.

Key Theory and Network Architecture

Visualization Results

Our method is robust to many factors, including gender, head decoration (hat), and face decoration (glasses, beard).

Bibtex

@article{Yang2023LARNeXt,
    author={Xiaolong Yang, Xiaohong Jia, Dihong Gong, Dong-Ming Yan, ZhiFeng Li, Wei Liu},
    title={LARNeXt: End-to-End Lie Algebra Residual Network for Face Recognition},
    journal={IEEE Transactions on Pattern Analysis and Machine Intelligence(T-PAMI)},
    volume={45},
    number={10},
    pages={11961-11976},
    year={2023}
}

@inproceedings{Yang2021LARNet,
    author={Xiaolong Yang, Xiaohong Jia, Dihong Gong, Dong-Ming Yan, ZhiFeng Li, Wei Liu},
    title={LARNet: Lie Algebra Residual Network for Face Recognition},
    booktitle= {Proceedings of the 38th International Conference on Machine Learning (ICML2021)},
    year={2021}
}