摘 要：针对人脸活体检测技术在复杂光线条件下识别率低，边缘设备硬件条件有限的问题，文章提出一种基于注意力机制融合的多模态人脸活体检测算法。将可见光图的 PLGF 特征图和近红外图作为输入，通过注意力机制进行融合，并采用颜色通道差值图进行辅助判定。设计了轻量化骨干网络提升算法在边缘设备中的推理效率，参数量相比 mobelienetV2 减少 80%。在自建数据集上的实验表明，人脸活体检测准确率为 99.93%，可有效提升算法在不同攻击方式、不同光照条件下的准确率。
关键词：注意力机制；多模态融合；人脸活体检测；近红外；深度神经网络
DOI:10.19850/j.cnki.2096-4706.2022.20.017
基金项目： 广州市科技计划项目（202206030001，202206030002）
中图分类号：TP391.4；TN948.6 文献标识码：A 文章编号：2096-4706（2022）20-0065-06
Attention-Based Multimodal Fusion for Face Anti-spoofing
WEI Dong1, YUE Xuyao1, ZHANG Liepiao2, HUANG Yuheng1
（1. GRG Banking Equipment Co., Ltd., Guangzhou 510700, China; 2.GRG Tally-vision I.T. Co., Ltd., Guangzhou 510700, China)
Abstract: In order to solve the problems of low recognition rate of face anti-spoofing technology in complex light conditions and limited hardware conditions of edge equipment, this paper proposes a Attention-Based Multimodal Fusion for face anti-spoofing. The PLGF characteristic image and near-infrared image of the visible light image are taken as the input, and the fusion is carried out through the attention mechanism, and the color channel difference image is used for auxiliary judgment. The reasoning efficiency of lightweight backbone network promotion algorithm in edge devices is designed, and the number of parameters is reduced by 80% compared with mobelinetV2. The experiment on the self built dataset shows that the accuracy rate of face anti-spoofing is 99.93%, which can effectively improve the accuracy of the algorithm under different attack modes and different lighting conditions.
Keywords: attention mechanism; multimodal fusion; face anti-spoofing; near infrared; deep neural network
参考文献：
[1] 邓雄，王洪春，赵立军，等 . 人脸识别活体检测研究方法综述 [J]. 计算机应用研究，2020，37（9）：25-31.
[2] LI J W，WANG Y H，TIENIU T，et al. Live face detection based on the analysis of Fourier spectra [J].Physica A: Statistical Mechanics and its Applications，2004，5404：296-303.
[3] SONG X，ZHAO X，FANG L J，et al. Discriminative representation combinations for accurate face spoofing detection [J]. Pattern Recognition，2019，85：220-231.
[4] LUO S, KAN M, WU S, et al. Face anti-spoofing with multiscale information [C]//2018 24th International Conference on Pattern Recognition (ICPR). Beijing：IEEE，2018：3402-3407.
[5] 邓茜文，冯子亮，邱晨鹏 . 基于近红外与可见光双目视觉的活体人脸检测方法 [J]. 计算机应用，2020，40（7）：2096-2103.
[6] SANDLER M，HOWARD A，ZHU M，et al. MobileNetv2: inverted residuals and linear bottlenecks [C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City：IEEE，2018：4510-4520.
[7] DEBOTOSH B，HIRANMOY R. Pattern of local gravitational force (plgf): A novel local image descriptor [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence，2021，43（2）：595-607.
[8] CHEN S，LIU Y，GAO X，et al. MobilefaceNets: efficient CNNs for accurate real-time face verification on mobile devices [J/OL].arXiv:1804.07573 [cs.CV].[2022-07-15].https://arxiv.org/ abs/1804.07573.
[9] HE K M, ZHANG X Y, REN S Q, et al. Delving deep into rectifiers: surpassing human-level performance on ImageNet classification [C]//2015 IEEE International Conference on Computer Vision (ICCV). Santiago：IEEE，2015：1026-1034.
[10] PARK E，HAN X，BERG T L, et al. Combining multiple sources of knowledge in deep CNNs for action recognition [C]// Proceedings of IEEE Winter Conference on Applications of Computer Vision (WACV).Lake Placid：IEEE，2016：1-8.
[11] 李盼盼，王朝立，孙占全 . 基于注意力机制的多特征融合人脸活体检测 [J]. 信息与控制，2021，50（5）：631-640.
作者简介：魏东（1977—），男，汉族，四川乐山人，副总经理，硕士研究生，研究方向：智能视频分析、深度学习；岳许要（1984—），男，汉族，河南郑州人，资深算法工程师，硕士研究生，研究方向：智能视频分析、模式识别；章烈剽（1980—），男，汉族，湖北咸宁人，高工，工学硕士，研究方向：人脸的活体检测、图像处理及人工智能在金融行业的工程应用；黄宇恒（1980—），男，汉族，广东佛山人，研发经理，博士研究生，研究方向：视频解析、自然语言处理。