Face Recognition Using Both Visible light Image and Near-infrared image and a Deep Network
可见光图像和近红外图像融合的人脸识别
哈工大深圳研究生院2017年发表在《Caai Transactions on Intelligence Technology》的一篇论文，模型开源，地址在论文中http://www.yongxu.org/lunwen.html。
概述：
不同的光照条件影响可见光下的人脸图像，往往导致基于可见光图像的人脸识别更加困难。本文提出使用对光照变化不敏感的近红外图像+普通可见光图像的人脸识别方案。
思想
（1）基于VGG训练一个基于RGB人脸图像的人脸识别模型，记为模型V；
（2）使用近红外人脸图像，以模型V为base，finetuning一个适用于红外图像的人脸识别模型，记为模型N；
（3）使用时，同时捕捉人脸的RGB图像和NIR，使用RGB和模型V得RBG人脸特征；使用NIR和模型N的NIR人脸特征；
（4）特征比对时，RGB特征与RGB特征做相似度比对得一个得分，VIR特征与VIR特征做相似度比对得一个得分;
（5）两个得分做加权融合，得最终得分。得分高的占比重高，得分低的占比重低。计算公式如下：
网络模型结构：
系统结构示意图：
训练：
（1）数据集，CelebA训练模型V，CASIA NIR dataset and PolyU NIR Face Database finetuning得模型N；
The CelebA dataset [25] contains 202599 face images captured from 10177 identities, and contains rich posture and background variations.
The CASIA NIR database contains 2490 NIR face images of 197 peoples and the PolyU NIR Face Database includes 35000 NIR Face images of 350 peoples.
（2）验证集
使用了包含VIS和NIR两种图像，且存在不同光照条件下图像的数据集
HIT LAB2 face dataset [28] and SunWin Face database
The SunWin Face database contains 4000 face images from
100 identities. It has two parts: 1) 2000 visible light pictures
from the 100 identities. For each person, 10 pictures are
collected under normal light, the other 10 pictures per person
are captured under abnormal light. 2) 2000 near-infrared
pictures from the 100 identities. For each person, 10 pictures are
also obtained under normal light and the other 10 pictures are
captured under abnormal light. The collected database contains
different facial expressions, lights and other changes. A visible
light camera and a near-infrared camera were used to collect
data at the same time.
The HITSZ Lab2 dataset was collected and issued by Harbin
Institute of Technology. The database contains a total of 2000
face images from 50 volunteers. The image size is 200X200.
These images were collected under the following different
lighting conditions: (a) natural light (b) natural light + left light
(c) natural light + right light (d) natural light + left and right side
lights. The image also contains significant posture or facial
expression changes.
（3）结果
可以看出在光照改变，尤其是明显光照改变的情况下，本文提出的两种模型融合的策略能够获得更高的准确度。
总结：
应该算是中规中矩，简单易实现，也有效的一个方法。模型可替换，融合策略可借鉴。