使用高分辨率网络在热红外图像上提取人脸关键区域温度

doi:10.19784/j.cnki.issn1672-0172.2020.06.001

摘要/Abstract

摘要： 近年来,个人热舒适模型取得了众多学者的研究和关注。其中,脸部皮肤表面关键区域的温度是模型中的关键输入特征。使用高分辨率特征学习网络（HRNet）在热外图像上直接进行人脸关键点检测,以提取人脸关键区域温度,其准确率高于同类热红外人脸关键点识别算法的最佳值。还将人脸属性作为权重加入到损失函数中,进一步提升了对于难例（如侧脸、遮挡、夸张表情）的识别准确率。最后,对比了热红外图像与不同环境下可见光图像的检测效果,证明了在一些特殊条件下（如低光照、隐私保护）,热红外图像可以代替可见光图像进行准确的关键点识别。

关键词: 人脸关键点识别, 热红外图像, 高分辨率特征学习网络

Abstract: In recent years, personal comfort model has attracted much attention. The temperature of facial skin surface is the key parameter in the model. This paper used high-resolution network (HRNet) to directly detect facial landmark on thermal images to extract facial temperature. Achieve the superior accuracy over the state-of-the-art algorithm. Furthermore, we add face attributes as weights to the loss function to further improve the accuracy of difficult cases (such as profile faces, occlusions, exaggerated expressions). Finally, compare its performance on infrared images and RGB images in different conditions, and prove that infrared images can replace visible light images for accurate landmark detection under some special conditions (such as low light, privacy protection).

Key words: Facial landmark detection, Thermal image, High-resolution network

中图分类号:

徐象国, 尹志鑫. 使用高分辨率网络在热红外图像上提取人脸关键区域温度[J]. 家电科技, 2020, 0(6): 28-33.

XU Xiangguo, YIN Zhixin. Applying the high-resolution network technology to extract face area temperature from thermal images[J]. Journal of Appliance Science & Technology, 2020, 0(6): 28-33.

参考文献

[1] Ghahramani A, Castro G, Becerik-Gerber B, et al.Infrared thermography of human face for monitoring thermoregulation performance and estimating personal thermal comfort[J]. Building and Environment, 2016, 109(Nov.): 1-11.
[2] Chaudhuri T, Zhai D, Soh Y C, et al.Thermal Comfort Prediction using Normalized Skin Temperature in a Uniform Built Environment[J]. Energy & Buildings, 2017: S0378778817327354.
[3] Choi J H, Yeom D.Development of the data-driven thermal satisfaction prediction model as a function of human physiological responses in a built environment[J]. Building & Environment, 2019, 150(MAR.): 206-218.
[4] Li D, Menassa C C, Kamat V R.Robust non-intrusive interpretation of occupant thermal comfort in built environments with low-cost networked thermal cameras[J]. Applied Energy, 2019, 251.
[5] Deng J, Guo J, Zhou Y, et al.RetinaFace: Single-stage Dense Face Localisation in the Wild[J]. 2019.
[6] Chu W T, Liu Y H.Thermal Facial Landmark Detection by Deep Multi-Task Learning[C]// 2019 IEEE 21st International Workshop on Multimedia Signal Processing (MMSP). IEEE, 2019.
[7] M. Kopaczka, K. Acar,D. Merhof.Robust facial landmark detection and face tracking in thermal infrared images using active appearance models. In VISIGRAPP (4: VISAPP), pages 150-158, 2016.
[8] Zhu J Y, Park T, Isola Pr, et al.Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks[C]// 2017 IEEE International Conference on Computer Vision (ICCV). IEEE, 2017.
[9] Chu W T, Liu Y H.Thermal Facial Landmark Detection by Deep Multi-Task Learning[C]// 2019 IEEE 21st International Workshop on Multimedia Signal Processing (MMSP). IEEE, 2019.
[10] Keong J, Dong X, Jin Z, et al.Multi-spectral Facial Landmark Detection[J]. 2020.
[11] Sun K, Xiao B, Liu D, et al.Deep High-Resolution Representation Learning for Human Pose Estimation[J]. arXiv e-prints, 2019.
[12] Wang, S.Liu, Z.Lv, S.Lv, Y.Wu, G.Peng, P.Chen, F.Wang, X. A Natural Visible and Infrared Facial Expression Database for Expression Recognition and Emotion Inference[J]. IEEE Transactions on Multimedia, 2010, 12(7):682-691.
[13] Wang S, Liu Z, Wang Z, et al.Analyses of a Multimodal Spontaneous Facial Expression Database[J]. IEEE Transactions on Affective Computing, 2013, 4(1):34-46.
[14] 袁浩期, 李扬, 王俊影, 等. 基于红外热像的行人面部温度高精度检测技术[J]. 红外技术, 2019, 41(12).
[15] OpenCV 4.4. Camera Calibration and 3D Reconstruction;2020.https://docs.opencv.org/4.4.0/d9/d0c/group__calib3d.html#ga4abc2ece9fab9398f2e560d53c8c9780.
[16] Newell A, Yang K, Deng J.Stacked Hourglass Networks for Human Pose Estimation[J]. 2016.
[17] Ronneberger O, Fischer P, Brox T.U-Net: Convolutional Networks for Biomedical Image Segmentation[J]. 2015.
[18] Wang J, Sun K, Cheng T, et al.Deep High-Resolution Representation Learning for Visual Recognition[J]. 2019.
[19] Zhang K, Zhang Z, Li Z, et al.Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks[J]. IEEE Signal Processing Letters, 2016, 23(10):1499-1503.
[20] Pandas 1.1.2.https://pandas.pydata.org/.
[21] Chen Y, Shen C, Wei X S, et al.Adversarial PoseNet: A Structure-aware Convolutional Network for Human Pose Estimation[J]. 2017.
[22] M. Kopaczka, K. Acar,D. Merhof.Robust facial landmark detection and face tracking in thermal infrared images using active appearance models. In VISIGRAPP (4: VISAPP), pages 150-158, 2016.
[23] W. Wu, C. Qian, S. Yang, Q. Wang, Y. Cai,Q. Zhou.Look at boundary: A boundary-aware face alignment algorithm.In CVPR, pages2129-2138, 2018. 2, 6, 7, 8.
[24] C. Sagonas, E. Antonakos,G, Tzimiropoulos, S. Zafeiriou, M. Pantic. 300 faces In-the-wild challenge: Database and results. Image and Vision Computing (IMAVIS), Special Issue on Facial Landmark Localisation "In-The-Wild". 2016.