使用高分辨率网络在热红外图像上提取人脸关键区域温度

doi:10.19784/j.cnki.issn1672-0172.2020.06.001

Abstract

Abstract: In recent years, personal comfort model has attracted much attention. The temperature of facial skin surface is the key parameter in the model. This paper used high-resolution network (HRNet) to directly detect facial landmark on thermal images to extract facial temperature. Achieve the superior accuracy over the state-of-the-art algorithm. Furthermore, we add face attributes as weights to the loss function to further improve the accuracy of difficult cases (such as profile faces, occlusions, exaggerated expressions). Finally, compare its performance on infrared images and RGB images in different conditions, and prove that infrared images can replace visible light images for accurate landmark detection under some special conditions (such as low light, privacy protection).

Key words: Facial landmark detection, Thermal image, High-resolution network

CLC Number:

XU Xiangguo, YIN Zhixin. Applying the high-resolution network technology to extract face area temperature from thermal images[J]. Journal of Appliance Science & Technology, 2020, 0(6): 28-33.

References

[1] Ghahramani A, Castro G, Becerik-Gerber B, et al.Infrared thermography of human face for monitoring thermoregulation performance and estimating personal thermal comfort[J]. Building and Environment, 2016, 109(Nov.): 1-11.
[2] Chaudhuri T, Zhai D, Soh Y C, et al.Thermal Comfort Prediction using Normalized Skin Temperature in a Uniform Built Environment[J]. Energy & Buildings, 2017: S0378778817327354.
[3] Choi J H, Yeom D.Development of the data-driven thermal satisfaction prediction model as a function of human physiological responses in a built environment[J]. Building & Environment, 2019, 150(MAR.): 206-218.
[4] Li D, Menassa C C, Kamat V R.Robust non-intrusive interpretation of occupant thermal comfort in built environments with low-cost networked thermal cameras[J]. Applied Energy, 2019, 251.
[5] Deng J, Guo J, Zhou Y, et al.RetinaFace: Single-stage Dense Face Localisation in the Wild[J]. 2019.
[6] Chu W T, Liu Y H.Thermal Facial Landmark Detection by Deep Multi-Task Learning[C]// 2019 IEEE 21st International Workshop on Multimedia Signal Processing (MMSP). IEEE, 2019.
[7] M. Kopaczka, K. Acar,D. Merhof.Robust facial landmark detection and face tracking in thermal infrared images using active appearance models. In VISIGRAPP (4: VISAPP), pages 150-158, 2016.
[8] Zhu J Y, Park T, Isola Pr, et al.Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks[C]// 2017 IEEE International Conference on Computer Vision (ICCV). IEEE, 2017.
[9] Chu W T, Liu Y H.Thermal Facial Landmark Detection by Deep Multi-Task Learning[C]// 2019 IEEE 21st International Workshop on Multimedia Signal Processing (MMSP). IEEE, 2019.
[10] Keong J, Dong X, Jin Z, et al.Multi-spectral Facial Landmark Detection[J]. 2020.
[11] Sun K, Xiao B, Liu D, et al.Deep High-Resolution Representation Learning for Human Pose Estimation[J]. arXiv e-prints, 2019.
[12] Wang, S.Liu, Z.Lv, S.Lv, Y.Wu, G.Peng, P.Chen, F.Wang, X. A Natural Visible and Infrared Facial Expression Database for Expression Recognition and Emotion Inference[J]. IEEE Transactions on Multimedia, 2010, 12(7):682-691.
[13] Wang S, Liu Z, Wang Z, et al.Analyses of a Multimodal Spontaneous Facial Expression Database[J]. IEEE Transactions on Affective Computing, 2013, 4(1):34-46.
[14] 袁浩期, 李扬, 王俊影, 等. 基于红外热像的行人面部温度高精度检测技术[J]. 红外技术, 2019, 41(12).
[15] OpenCV 4.4. Camera Calibration and 3D Reconstruction;2020.https://docs.opencv.org/4.4.0/d9/d0c/group__calib3d.html#ga4abc2ece9fab9398f2e560d53c8c9780.
[16] Newell A, Yang K, Deng J.Stacked Hourglass Networks for Human Pose Estimation[J]. 2016.
[17] Ronneberger O, Fischer P, Brox T.U-Net: Convolutional Networks for Biomedical Image Segmentation[J]. 2015.
[18] Wang J, Sun K, Cheng T, et al.Deep High-Resolution Representation Learning for Visual Recognition[J]. 2019.
[19] Zhang K, Zhang Z, Li Z, et al.Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks[J]. IEEE Signal Processing Letters, 2016, 23(10):1499-1503.
[20] Pandas 1.1.2.https://pandas.pydata.org/.
[21] Chen Y, Shen C, Wei X S, et al.Adversarial PoseNet: A Structure-aware Convolutional Network for Human Pose Estimation[J]. 2017.
[22] M. Kopaczka, K. Acar,D. Merhof.Robust facial landmark detection and face tracking in thermal infrared images using active appearance models. In VISIGRAPP (4: VISAPP), pages 150-158, 2016.
[23] W. Wu, C. Qian, S. Yang, Q. Wang, Y. Cai,Q. Zhou.Look at boundary: A boundary-aware face alignment algorithm.In CVPR, pages2129-2138, 2018. 2, 6, 7, 8.
[24] C. Sagonas, E. Antonakos,G, Tzimiropoulos, S. Zafeiriou, M. Pantic. 300 faces In-the-wild challenge: Database and results. Image and Vision Computing (IMAVIS), Special Issue on Facial Landmark Localisation "In-The-Wild". 2016.

[1]	. [J]. Journal of Appliance Science & Technology, 2020, 0(4): 8 .
[2]	. [J]. Journal of Appliance Science & Technology, 2020, 0(4): 9 -17 .
[3]	. [J]. Journal of Appliance Science & Technology, 2020, 0(4): 18 -25 .
[4]	. [J]. Journal of Appliance Science & Technology, 2020, 0(4): 26 -27 .
[5]	. [J]. Journal of Appliance Science & Technology, 2020, 0(5): 2 .
[6]	. [J]. Journal of Appliance Science & Technology, 2020, 0(6): 2 .
[7]	. [J]. Journal of Appliance Science & Technology, 2020, 0(6): 8 -13 .
[8]	. [J]. Journal of Appliance Science & Technology, 2020, 0(6): 14 -21 .
[9]	. [J]. Journal of Appliance Science & Technology, 2020, 0(6): 22 -24 .
[10]	. [J]. Journal of Appliance Science & Technology, 2020, 0(6): 27 .

Applying the high-resolution network technology to extract face area temperature from thermal images

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Metrics

Comments

Recommended 10

[1]	MA Jian, ZHU Xiaobing, LI Xiaofeng, HUANG Dong, CAO Jiankang, CHEN Hong. Food storage principle based on multi-factor control and flexible partitioned storage [J]. Journal of Appliance Science & Technology, 2020, 0(6): 34-38.
[2]	LAI Xiaocheng, XIONG Shuo. Research and analysis on enhanced vapor injection system of fixed frequency air conditioning [J]. Journal of Appliance Science & Technology, 2020, 0(6): 48-50.
[3]	HE Jie, ZHOU Hongming. Research on intelligent anti pinch control technology of switch door based on hall detection [J]. Journal of Appliance Science & Technology, 2020, 0(6): 74-76.
[4]	REN Tao, XIAO Chengjin, LIU Jiangbin, CHAI Ting, MENG Qingliang, SONG Qiang. Experimental investigation on the R410A two phase pressure drop in capillary tubes [J]. Journal of Appliance Science & Technology, 2020, 0(6): 82-85.
[5]	ZHANG Ziqi, YANG Shuang, WU Xiaoli, LI Xin. Interpretation of GB/T 23137-2020 "Heat pump water heater for household and similar application" [J]. Journal of Appliance Science & Technology, 2020, 0(6): 108-111.
[6]	LI Xiangwen, ZHANG Yu, WANG Qianxu, LIU Yicai. Research progress on suppressing throttling jet noise by steam super cavitation [J]. Journal of Appliance Science & Technology, 2020, 0(5): 59-61.
[7]	ZHOU Mengran, LUO Yongqian, SHI Qingqing, HE Lei. Analysis of the influence of variable frequency multiple on-line oil return and other factors on condensation [J]. Journal of Appliance Science & Technology, 2020, 0(5): 86-89.
[8]	LIU Jianru, HUANG Dong, QIAO Lin, ZHAO Rijing. Analysis on key technologies of energy saving, environmental protection and preservation of air-cooled refrigerators (1) [J]. Journal of Appliance Science & Technology, 2020, 0(3): 72-75.
[9]	CHEN Tun, SHEN Jian, YANG Leting, HUANG Yong, CUI Huailei. Improvement of biological reagents storage in household refrigerators based on TRIZ [J]. Journal of Appliance Science & Technology, 2020, 0(2): 36-37.
[10]	WANG Fei. Design and experiment of new heat pipe refrigerant system [J]. Journal of Appliance Science & Technology, 2020, 0(1): 36-39.
[11]	FEI Ling, HU Haimei, WEI Huiming. Research on electric field preservation technology of refrigerator reheating dishes [J]. Journal of Appliance Science & Technology, 2020, 0(1): 43-45.
[12]	HE Jiakuan, WANG Hui, JIANG Zou, WANG Xiaoni, HE Zhichao. Research on the relation between the modal and strain of air-condition pipe [J]. Journal of Appliance Science & Technology, 2020, 0(1): 58-61.
[13]	YANG Yue, ZHAO Ruochen, WANG Weige, LI Dasen. Research on improvement method of household air conditioner parallel flow evaporator abnormal fin noise [J]. Journal of Appliance Science & Technology, 2020, 0(3): 44-48.
[14]	CHEN Tun, SHEN Jian, YANG Leting, CUI Huailei, BAO Xiaoqing, HUANG Yong. Preparation of easy-to-clean coating and its application in refrigerator [J]. Journal of Appliance Science & Technology, 2020, 0(4): 47-49.
[15]	LI Dongyuan. Application analysis of algebraic spiral in scroll compressor [J]. Journal of Appliance Science & Technology, 2020, 0(4): 72-76.