轻量化神经网络和哈希跟踪算法在嵌入式人脸抓拍系统中的应用

doi:10.12066/j.issn.1007-2861.2187

上海大学学报(自然科学版) ›› 2021, Vol. 27 ›› Issue (6): 1018-1028.doi: 10.12066/j.issn.1007-2861.2187

轻量化神经网络和哈希跟踪算法在嵌入式人脸抓拍系统中的应用

许庚林, 冉峰(), 邓良, 史华康, 郭爱英

上海大学微电子研究与开发中心, 上海 200444

收稿日期:2019-09-25 出版日期:2021-12-31 发布日期:2020-01-09
通讯作者: 冉峰 E-mail:ranfeng@shu.edu.cn
作者简介:冉峰(1954—), 男, 教授, 博士生导师, 博士, 研究方向为 SoC 设计、OLED 驱动电路. E-mail: ranfeng@shu.edu.cn
基金资助:
国家自然科学基金资助项目(61774100);上海市重大基础研发项目(18JC1410402)

Application of lightweight neural network and Hash tracking algorithm in embedded face capture system

XU Genglin, RAN Feng(), DENG Liang, SHI Huakang, GUO Aiying

Microelectronic Research and Development Center, Shanghai University, Shanghai 200444, China

Received:2019-09-25 Online:2021-12-31 Published:2020-01-09
Contact: RAN Feng E-mail:ranfeng@shu.edu.cn

摘要/Abstract

摘要：

针对嵌入式设备上难以兼顾人脸抓拍的速度和准确率的问题, 基于轻量化神经网络和哈希 (Hash) 跟踪算法设计了一种快速精准的嵌入式人脸抓拍系统. 首先, 对轻量化网络 MobileNet 固态硬盘 (solid state disk, SSD) 剪枝和优化网络结构构建人脸检测网络; 其次, 人脸对齐后基于均值哈希 (average Hash, aHash) 与感知哈希 (perceptual Hash, pHash) 设计融合哈希 (fusion Hash, fHash) 算法跟踪人脸, 使用关键点欧氏距离、人脸尺寸和四方向 Sobel 算子三标准提取最佳的人脸图像; 最后, 使用 MobileFaceNet 对最佳人脸进行识别. 实验结果表明: 与 MobileNet SSD 相比, 该人脸检测算法速度提升了 22.6%; 与均值哈希和感知哈希算法相比, 该融合哈希算法匹配准确率提高了 21.7% 和 10.1%; 实际场景中系统人脸抓拍准确率超过 95%, 抓拍速度达到 28 帧/s.

关键词: 人脸抓拍, 嵌入式, 哈希跟踪, 深度学习

Abstract:

To address the difficulty of balancing the speed and accuracy of face capture systems embedded in mobile devices, a fast and accurate embedded face capture system is designed based on the combination of a lightweight neural network and a Hash tracking algorithm. Firstly, an optimised face detection network is constructed based on the lightweight MobileNet solid state disk (SSD) object detector. After pruning and face alignment, a Hash algorithm is combined to track a face based on the average Hash (aHash) and a perceptual Hash (pHash) design, in which the key point distance and face size as well as the four-direction Sobel operator are used as the three standards to extract the best face image. Finally, MobileFaceNet is applied to identify the best face. The experimental results indicate that the speed of the proposed face detection algorithm is increased by 22.6% compared to using the standard MobileNet SSD. Furthermore, compared with using only the mean Hash or the pHash algorithm, the matching accuracy is improved by 21.7% and 10.1% when the combined Hash algorithm is applied, respectively. The face capture accuracy is greater than 95%, and the capture speed can reach 28 frame/s.

Key words: face capture, embedded, Hash tracking, deep learning

中图分类号:

许庚林, 冉峰, 邓良, 史华康, 郭爱英. 轻量化神经网络和哈希跟踪算法在嵌入式人脸抓拍系统中的应用[J]. 上海大学学报(自然科学版), 2021, 27(6): 1018-1028.

XU Genglin, RAN Feng, DENG Liang, SHI Huakang, GUO Aiying. Application of lightweight neural network and Hash tracking algorithm in embedded face capture system[J]. Journal of Shanghai University（Natural Science Edition）, 2021, 27(6): 1018-1028.

图/表 16

图1

图2

图3

图4

表1

图5

表2

表3

表4

表5

表6

图6

表7

图7

图8

表8

参考文献 14

[1]	Ren S Q, He K M, Ross G, et al. Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE Trans on Pattern Anal Mach Intell, 2015, 39(6): 1137-1149. doi: 10.1109/TPAMI.2016.2577031
[2]	He K, Zhang X, Ren S, et al. Deep residual learning for image recognition[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016: 770-778.
[3]	Zhang K, Zhang Z, Li Z, et al. Joint face detection and alignment using multi-task cascaded convolutional networks[J]. Signal Processing Letters, 2016, 23(10): 1499-1503. doi: 10.1109/LSP.2016.2603342
[4]	Deng J, Guo J, Xue N, et al. ArcFace: additive angular margin loss for deep face recog-nition[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019: 4690-4699.
[5]	Howard A G, Zhu M, Chen B, et al. Mobilenets: efficient convolutional neural networks for mobile vision applications [EB/OL]. (2017-04-17) [2019-08-10]. https://arxiv.org/abs/1704.04861.
[6]	Zhang X, Zhou X, Lin M, et al. Shufflenet: an extremely efficient convolutional neural network for mobile devices[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018: 6848-6856.
[7]	Liu C M, Wu K G. A single kalman filtering algorithm for maneuvering target tracking[J]. Aeousties, Speech, and Signal Proeessing, 1994, 6:193-196.
[8]	Yu F W, Li W B, Li Q Q, et al. Multiple object tracking with high performance detection and appearance feature[C]// European Conference on Computer Vision. 2016: 36-42.
[9]	Bochinski E, Eiselein V, Sikora T. High-speed tracking-by-detection without using image information[C]// Proceedings of IEEE International Conference on Advanced Video and Signals-based Surveillance. 2017: 1-6.
[10]	Hilke K, Hubner W, Michael A. Joint detection and online multi-object tracking[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018: 1540-1548.
[11]	Liu W, Anguelov D, Erhan D, et al. SSD: single shot multibox detector[C]// European Conference on Computer Vision. 2016: 21-37.
[12]	钱青, 臧冬菊. 一种改进的 Sobel 算子图像清晰度评价函数[J]. 计算机与数字工程, 2015, 43(10): 1865-1870.
	Qian Q, Zang D J. A modified sharpness-evaluation function of image based on Sobel[J]. Computer and Digital Engineering, 2015, 43(10): 1865-1870.
[13]	Chen S, Liu Y, Gao X, et al. MobileFaceNets: efficient CNNs for accurate real-time face verification on mobile devices [EB/OL]. (2018-06-15) [2019-08-18]. https://arxiv.org/abs/1804.07573.
[14]	郭仪权. 基于哈希的多目标跟踪算法的研究[D]. 合肥: 安徽大学, 2017.
	Guo Y Q. Research on multiple target tracking algorithm based on Hash[D]. Hefei: Anhui University, 2017.

轻量化神经网络和哈希跟踪算法在嵌入式人脸抓拍系统中的应用

Application of lightweight neural network and Hash tracking algorithm in embedded face capture system

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 16

参考文献 14

相关文章 10

编辑推荐

Metrics

本文评价

[1]	邢毅雪, 朱永华, 高海燕, 周金, 张克. 基于注意力机制的远程监督实体关系抽取[J]. 上海大学学报(自然科学版), 2021, 27(5): 983-992.
[2]	邵慧翔, 曾丹. 基于改进 YOLOv3 算法的水下小目标分类与识别[J]. 上海大学学报(自然科学版), 2021, 27(3): 481-491.
[3]	陈钰, 丁友东, 于冰, 徐敏. 基于像素流的视频彩色化[J]. 上海大学学报(自然科学版), 2021, 27(1): 18-27.
[4]	胡嘉成, 王向阳, 刘晗. 基于深度学习的连铸坯表面缺陷检测[J]. 上海大学学报(自然科学版), 2019, 25(4): 445-452.
[5]	谢志峰, 叶冠桦, 闫淑萁, 何绍荣, 丁友东. 基于生成对抗网络的HDR图像风格迁移技术[J]. 上海大学学报(自然科学版), 2018, 24(4): 524-534.
[6]	郎玥1, 周霁婷1, 梁小龙2, 张文俊1. 基于人脸识别的影视剧镜头自动标注及重剪系统[J]. 上海大学学报(自然科学版), 2017, 23(3): 353-363.
[7]	沈文枫, 张建蕾, 周丁倩, 陈圣波, 邱峰. 大数据时代的车牌汉字识别[J]. 上海大学学报(自然科学版), 2016, 22(1): 88-96.
[8]	王志恒, 金翊, 欧阳山. 三值光学计算机SD11 解码器的实现及改进[J]. 上海大学学报(自然科学版), 2015, 21(2): 245-256.
[9]	陈俊丽，卿定湖，李翔，万旺根. 基于多小波的彩色图像分层树集合分裂算法[J]. 上海大学学报(自然科学版), 2011, 17(1): 39-43.
[10]	李炜，潘志浩，汪敏. 基于APS-CMOS图像传感器的运动体捕捉方法[J]. 上海大学学报(自然科学版), 2009, 15(1): 47-50.