基于人脸识别的影视剧镜头自动标注及重剪系统

doi:10.12066/j.issn.1007-2861.1713

Journal of Shanghai University（Natural Science Edition） ›› 2017, Vol. 23 ›› Issue (3): 353-363.doi: 10.12066/j.issn.1007-2861.1713

Previous Articles Next Articles

Automatic annotation for film and Television drama shots and recut system based on face identification

LANG Yue¹, ZHOU Jiting¹, LIANG Xiaolong², ZHANG Wenjun¹

1. Shanghai Film Academy, Shanghai University, Shanghai 200072, China;
2. Tencent Inc., Tencent-BestImage, Shanghai 200030, China

Received:2015-11-23 Online:2017-06-30 Published:2017-06-30

Abstract

Abstract:

This paper proposes an automatic editing system named Star Cut based on face recognition using deep learning and a video shot detection technique. The purpose is to establish a system for management, retrieval, and automatic recut of film and TV shots. First, the system with over 350 faces of pop stars and actors using a U-face model is trained to learn facial features. The system uses the change rate of edges to detect shot edge. After shot segmentation, the system uses the pre-trained face models to identify faces in the input film or TV drama shot by shot. Users can either choose to recognize all figures in these shots or just choose selected one to recut all the shots containing him/her together automatically. The recall rate of shot segmentation is above 95%, and the recognition rate of faces in an shooting angle of 45° is 92.45%. Test results show that the proposed system has good robustness.

Key words: deep learning, face identification, shot segmentation, face detection

LANG Yue1, ZHOU Jiting1, LIANG Xiaolong2, ZHANG Wenjun1. Automatic annotation for film and Television drama shots and recut system based on face identification[J]. Journal of Shanghai University（Natural Science Edition）, 2017, 23(3): 353-363.

References

[1] Zhao W, Chellappa R, Phillips P J, et al. Face recognition: a literature survey [J]. ACM Computing Surveys, 2003, 35(4): 399-458.
[2] Bhele S G, Mankar V H. A review paper on face recognition techniques [J]. International Journal of Advanced Research in Computer Engineering & Technology, 2012, 1(8): 339-346.
[3] Taigman Y, Yang M, Ranzato M A, et al. Deepface: closing the gap to human-level performance in face verification [C]//2014 IEEE Conference on Computer Vision and Pattern Recognition(CVPR). 2014: 1701-1708.
[4] Sun Y, Wang X, Tang X. Deeply learned face representations are sparse, selective, and robust [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.
2015: 2892-2900.
[5] Schroff F, Kalenichenko D, Philbin J. Facenet: a unified embedding for face recognition and clustering [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern
Recognition. 2015: 815-823.
[6] Zhou E, Cao Z, Yin Q. Naive-deep face recognition: touching the limit of LFW benchmark or not? [EB/OL]. (2015-02-20)[2015-11-23]. http://arxiv.org/pdf/1501.04690vl.pdf.
[7] 孟一飞, 袁雪, 魏学业, 等. 基于样本扩展和线性子空间特征提取的单样本人脸识别[J]. 应用科学学报, 2013, 31(5): 488-494.
[8] 严严, 章毓晋. 基于视频的人脸识别研究进展[J]. 计算机学报, 2009, 32(5): 878-886.
[9] Gorodnichy D O, Granger E, Radtke P, et al. Survey of commercial technologies for face recognition in video [R]. Ottawa: Departmant of National Defence(Canada), 2014.
[10] Granger E, Radtke P, Gorodnichy D, et al. Survey of academic research and prototypes for face recognition in video [R]. Ottawa: Department of National Defence(Canada), 2014.
[11] Lei Y, Bennamoun M, Hayat M, et al. An efficient 3D face recognition approach using local geometrical signatures [J]. Pattern Recognition, 2014, 47(2): 509-524.
[12] Face Analysisl Tencent Youtu [EB/OL]. [2015-10-11]. http://bestimage.qq.com/faceanalyze.html.

Automatic annotation for film and Television drama shots and recut system based on face identification

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 4

Recommended Articles

Metrics

Comments

[1]	Jiacheng HU, Xiangyang WANG, Han LIU. Defect detection of continuous casting slabs based on deep learning [J]. Journal of Shanghai University（Natural Science Edition）, 2019, 25(4): 445-452.
[2]	XIE Zhifeng, YE Guanhua, YAN Shuqi, HE Shaorong, DING Youdong. HDR image style transfer technique based on generative adversarial networks [J]. Journal of Shanghai University（Natural Science Edition）, 2018, 24(4): 524-534.
[3]	SHEN Wenfeng, ZHANG Jianlei, ZHOU Dingqian, CHEN Shengbo, QIU Feng. Recognition of Chinese characters on license plates based on big data [J]. Journal of Shanghai University（Natural Science Edition）, 2016, 22(1): 88-96.
[4]	DING You-dong;DU Xiao-feng;Li Xiao-qiang. Human Face Detection Based on Cluster Skin Color Model [J]. Journal of Shanghai University（Natural Science Edition）, 2007, 13(5): 511-515 .