中文

Journal of Shanghai University >

2017 , Vol. 23 >Issue 3: 353 - 363

DOI: https://doi.org/10.12066/j.issn.1007-2861.1713

Automatic annotation for film and Television drama shots and recut system based on face identification

Expand
  • 1. Shanghai Film Academy, Shanghai University, Shanghai 200072, China;
    2. Tencent Inc., Tencent-BestImage, Shanghai 200030, China

Received date: 2015-11-23

  Online published: 2017-06-30

Fold

Abstract

This paper proposes an automatic editing system named Star Cut based on face recognition using deep learning and a video shot detection technique. The purpose is to establish a system for management, retrieval, and automatic recut of film and TV shots. First, the system with over 350 faces of pop stars and actors using a U-face model is trained to learn facial features. The system uses the change rate of edges to detect shot edge. After shot segmentation, the system uses the pre-trained face models to identify faces in the input film or TV drama shot by shot. Users can either choose to recognize all figures in these shots or just choose selected one to recut all the shots containing him/her together automatically. The recall rate of shot segmentation is above 95%, and the recognition rate of faces in an shooting angle of 45° is 92.45%. Test results show that the proposed system has good robustness.

Key words： deep learning; face identification; shot segmentation; face detection

Cite this article

LANG Yue1, ZHOU Jiting1, LIANG Xiaolong2, ZHANG Wenjun1 . Automatic annotation for film and Television drama shots and recut system based on face identification[J]. Journal of Shanghai University, 2017 , 23(3) : 353 -363 . DOI: 10.12066/j.issn.1007-2861.1713

References

[1] Zhao W, Chellappa R, Phillips P J, et al. Face recognition: a literature survey [J]. ACM Computing Surveys, 2003, 35(4): 399-458.
[2] Bhele S G, Mankar V H. A review paper on face recognition techniques [J]. International Journal of Advanced Research in Computer Engineering & Technology, 2012, 1(8): 339-346.
[3] Taigman Y, Yang M, Ranzato M A, et al. Deepface: closing the gap to human-level performance in face verification [C]//2014 IEEE Conference on Computer Vision and Pattern Recognition(CVPR). 2014: 1701-1708.
[4] Sun Y, Wang X, Tang X. Deeply learned face representations are sparse, selective, and robust [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.
2015: 2892-2900.
[5] Schroff F, Kalenichenko D, Philbin J. Facenet: a unified embedding for face recognition and clustering [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern
Recognition. 2015: 815-823.
[6] Zhou E, Cao Z, Yin Q. Naive-deep face recognition: touching the limit of LFW benchmark or not? [EB/OL]. (2015-02-20)[2015-11-23]. http://arxiv.org/pdf/1501.04690vl.pdf.
[7] 孟一飞, 袁雪, 魏学业, 等. 基于样本扩展和线性子空间特征提取的单样本人脸识别[J]. 应用科学学报, 2013, 31(5): 488-494.
[8] 严严, 章毓晋. 基于视频的人脸识别研究进展[J]. 计算机学报, 2009, 32(5): 878-886.
[9] Gorodnichy D O, Granger E, Radtke P, et al. Survey of commercial technologies for face recognition in video [R]. Ottawa: Departmant of National Defence(Canada), 2014.
[10] Granger E, Radtke P, Gorodnichy D, et al. Survey of academic research and prototypes for face recognition in video [R]. Ottawa: Department of National Defence(Canada), 2014.
[11] Lei Y, Bennamoun M, Hayat M, et al. An efficient 3D face recognition approach using local geometrical signatures [J]. Pattern Recognition, 2014, 47(2): 509-524.
[12] Face Analysisl Tencent Youtu [EB/OL]. [2015-10-11]. http://bestimage.qq.com/faceanalyze.html.

Options
Outlines

/

〈 〉