收稿日期: 2015-11-23
网络出版日期: 2017-06-30
基金资助
国家自然科学基金资助项目(61303093); 上海市教委科研创新基金资助项目(14YZ023)
Automatic annotation for film and Television drama shots and recut system based on face identification
Received date: 2015-11-23
Online published: 2017-06-30
利用基于深度学习的人脸识别技术, 建立了一种基于人脸识别的影视剧镜头自动标注及重剪系统, 用于实现影视剧重编辑过程中对镜头片段更好地管理、查找和重剪. 先对输入的影视剧视频进行镜头检测和分割, 获得并建立分镜参数. 在此基础上, 对镜头中出现的所有人脸进行检测和切割, 并采用预先训练好的包含350多位明星特征的模型库予以身份识别, 聚类后实现镜头的演员标注. 该系统也可依据指定演员对影视剧进行搜索, 并将其中所有包含该演员的片段自动重剪在一起. 实验结果表明, 该系统镜头分割模块的平均召回率达到95%以上, 对45°以内的人脸识别率达到92.45%, 且具有良好的鲁棒性.
郎玥1, 周霁婷1, 梁小龙2, 张文俊1 . 基于人脸识别的影视剧镜头自动标注及重剪系统[J]. 上海大学学报(自然科学版), 2017 , 23(3) : 353 -363 . DOI: 10.12066/j.issn.1007-2861.1713
This paper proposes an automatic editing system named Star Cut based on face recognition using deep learning and a video shot detection technique. The purpose is to establish a system for management, retrieval, and automatic recut of film and TV shots. First, the system with over 350 faces of pop stars and actors using a U-face model is trained to learn facial features. The system uses the change rate of edges to detect shot edge. After shot segmentation, the system uses the pre-trained face models to identify faces in the input film or TV drama shot by shot. Users can either choose to recognize all figures in these shots or just choose selected one to recut all the shots containing him/her together automatically. The recall rate of shot segmentation is above 95%, and the recognition rate of faces in an shooting angle of 45° is 92.45%. Test results show that the proposed system has good robustness.
Key words： deep learning; face identification; shot segmentation; face detection
[1] Zhao W, Chellappa R, Phillips P J, et al. Face recognition: a literature survey [J]. ACM Computing Surveys, 2003, 35(4): 399-458.
[2] Bhele S G, Mankar V H. A review paper on face recognition techniques [J]. International Journal of Advanced Research in Computer Engineering & Technology, 2012, 1(8): 339-346.
[3] Taigman Y, Yang M, Ranzato M A, et al. Deepface: closing the gap to human-level performance in face verification [C]//2014 IEEE Conference on Computer Vision and Pattern Recognition(CVPR). 2014: 1701-1708.
[4] Sun Y, Wang X, Tang X. Deeply learned face representations are sparse, selective, and robust [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.
2015: 2892-2900.
[5] Schroff F, Kalenichenko D, Philbin J. Facenet: a unified embedding for face recognition and clustering [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern
Recognition. 2015: 815-823.
[6] Zhou E, Cao Z, Yin Q. Naive-deep face recognition: touching the limit of LFW benchmark or not? [EB/OL]. (2015-02-20)[2015-11-23]. http://arxiv.org/pdf/1501.04690vl.pdf.
[7] 孟一飞, 袁雪, 魏学业, 等. 基于样本扩展和线性子空间特征提取的单样本人脸识别[J]. 应用科学学报, 2013, 31(5): 488-494.
[8] 严严, 章毓晋. 基于视频的人脸识别研究进展[J]. 计算机学报, 2009, 32(5): 878-886.
[9] Gorodnichy D O, Granger E, Radtke P, et al. Survey of commercial technologies for face recognition in video [R]. Ottawa: Departmant of National Defence(Canada), 2014.
[10] Granger E, Radtke P, Gorodnichy D, et al. Survey of academic research and prototypes for face recognition in video [R]. Ottawa: Department of National Defence(Canada), 2014.
[11] Lei Y, Bennamoun M, Hayat M, et al. An efficient 3D face recognition approach using local geometrical signatures [J]. Pattern Recognition, 2014, 47(2): 509-524.
[12] Face Analysisl Tencent Youtu [EB/OL]. [2015-10-11]. http://bestimage.qq.com/faceanalyze.html.
/
| 〈 |
|
〉 |
