基于人脸识别的影视剧镜头自动标注及重剪系统

doi:10.12066/j.issn.1007-2861.1713

上海大学学报(自然科学版) ›› 2017, Vol. 23 ›› Issue (3): 353-363.doi: 10.12066/j.issn.1007-2861.1713

基于人脸识别的影视剧镜头自动标注及重剪系统

郎玥¹, 周霁婷¹, 梁小龙², 张文俊¹

1. 上海大学上海电影学院, 上海 200072; 2. 腾讯公司优图项目组, 上海 200030

收稿日期:2015-11-23 出版日期:2017-06-30 发布日期:2017-06-30
通讯作者: 周霁婷(1980—), 女, 博士, 研究方向为数字多媒体通信等. E-mail: zjting@shu.edu.cn
作者简介:周霁婷(1980—), 女, 博士, 研究方向为数字多媒体通信等. E-mail: zjting@shu.edu.cn
基金资助:
国家自然科学基金资助项目(61303093); 上海市教委科研创新基金资助项目(14YZ023)

Automatic annotation for film and Television drama shots and recut system based on face identification

LANG Yue¹, ZHOU Jiting¹, LIANG Xiaolong², ZHANG Wenjun¹

1. Shanghai Film Academy, Shanghai University, Shanghai 200072, China;
2. Tencent Inc., Tencent-BestImage, Shanghai 200030, China

Received:2015-11-23 Online:2017-06-30 Published:2017-06-30

摘要/Abstract

摘要：

利用基于深度学习的人脸识别技术, 建立了一种基于人脸识别的影视剧镜头自动标注及重剪系统, 用于实现影视剧重编辑过程中对镜头片段更好地管理、查找和重剪. 先对输入的影视剧视频进行镜头检测和分割, 获得并建立分镜参数. 在此基础上, 对镜头中出现的所有人脸进行检测和切割, 并采用预先训练好的包含350多位明星特征的模型库予以身份识别, 聚类后实现镜头的演员标注. 该系统也可依据指定演员对影视剧进行搜索, 并将其中所有包含该演员的片段自动重剪在一起. 实验结果表明, 该系统镜头分割模块的平均召回率达到95%以上, 对45°以内的人脸识别率达到92.45%, 且具有良好的鲁棒性.

关键词: 镜头分割, 人脸识别, 深度学习, 人脸检测

Abstract:

This paper proposes an automatic editing system named Star Cut based on face recognition using deep learning and a video shot detection technique. The purpose is to establish a system for management, retrieval, and automatic recut of film and TV shots. First, the system with over 350 faces of pop stars and actors using a U-face model is trained to learn facial features. The system uses the change rate of edges to detect shot edge. After shot segmentation, the system uses the pre-trained face models to identify faces in the input film or TV drama shot by shot. Users can either choose to recognize all figures in these shots or just choose selected one to recut all the shots containing him/her together automatically. The recall rate of shot segmentation is above 95%, and the recognition rate of faces in an shooting angle of 45° is 92.45%. Test results show that the proposed system has good robustness.

Key words: deep learning, face identification, shot segmentation, face detection

郎玥1, 周霁婷1, 梁小龙2, 张文俊1. 基于人脸识别的影视剧镜头自动标注及重剪系统[J]. 上海大学学报(自然科学版), 2017, 23(3): 353-363.

LANG Yue1, ZHOU Jiting1, LIANG Xiaolong2, ZHANG Wenjun1. Automatic annotation for film and Television drama shots and recut system based on face identification[J]. Journal of Shanghai University（Natural Science Edition）, 2017, 23(3): 353-363.

参考文献

[1] Zhao W, Chellappa R, Phillips P J, et al. Face recognition: a literature survey [J]. ACM Computing Surveys, 2003, 35(4): 399-458.
[2] Bhele S G, Mankar V H. A review paper on face recognition techniques [J]. International Journal of Advanced Research in Computer Engineering & Technology, 2012, 1(8): 339-346.
[3] Taigman Y, Yang M, Ranzato M A, et al. Deepface: closing the gap to human-level performance in face verification [C]//2014 IEEE Conference on Computer Vision and Pattern Recognition(CVPR). 2014: 1701-1708.
[4] Sun Y, Wang X, Tang X. Deeply learned face representations are sparse, selective, and robust [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.
2015: 2892-2900.
[5] Schroff F, Kalenichenko D, Philbin J. Facenet: a unified embedding for face recognition and clustering [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern
Recognition. 2015: 815-823.
[6] Zhou E, Cao Z, Yin Q. Naive-deep face recognition: touching the limit of LFW benchmark or not? [EB/OL]. (2015-02-20)[2015-11-23]. http://arxiv.org/pdf/1501.04690vl.pdf.
[7] 孟一飞, 袁雪, 魏学业, 等. 基于样本扩展和线性子空间特征提取的单样本人脸识别[J]. 应用科学学报, 2013, 31(5): 488-494.
[8] 严严, 章毓晋. 基于视频的人脸识别研究进展[J]. 计算机学报, 2009, 32(5): 878-886.
[9] Gorodnichy D O, Granger E, Radtke P, et al. Survey of commercial technologies for face recognition in video [R]. Ottawa: Departmant of National Defence(Canada), 2014.
[10] Granger E, Radtke P, Gorodnichy D, et al. Survey of academic research and prototypes for face recognition in video [R]. Ottawa: Department of National Defence(Canada), 2014.
[11] Lei Y, Bennamoun M, Hayat M, et al. An efficient 3D face recognition approach using local geometrical signatures [J]. Pattern Recognition, 2014, 47(2): 509-524.
[12] Face Analysisl Tencent Youtu [EB/OL]. [2015-10-11]. http://bestimage.qq.com/faceanalyze.html.

基于人脸识别的影视剧镜头自动标注及重剪系统

Automatic annotation for film and Television drama shots and recut system based on face identification

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 7

编辑推荐

Metrics

本文评价

[1]	胡嘉成, 王向阳, 刘晗. 基于深度学习的连铸坯表面缺陷检测[J]. 上海大学学报(自然科学版), 2019, 25(4): 445-452.
[2]	谢志峰, 叶冠桦, 闫淑萁, 何绍荣, 丁友东. 基于生成对抗网络的HDR图像风格迁移技术[J]. 上海大学学报(自然科学版), 2018, 24(4): 524-534.
[3]	冉鹏, 王灵, 李昕, 刘鹏伟. 改进Softmax分类器的深度卷积神经网络及其在人脸识别中的应用[J]. 上海大学学报(自然科学版), 2018, 24(3): 352-366.
[4]	沈文枫, 张建蕾, 周丁倩, 陈圣波, 邱峰. 大数据时代的车牌汉字识别[J]. 上海大学学报(自然科学版), 2016, 22(1): 88-96.
[5]	丁友东;杜晓凤;李晓强. 基于聚类肤色模型的人脸检测[J]. 上海大学学报(自然科学版), 2007, 13(5): 511-515 .
[6]	黄翠荣;严佩敏;时鹿鸣. 基于方差相似度计算的人脸识别技术[J]. 上海大学学报(自然科学版), 2006, 12(4): 347-350 .
[7]	张婷;朱秋煜;朱军林. 人脸识别考勤系统中有效人脸特征提取[J]. 上海大学学报(自然科学版), 2006, 13(3): 244-248 .