基于分组鲁棒主成分分析的老电影修复

doi:10.12066/j.issn.1007-2861.1923

上海大学学报(自然科学版) ›› 2017, Vol. 23 ›› Issue (3): 315-323.doi: 10.12066/j.issn.1007-2861.1923

• 数字影视技术 • 下一篇

基于分组鲁棒主成分分析的老电影修复

于冰^1,2, 丁友东^1,2, 董荪^1,2, 黄曦^1,2

1. 上海大学上海电影学院, 上海 200072;
2. 上海大学上海电影特效工程技术研究中心, 上海 200072

收稿日期:2017-03-17 出版日期:2017-06-30 发布日期:2017-06-30
通讯作者: 丁友东(1967—), 男, 教授, 博士生导师, 博士, 研究方向为计算机图形学、数字影视动技术. E-mail: ydding@shu.edu.cn
作者简介:丁友东(1967—), 男, 教授, 博士生导师, 博士, 研究方向为计算机图形学、数字影视动技术. E-mail: ydding@shu.edu.cn
基金资助:
国家自然科学基金资助项目(61402278); 上海市自然科学基金资助项目(14ZR1415800); 上海大学电影学高峰学科和上海电影特效工程技术研究中心资助项目(16dz2251300); 上海市科委科技攻关资助项目(16511101302)

Group-based vintage film inpainting using robust principal component analysis

YU Bing^1,2, DING Youdong^1,2, DONG Sun^1,2, HUANG Xi^1,2

1. Shanghai Film Academy, Shanghai University, Shanghai 200072, China;
2. Shanghai Engineering Research Center of Motion Picture Special Effects, Shanghai University, Shanghai 200072, China

Received:2017-03-17 Online:2017-06-30 Published:2017-06-30

摘要/Abstract

摘要：

以老电影视频为研究对象, 针对序列中存在的多种损伤类别, 提出一种基于分组鲁棒主成分分析(robust principal component analysis, RPCA)的统一修复方法. 采用镜头分割和去闪烁实现对视频序列的预处理. 在多分辨率金字塔框架下, 采用时空域分组的方式在最粗糙层构造观测矩阵, 依次执行基于交替线性法的RPCA 变换后, 根据帧间误差信息得到大面积破损位置; 利用上采样方式构造初步修复结果序列、破损掩模序列以及最近邻偏移矩阵集合, 继而对原始序列进行修改, 重复时空域分组RPCA 变换, 实现对老电影视频序列的修复. 实验结果证明, 该方法能够同时修复画面中的不同损伤, 并取得良好的效果.

关键词: 鲁棒主成分分析, 视频修复, 老电影

Abstract:

In this paper, a new group-based method using robust principal component analysis (RPCA) is proposed to deal with multiple categories of damage in vintage film sequences. Pre-processing of the video sequence is achieved by shot segmentation and flicker elimination. In a framework of multi-resolution pyramid, an observation matrix is constructed on the coarsest level by space-time domain grouping. After performing RPCA transform based on the alternating linear method in sequence, locations of large area damage are obtained based on inter-frame error information. An initial inpainting result sequence, a break mask sequence, and a nearest neighbor offset matrix set using an upsampling method are constructed. The original sequence is then modified. By repeating the space-time grouping RPCA transform, inpainting of the vintage film sequence is realized. Experimental results show that the method can simultaneously repair different damages in the screen with good performance.

Key words: robust principal component analysis (RPCA), video inpainting, vintage film

于冰1,2, 丁友东1,2, 董荪1,2, 黄曦1,2. 基于分组鲁棒主成分分析的老电影修复[J]. 上海大学学报(自然科学版), 2017, 23(3): 315-323.

YU Bing1,2, DING Youdong1,2, DONG Sun1,2, HUANG Xi1,2. Group-based vintage film inpainting using robust principal component analysis[J]. Journal of Shanghai University（Natural Science Edition）, 2017, 23(3): 315-323.

参考文献

[1] Simone C, Tunc O A, Nikolce S, et al. Advanced tools and framework for historical film restoration [J]. Journal of Electronic Imaging, 2017, 26(1): 11-21.

[2] Sadhar S I, Rajagopalan A N. Image estimation in film-grain noise [J]. IEEE Signal Processing Letters, 2005, 12(3): 238-241.
[3] Gullu M K, Urhan O, Erturk S. Scratch detection via temporal coherency analysis and removal using edge priority based interpolation [C]// IEEE International Symposium on Circuits
and Systems. 2006: 1-4.
[4] Ren J, Vlachos T. Efficient detection of temporally impulsive dirt impairments in archived films [J]. Signal Processing, 2007, 87(3): 541-551.
[5] Ahmed M A, Pitie F, Kokaram A C. Extraction of non-binary blotch mattes [C]// IEEE International Conference on Image Processing. 2009: 2757-2760.
[6] Elgharib M A, Pitie F, Kokaram A. Blotch and scratch removal in archived film using a semi-transparent corruption model and a ground-truth generation technique [J]. Current Opinion
in Hematology, 2013, 2013(11): 1-20.
[7] Bouwmans T, Zahzah E H. Robust PCA via principal component pursuit: a review for a comparative evaluation in video surveillance [J]. Computer Vision & Image Understanding,
2014, 122(4): 22-34.
[8] Wu K K,Wang L, Soong F K, et al. A sparse and low-rank approach to efficient face alignment for photo-real talking head synthesis [C]//IEEE International Conference on Acoustics. 2011: 1397-1400.
[9] Wu L, Ganesh A, Shi B, et al. Robust photometric stereo via low-rank matrix completion and recovery [C]//Computer Vision—ACCV 2010. 2010: 703-717.
[10] Dabov K, Foi A, Egiazarian K. Video denoising by sparse 3D transform-domain collaborative filtering [C]//Signal Processing Conference. 2007: 145-149.
[11] Ji H, Huang S, Shen Z, et al. Robust video restoration by joint sparse and low rank matrix approximation [J]. SIAM Journal on Imaging Sciences, 2011, 4(4): 1122-1142.
[12] Wright J, Ganesh A, Rao S, et al. Robust principal component analysis: exact recovery of corrupted low-rank matrices via convex optimization [C]// 23rd Annual Conference on Neural
Information Processing Systems. 2009: 20-56.
[13] Adjeroh D A, Lee M C. Scene-adaptive transform domain video partitioning [J]. IEEE Transactions on Multimedia, 2004, 6(1): 58-69.
[14] Hacohen Y, Shechtman E, Dan B G, et al. Non-rigid dense correspondence with applications for image enhancement [J]. ACM Transactions on Graphics, 2011, 30(4): 70-79.
[15] Barnes C, Shechtman E, Finkelstein A, et al. PatchMatch: a randomized correspondence algorithm for structural image editing [J]. ACM Transactions on Graphics, 2009, 28(3): 24-33.
[16] Nesterov Y. Smooth minimization of non-smooth functions [J]. Mathematical Programming, 2005, 103(1): 127-152.
[17] Goldfarb D, Ma S, Scheinberg K. Fast alternating linearization methods for minimizing the sum of two convex functions [J]. Mathematical Programming, 2013, 141(1): 349-382.

基于分组鲁棒主成分分析的老电影修复

Group-based vintage film inpainting using robust principal component analysis

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 2

编辑推荐

Metrics

本文评价

[1]	夏天然, 丁友东, 于冰, 黄曦. 基于子帧缝合的老电影大面积破损修复[J]. 上海大学学报(自然科学版), 2018, 24(4): 503-511.
[2]	赵培栋1，韩军1，闵友钢2，蒋慧钧2. 基于多帧参考的分块视频闪烁消除算法[J]. 上海大学学报(自然科学版), 2010, 16(4): 342-348.