收稿日期: 2017-03-17
网络出版日期: 2017-06-30
基金资助
国家自然科学基金资助项目(61402278); 上海市自然科学基金资助项目(14ZR1415800); 上海大学电影学高峰学科和上海电影特效工程技术研究中心资助项目(16dz2251300); 上海市科委科技攻关资助项目(16511101302)
Group-based vintage film inpainting using robust principal component analysis
Received date: 2017-03-17
Online published: 2017-06-30
以老电影视频为研究对象, 针对序列中存在的多种损伤类别, 提出一种基于分组鲁棒主成分分析(robust principal component analysis, RPCA)的统一修复方法. 采用镜头分割和去闪烁实现对视频序列的预处理. 在多分辨率金字塔框架下, 采用时空域分组的方式在最粗糙层构造观测矩阵, 依次执行基于交替线性法的RPCA 变换后, 根据帧间误差信息得到大面积破损位置; 利用上采样方式构造初步修复结果序列、破损掩模序列以及最近邻偏移矩阵集合, 继而对原始序列进行修改, 重复时空域分组RPCA 变换, 实现对老电影视频序列的修复. 实验结果证明, 该方法能够同时修复画面中的不同损伤, 并取得良好的效果.
于冰1,2, 丁友东1,2, 董荪1,2, 黄曦1,2 . 基于分组鲁棒主成分分析的老电影修复[J]. 上海大学学报(自然科学版), 2017 , 23(3) : 315 -323 . DOI: 10.12066/j.issn.1007-2861.1923
In this paper, a new group-based method using robust principal component analysis (RPCA) is proposed to deal with multiple categories of damage in vintage film sequences. Pre-processing of the video sequence is achieved by shot segmentation and flicker elimination. In a framework of multi-resolution pyramid, an observation matrix is constructed on the coarsest level by space-time domain grouping. After performing RPCA transform based on the alternating linear method in sequence, locations of large area damage are obtained based on inter-frame error information. An initial inpainting result sequence, a break mask sequence, and a nearest neighbor offset matrix set using an upsampling method are constructed. The original sequence is then modified. By repeating the space-time grouping RPCA transform, inpainting of the vintage film sequence is realized. Experimental results show that the method can simultaneously repair different damages in the screen with good performance.
[1] Simone C, Tunc O A, Nikolce S, et al. Advanced tools and framework for historical film restoration [J]. Journal of Electronic Imaging, 2017, 26(1): 11-21.
[2] Sadhar S I, Rajagopalan A N. Image estimation in film-grain noise [J]. IEEE Signal Processing Letters, 2005, 12(3): 238-241.
[3] Gullu M K, Urhan O, Erturk S. Scratch detection via temporal coherency analysis and removal using edge priority based interpolation [C]// IEEE International Symposium on Circuits
and Systems. 2006: 1-4.
[4] Ren J, Vlachos T. Efficient detection of temporally impulsive dirt impairments in archived films [J]. Signal Processing, 2007, 87(3): 541-551.
[5] Ahmed M A, Pitie F, Kokaram A C. Extraction of non-binary blotch mattes [C]// IEEE International Conference on Image Processing. 2009: 2757-2760.
[6] Elgharib M A, Pitie F, Kokaram A. Blotch and scratch removal in archived film using a semi-transparent corruption model and a ground-truth generation technique [J]. Current Opinion
in Hematology, 2013, 2013(11): 1-20.
[7] Bouwmans T, Zahzah E H. Robust PCA via principal component pursuit: a review for a comparative evaluation in video surveillance [J]. Computer Vision & Image Understanding,
2014, 122(4): 22-34.
[8] Wu K K,Wang L, Soong F K, et al. A sparse and low-rank approach to efficient face alignment for photo-real talking head synthesis [C]//IEEE International Conference on Acoustics. 2011: 1397-1400.
[9] Wu L, Ganesh A, Shi B, et al. Robust photometric stereo via low-rank matrix completion and recovery [C]//Computer Vision—ACCV 2010. 2010: 703-717.
[10] Dabov K, Foi A, Egiazarian K. Video denoising by sparse 3D transform-domain collaborative filtering [C]//Signal Processing Conference. 2007: 145-149.
[11] Ji H, Huang S, Shen Z, et al. Robust video restoration by joint sparse and low rank matrix approximation [J]. SIAM Journal on Imaging Sciences, 2011, 4(4): 1122-1142.
[12] Wright J, Ganesh A, Rao S, et al. Robust principal component analysis: exact recovery of corrupted low-rank matrices via convex optimization [C]// 23rd Annual Conference on Neural
Information Processing Systems. 2009: 20-56.
[13] Adjeroh D A, Lee M C. Scene-adaptive transform domain video partitioning [J]. IEEE Transactions on Multimedia, 2004, 6(1): 58-69.
[14] Hacohen Y, Shechtman E, Dan B G, et al. Non-rigid dense correspondence with applications for image enhancement [J]. ACM Transactions on Graphics, 2011, 30(4): 70-79.
[15] Barnes C, Shechtman E, Finkelstein A, et al. PatchMatch: a randomized correspondence algorithm for structural image editing [J]. ACM Transactions on Graphics, 2009, 28(3): 24-33.
[16] Nesterov Y. Smooth minimization of non-smooth functions [J]. Mathematical Programming, 2005, 103(1): 127-152.
[17] Goldfarb D, Ma S, Scheinberg K. Fast alternating linearization methods for minimizing the sum of two convex functions [J]. Mathematical Programming, 2013, 141(1): 349-382.
/
| 〈 |
|
〉 |
