Journal of Shanghai University >
Group-based vintage film inpainting using robust principal component analysis
Received date: 2017-03-17
Online published: 2017-06-30
In this paper, a new group-based method using robust principal component analysis (RPCA) is proposed to deal with multiple categories of damage in vintage film sequences. Pre-processing of the video sequence is achieved by shot segmentation and flicker elimination. In a framework of multi-resolution pyramid, an observation matrix is constructed on the coarsest level by space-time domain grouping. After performing RPCA transform based on the alternating linear method in sequence, locations of large area damage are obtained based on inter-frame error information. An initial inpainting result sequence, a break mask sequence, and a nearest neighbor offset matrix set using an upsampling method are constructed. The original sequence is then modified. By repeating the space-time grouping RPCA transform, inpainting of the vintage film sequence is realized. Experimental results show that the method can simultaneously repair different damages in the screen with good performance.
YU Bing1,2, DING Youdong1,2, DONG Sun1,2, HUANG Xi1,2 . Group-based vintage film inpainting using robust principal component analysis[J]. Journal of Shanghai University, 2017 , 23(3) : 315 -323 . DOI: 10.12066/j.issn.1007-2861.1923
[1] Simone C, Tunc O A, Nikolce S, et al. Advanced tools and framework for historical film restoration [J]. Journal of Electronic Imaging, 2017, 26(1): 11-21.
[2] Sadhar S I, Rajagopalan A N. Image estimation in film-grain noise [J]. IEEE Signal Processing Letters, 2005, 12(3): 238-241.
[3] Gullu M K, Urhan O, Erturk S. Scratch detection via temporal coherency analysis and removal using edge priority based interpolation [C]// IEEE International Symposium on Circuits
and Systems. 2006: 1-4.
[4] Ren J, Vlachos T. Efficient detection of temporally impulsive dirt impairments in archived films [J]. Signal Processing, 2007, 87(3): 541-551.
[5] Ahmed M A, Pitie F, Kokaram A C. Extraction of non-binary blotch mattes [C]// IEEE International Conference on Image Processing. 2009: 2757-2760.
[6] Elgharib M A, Pitie F, Kokaram A. Blotch and scratch removal in archived film using a semi-transparent corruption model and a ground-truth generation technique [J]. Current Opinion
in Hematology, 2013, 2013(11): 1-20.
[7] Bouwmans T, Zahzah E H. Robust PCA via principal component pursuit: a review for a comparative evaluation in video surveillance [J]. Computer Vision & Image Understanding,
2014, 122(4): 22-34.
[8] Wu K K,Wang L, Soong F K, et al. A sparse and low-rank approach to efficient face alignment for photo-real talking head synthesis [C]//IEEE International Conference on Acoustics. 2011: 1397-1400.
[9] Wu L, Ganesh A, Shi B, et al. Robust photometric stereo via low-rank matrix completion and recovery [C]//Computer Vision—ACCV 2010. 2010: 703-717.
[10] Dabov K, Foi A, Egiazarian K. Video denoising by sparse 3D transform-domain collaborative filtering [C]//Signal Processing Conference. 2007: 145-149.
[11] Ji H, Huang S, Shen Z, et al. Robust video restoration by joint sparse and low rank matrix approximation [J]. SIAM Journal on Imaging Sciences, 2011, 4(4): 1122-1142.
[12] Wright J, Ganesh A, Rao S, et al. Robust principal component analysis: exact recovery of corrupted low-rank matrices via convex optimization [C]// 23rd Annual Conference on Neural
Information Processing Systems. 2009: 20-56.
[13] Adjeroh D A, Lee M C. Scene-adaptive transform domain video partitioning [J]. IEEE Transactions on Multimedia, 2004, 6(1): 58-69.
[14] Hacohen Y, Shechtman E, Dan B G, et al. Non-rigid dense correspondence with applications for image enhancement [J]. ACM Transactions on Graphics, 2011, 30(4): 70-79.
[15] Barnes C, Shechtman E, Finkelstein A, et al. PatchMatch: a randomized correspondence algorithm for structural image editing [J]. ACM Transactions on Graphics, 2009, 28(3): 24-33.
[16] Nesterov Y. Smooth minimization of non-smooth functions [J]. Mathematical Programming, 2005, 103(1): 127-152.
[17] Goldfarb D, Ma S, Scheinberg K. Fast alternating linearization methods for minimizing the sum of two convex functions [J]. Mathematical Programming, 2013, 141(1): 349-382.
/
| 〈 |
|
〉 |
