Research Articles

Image matting based on deep learning

Expand
  • 1. Shanghai Film Academy, Shanghai University, Shanghai 200072, China
    2. Shanghai Institute for Advanced Communication and Data Science, Shanghai University, Shanghai 200444, China
    3. Shanghai Engineering Research Center of Motion Picture Special Effects, Shanghai University, Shanghai 200072, China

Received date: 2020-03-13

  Online published: 2020-12-15

Abstract

Image editing technology, which is widely used in the post-production of film and television and in daily life, is based on image matting. In this study, an image matting network based on deep learning which estimates the value of each pixel by inputting the original image and trimap is proposed. Based on the original down- and up-sampling network and to address the problem of slow network convergence caused by the large difference between matting dataset pictures, batch normalisation (BN) is applied after each convolution layer in this study. In the normalisation layer, the input data are normalised to speed up the convergence of the model. This enables the update direction of the parameters to be more consistent with the overall characteristics of the dataset. Because the edge of the object should be carefully considered in the matting task, a deformable convolution layer is used instead of the custom convolution layer. The deformable convolution layer can adaptively learn the shape of the convolution kernel according to different input data, effectively expand the range of the receptive field, and improve the prediction effect in detailed image parts.

Cite this article

WANG Rongrong, XU Shugong, HUANG Jianbo . Image matting based on deep learning[J]. Journal of Shanghai University, 2022 , 28(2) : 261 -269 . DOI: 10.12066/j.issn.1007-2861.2287

References

[1] 孙国星. 全自动抠图技术的研究[D]. 济南: 山东师范大学, 2017.
[2] 栗大智, 孙冰心, 朱少强, 等. 基于 image matting 的旅游相片处理[J]. 福建电脑, 2017, 33(5): 129, 153.
[3] Russakovsky O, Deng J, Su H, et al. ImageNet large scale visual recognition challenge[J]. International Journal of Computer Vision, 2015, 115(3): 211-252.
[4] Porter T K, Duff T D. Compositing digital images[J]. ACM Siggraph Computer Graphics, 1984, 18(3): 253-259.
[5] Wang J, Michael F. Cohen. Optimized Color Sampling for Robust Matting[C]// 2007 IEEE Conference on Computer Vision and Pattern Recognition. 2007.
[6] Chen Q F, Ding Z Y, Tang C K. KNN matting[C]// Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2012.
[7] Levin A, Lischinski D, Weiss Y. A closed form solution to natural image matting[C]// IEEE Computer Society. 2006: 61-68.
[8] Zheng Y J, Kambhamettu C. Learning based digital matting[C]// IEEE 12th International Conference on Computer Vision. 2009.
[9] Cho D, Tai Y W, Kweon I. Natural image matting using deep convolutional neural networks[C]// European Conference on Computer Vision. 2016.
[10] Ning X, Brian P, Scott C, et al. Deep image matting[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017: 2970-2979.
[11] Ioffe S, Szegedy C. Batch normalization: accelerating deep network training by reducing internal covariate shift[C]// International Conference on Machine Learning. 2015.
[12] Dai J F, Qi H Z, Xiong Y W, et al. Deformable convolutional networks[C]// Proceedings of the IEEE International Conference on Computer Version. 2017: 764-773.
[13] Rhemann C, Rother C, Wang J, et al. A perceptually motivated online benchmark for image matting[C]// IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2009). 2009.
Outlines

/