上海大学学报(自然科学版) ›› 2021, Vol. 27 ›› Issue (1): 18-27.doi: 10.12066/j.issn.1007-2861.2119
收稿日期:
2019-01-13
出版日期:
2021-02-28
发布日期:
2021-02-28
通讯作者:
丁友东
E-mail:ydding@shu.edu.cn
作者简介:
丁友东(1967---), 男, 教授, 博士生导师, 博士, 研究方向为计算机图形学、数字影视技术. E-mail: ydding@shu.edu.cn基金资助:
CHEN Yu1, DING Youdong1,2(), YU Bing1,2, XU Min1,2
Received:
2019-01-13
Online:
2021-02-28
Published:
2021-02-28
Contact:
DING Youdong
E-mail:ydding@shu.edu.cn
摘要:
针对利用传统光流传递关键帧颜色信息的视频彩色化方法计 算耗时问题, 以及全局传递颜色的视频彩色化方法导致欠饱和度问题, 提出基于像素流的视频彩色化方法. 首先, 将参考帧与目标帧转换到 Lab 颜色空间中, 利用其亮度通道通过一个深度学习网络得到像素流, 该像素流中的数值指示了目标帧的颜色在参考帧中的位置; 然后, 利用该像素流对参考帧颜色通道进行双线性插值得到目标帧颜色通道; 最后, 将得到的颜色通道与目标帧亮度通道组合得到最终的彩色化图像. 实验结果表明: 该方法得到的彩色化图像能够保持参考帧颜色的饱和度且边缘颜色清晰, 具有更高的峰值信噪比; 运算速度较基于传统光流彩色化方法大大提高.
中图分类号:
陈钰, 丁友东, 于冰, 徐敏. 基于像素流的视频彩色化[J]. 上海大学学报(自然科学版), 2021, 27(1): 18-27.
CHEN Yu, DING Youdong, YU Bing, XU Min. Video colourisation based on voxel flow[J]. Journal of Shanghai University(Natural Science Edition), 2021, 27(1): 18-27.
[1] |
Levin A, Lischinski D, Weiss Y. Colorization using optimization[J]. ACM Transactions on Graphics, 2004,23(3):689-694.
doi: 10.1145/1015706.1015780 |
[2] | Lucas B D, Kanade T. An iterative image registration technique with an application to stereo vision[C] // 7th International Joint Conference on Artificial Intelligence (IJCAI). 1981: 674-679. |
[3] | Irony R, Cohen-Or D, Lischinski D. Colorization by example[C] // Proceedings of the 16th Eruographics Conference on Rendering Techniques. 2005: 201-210. |
[4] | Zhen Z, Yan G, Li Z M. An automatic image, video colorization algorithm based on pattern continuity[C] // International Conference on Audio, Language, Image Processing. 2012: 531-536. |
[5] | Lang M, Wang O, Aydin T, et al. Practical temporal consistency for image-based graphics applications[J]. ACM Transactions on Graphics, 2012,31:34-42. |
[6] |
Yatziv L, Sapiro G. Fast image and video colorization using chrominance blending[J]. IEEE Transactions on Image Processing, 2006,15(5):1120-1129.
doi: 10.1109/tip.2005.864231 pmid: 16671293 |
[7] |
Sheng B, Sun H, Chen S, et al. Colorization using the rotation-invariant feature space[J]. IEEE Computer Graphics Applications, 2011,31(2):24-35.
doi: 10.1109/MCG.2011.18 pmid: 24808026 |
[8] |
Manjunath B S, Ma W Y. Texture features for browsing, retrieval of image data[J]. IEEE Trans Pattern Anal Machine Intelligence, 1996,18(8):837-842
doi: 10.1109/34.531803 |
[9] | Heu J H, Hyun D Y, Kim C S, et al. Image, video colorization based on prioritized source propagation[C] // IEEE International Conference on Image Processing. 2009: 465-468. |
[10] |
Paul S, Bhattacharya S, Gupta S. Spatiotemporal colorization of video using 3D steerable pyramids[J]. IEEE Transactions on Circuits, Systems for Video Technology, 2017,27(8):1605-1619.
doi: 10.1109/TCSVT.2016.2539539 |
[11] |
Welsh T, Ashikhmin M, Mueller K. Transferring color to greyscale images[J]. ACM Trans Graphics, 2002,21(3):277-280.
doi: 10.1145/566654.566576 |
[12] | Sykora D, Burianek J, Zara J. Unsupervised colorization of black-and-white cartoons[C] // The 3rd International Symposium on Non-photorealistic Animation, Rendering. 2004: 121-127. |
[13] | Pan Z, Dong Z, Zhang M. A new algorithm for adding color to video or animation clips[J]. Journal of WSCG, 2004,12(1/2/3):515-519. |
[14] | Teng S, Shen Y, Zhao Z, et al. An interactive framework for video colorization[C] // IEEE International Conference on Image, Graphics. 2013: 89-94. |
[15] | Xia S, Liu J, Fang Y, et al. Robust, automatic video colorization via multiframe reordering refinement[C] // IEEE International Conference on Image Processing. 2016: 4017-4021. |
[16] |
Pierre F, Aujol J F, Bugeau A, et al. Interactive video colorization within a variational framework[J]. SIAM Journal on Imaging Sciences, 2017,10(4):2293-2325.
doi: 10.1137/16M1108625 |
[17] | Jampani V, Gadde R, Gehler P V. Video propagation networks[C] // IEEE Conference on Computer Vision, Pattern Recognition. 2017: 3154-3164. |
[18] | Liu S, Zhong G, de Mello S, et al. Switchable temporal propagation network[C] // 15th European Conference on Computer Vision. 2018: 89-104. |
[19] | Meyer S, Cornillère V, Djelouah A, et al. Deep video color propagation [EB/OL]. (2018-08-09) [2019-01-11]. https://arxiv.org/abs/1808.03232. |
[20] | Lefkimmiatis S. Universal denoising networks: a novel CNN architecture for image denoising[C] // IEEE Conference on Computer Vision, Pattern Recognition. 2018: 3204-3213. |
[21] | Yu J, Lin Z, Yang J, et al. Generative image inpainting with contextual attention[C] // IEEE Conference on Computer Vision, Pattern Recognition. 2018: 5505-5514. |
[22] | Chen Y S, Wang Y C, Kao M H, et al. Deep photo enhancer: unpaired learning for image enhancement from photographs with GANs[C] // IEEE Conference on Computer Vision, Pattern Recognition. 2018: 6306-6314. |
[23] | Niklaus S, Mai L, Liu F. Video frame interpolation via adaptive separable convolution[C] // IEEE International Conference on Computer Vision. 2017: 261-270. |
[24] | Liu Z, Yeh R A, Tang X, et al. Video frame synconfproc using deep voxel flow[C] // IEEE International Conference on Computer Vision. 2017: 4473-4481. |
[25] | He K, Zhang X, Ren S, et al. Deep residual learning for image recognition[C] // IEEE Conference on Computer Vision, Pattern Recognition. 2016: 770-778. |
[26] | He K, Zhang X, Ren S, et al. Delving deep into rectifiers: surpassing human-level performance on imagenet classification[C] // IEEE International Conference on Computer Vision. 2015: 1026-1034. |
[27] | Long G, Kneip L, Alvarez J M, et al. Learning image matching by simply watching video[C] // European Conference on Computer Vision. 2016: 434-450. |
[28] | Perazzi F, Pont-Tuset J, McWilliams B, et al. A benchmark dataset, evaluation methodology for video object segmentation[C] // IEEE Conference on Computer Vision, Pattern Recognition. 2016: 724-732. |
[29] | Pont-Tuset J, Perazzi F, Caelles S, et al. The 2017 davis challenge on video object segmentation [EB/OL]. (2018-03-01) [2019-01-11]. https://arxiv.org/abs/1704.00675. |
[30] | Su S, Delbracio M, Wang J, et al. Deep video deblurring for hand-held cameras[C] // IEEE Conference on Computer Vision, Pattern Recognition. 2017: 237-246. |
[31] | Kingma D P, Ba J. Adam: a method for stochastic optimization [EB/OL]. (2017-01-30) [2019-01-11]. https://arxiv.org/abs/1412.6980v9. |
[32] | Soomro K, Zamir A R, Shah M. UCF101: a dataset of 101 human actions classes from videos in the wild [EB/OL]. (2012-12-03) [2019-01-11]. https://arxiv.org/abs/1212.0402. |
[33] | Li Y, Liu M Y, Li X, et al. A closed-form solution to photorealistic image stylization [EB/OL]. (2018-02-19) [2019-01-11]. https://arxiv.org/abs/1802.06474. |
[34] | Zhang R, Zhu J Y, Isola P, et al. Real-time user-guided image colorization with learned deep priors[J]. ACM Transactions on Graphics, 2017,36(4):1-11. |
[1] | 黄嘉宝, 朱永华, 周霁婷, 高文靖. 基于卷积神经网络的多肉植物细粒度图像分类[J]. 上海大学学报(自然科学版), 2020, 26(2): 283-291. |
[2] | 王雷, 陆金桂, 李乐为. 叶片疲劳寿命神经网络近似计算模型数值实验[J]. 上海大学学报(自然科学版), 2019, 25(6): 870-878. |
[3] | 胡嘉成, 王向阳, 刘晗. 基于深度学习的连铸坯表面缺陷检测[J]. 上海大学学报(自然科学版), 2019, 25(4): 445-452. |
[4] | 陈高攀, 徐美华, 王琪, 郭爱英. 一种基于单目视觉的前方车辆检测算法[J]. 上海大学学报(自然科学版), 2019, 25(1): 56-65. |
[5] | 周敬一, 郭燕, 丁友东. 基于深度学习的中文影评情感分析[J]. 上海大学学报(自然科学版), 2018, 24(5): 703-712. |
[6] | 谢志峰, 叶冠桦, 闫淑萁, 何绍荣, 丁友东. 基于生成对抗网络的HDR图像风格迁移技术[J]. 上海大学学报(自然科学版), 2018, 24(4): 524-534. |
[7] | 冉鹏, 王灵, 李昕, 刘鹏伟. 改进Softmax分类器的深度卷积神经网络及其在人脸识别中的应用[J]. 上海大学学报(自然科学版), 2018, 24(3): 352-366. |
[8] | 刘惠彬, 陈强, 吴飞, 赵毅. 基于卷积神经网络的固定群体中目标人物分类[J]. 上海大学学报(自然科学版), 2017, 23(6): 874-. |
[9] | 郎玥1, 周霁婷1, 梁小龙2, 张文俊1. 基于人脸识别的影视剧镜头自动标注及重剪系统[J]. 上海大学学报(自然科学版), 2017, 23(3): 353-363. |
[10] | 刘燕平, 田金鹏, 陈泳. 特征融合的射频指纹识别方法[J]. 上海大学学报(自然科学版), 2017, 23(3): 408-413. |
[11] | 沈文枫, 张建蕾, 周丁倩, 陈圣波, 邱峰. 大数据时代的车牌汉字识别[J]. 上海大学学报(自然科学版), 2016, 22(1): 88-96. |
[12] | 卜峰, 李传江, 陈佳佳, 李欢, 郭伟海. 基于ARM 的肌电假肢手控制器[J]. 上海大学学报(自然科学版), 2014, 20(4): 442-449. |
[13] | 徐秦乐1, 张金艺1,2,3, 徐惠政1, 李若涵2, 张晶晶2. 高精度故障电弧检测多传感器数据融合算法[J]. 上海大学学报(自然科学版), 2014, 20(2): 165-173. |
[14] | 黄青华, 李 琳, 赖士村. 基于 RBF 神经网络的头相关传输函数的个性化建模方法[J]. 上海大学学报(自然科学版), 2014, 20(2): 157-164. |
[15] | 刘军伟,余小清,万旺根,张静,杨薇. 基于改进型BP神经网络的音频多分类[J]. 上海大学学报(自然科学版), 2012, 18(2): 127-131. |
阅读次数 | ||||||
全文 |
|
|||||
摘要 |
|
|||||