收稿日期: 2021-04-02
网络出版日期: 2021-05-28
基金资助
国家自然科学基金资助项目(61671283)
HDR video reconstruction based on attention and feedback mechanism
Received date: 2021-04-02
Online published: 2021-05-28
对基于深度学习的高动态范围(high dynamic range, HDR) 重建进行研究, 提出一种基于注意力和反馈机制的HDR 重建方法. 首先, 将时间上连续、循环曝光的3 张图像作为网络的输入, 通过引入注意力模块生成注意力图像, 对获取的特征进行自适应的加权, 以优化网络的特征提取和减少鬼影现象的出现; 然后, 将反馈机制引入到网络中, 进一步提高特征信息的利用率, 优化网络在特征融合和重建方面的性能; 最后, 在L1 损失函数的基础上, 考虑色彩相似度损失函数和VGG (Visual Geometry Group) 损失函数以增强重建后HDR 图像的色彩表现及高频细节. 实验结果表明, 本方法不仅可获得更好的主观和客观重建质量, 而且优于目前存在的主流算法.
杨英杰, 王永芳, 张涵 . 基于注意力和反馈机制的 HDR 视频重建[J]. 上海大学学报(自然科学版), 2023 , 29(1) : 56 -67 . DOI: 10.12066/j.issn.1007-2861.2307
In the study, we developed a high dynamic range (HDR) reconstruction method based on attention and feedback mechanism. First, three continuous frames with cyclic exposure were captured as the input of the network. The attention image was generated by introducing the attention module, and the acquired features were weighted adaptively to optimise the feature extraction of the network and reduce ghost phenomenon occurrence. Subsequently, the feedback mechanism was introduced into the network to improve the efficiency of feature information further and optimise the network performance in feature fusion and reconstruction. Finally, based on the L1 loss function, the proposed network added colour similarity and VGG loss functions to enhance the colour similarity and reconstructed HDR image details. The experimental results show that the proposed HDR reconstruction method based on attention and feedback mechanism can achieve better subjective and objective reconstruction quality and is superior to the existing mainstream algorithm.
| [1] | 范劲松, 范彦斌, 裴继刚. 高动态范围图像(HDRI) 编码及色调映射技术研究[J]. 图学学报, 2010, 31(1): 124-128. |
| [2] | Nayar S K, Mitsunaga T. High dynamic range imaging: spatially varying pixel exposures[C]// IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2000: 472-479. |
| [3] | Kang S B, Uyttendaele M, Winder S, et al. High dynamic range video[J]. ACM Transactions on Graphics, 2003, 22(3): 319-325. |
| [4] | Mangiat S, Gibson J. High dynamic range video with ghost removal[C]// SPIE Applications of Digital Image Processing ⅩⅩⅩⅢ. 2010, 7798: 307-314. |
| [5] | 周飞燕, 金林鹏, 董军. 卷积神经网络研究综述[J]. 计算机学报, 2017, 40(6): 1229-1251. |
| [6] | Eilertsen G, Krinader J, Denes G, et al. HDR image reconstruction from a single exposure using deep CNNs[J]. ACM Transactions on Graphics, 2017, 36(6): 178. |
| [7] | Wu S, Xu J, Tai Y, et al. Deep high dynamic range imaging with large foreground motions[C]// IEEE European Conference on Computer Vision (ECCV). 2018: 117-132. |
| [8] | Li Z, Yang J, Liu Z, et al. Feedback network for image super-resolution[C]// IEEE CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2019: 3862-3871. |
| [9] | Endo Y, Kanamori Y, Mitani J. Deep reverse tone mapping[J]. ACM Transactions on Graphics, 2017, 36(6): 177. |
| [10] | Marnerides D, Bashford-Rogers T, Hatchett J, et al. ExpandNet: a deep convolutional neural network for high dynamic range expansion from low dynamic range content[J]. Computer Graphics Forum, 2017, 37(2): 37-49. |
| [11] | Lian J, Wang Y F, Wang C. Dual-streams global guided learning for high dynamic range image reconstruction[C]// IEEE Visual Communications and Image Processing (VCIP). 2019: 1-4. |
| [12] | Khan Z, Khanna M, Raman S. FHDR: HDR image reconstruction from a single LDR image using feedback network[C]// IEEE Global Conference on Signal and Information Processing (GlobalSIP). 2019: 1-5. |
| [13] | Yan Q, Gong D, Shi Q. Attention-guided network for ghost-free high dynamic range imaging[C]// IEEE CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2019: 1751-1760. |
| [14] | Kalantari N K, Ramamoorthi R. Deep high dynamic range imaging of dynamic scenes[J]. ACM Transactions on Graphics, 2017, 36(4): 144-. |
| [15] | Mantiuk R, Kim K J, Rempel A G, et al. HDR-VDP-2: a calibrated visual metric for visibility and quality predictions in all luminance conditions[J]. ACM Transactions on Graph, 2011, 30(4): 40. |
| [16] | Sen P, Kalantari K N, Maziaraziar Y, et al. Robust patch-based HDR reconstruction of dynamic scenes[J]. ACM Transactions on Graphics, 2012, 31(6): 203. |
/
| 〈 |
|
〉 |