Journal of Shanghai University >
HDR video reconstruction based on attention and feedback mechanism
Received date: 2021-04-02
Online published: 2021-05-28
In the study, we developed a high dynamic range (HDR) reconstruction method based on attention and feedback mechanism. First, three continuous frames with cyclic exposure were captured as the input of the network. The attention image was generated by introducing the attention module, and the acquired features were weighted adaptively to optimise the feature extraction of the network and reduce ghost phenomenon occurrence. Subsequently, the feedback mechanism was introduced into the network to improve the efficiency of feature information further and optimise the network performance in feature fusion and reconstruction. Finally, based on the L1 loss function, the proposed network added colour similarity and VGG loss functions to enhance the colour similarity and reconstructed HDR image details. The experimental results show that the proposed HDR reconstruction method based on attention and feedback mechanism can achieve better subjective and objective reconstruction quality and is superior to the existing mainstream algorithm.
YANG Yingjie, WANG Yongfang, ZHANG Han . HDR video reconstruction based on attention and feedback mechanism[J]. Journal of Shanghai University, 2023 , 29(1) : 56 -67 . DOI: 10.12066/j.issn.1007-2861.2307
| [1] | 范劲松, 范彦斌, 裴继刚. 高动态范围图像(HDRI) 编码及色调映射技术研究[J]. 图学学报, 2010, 31(1): 124-128. |
| [2] | Nayar S K, Mitsunaga T. High dynamic range imaging: spatially varying pixel exposures[C]// IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2000: 472-479. |
| [3] | Kang S B, Uyttendaele M, Winder S, et al. High dynamic range video[J]. ACM Transactions on Graphics, 2003, 22(3): 319-325. |
| [4] | Mangiat S, Gibson J. High dynamic range video with ghost removal[C]// SPIE Applications of Digital Image Processing ⅩⅩⅩⅢ. 2010, 7798: 307-314. |
| [5] | 周飞燕, 金林鹏, 董军. 卷积神经网络研究综述[J]. 计算机学报, 2017, 40(6): 1229-1251. |
| [6] | Eilertsen G, Krinader J, Denes G, et al. HDR image reconstruction from a single exposure using deep CNNs[J]. ACM Transactions on Graphics, 2017, 36(6): 178. |
| [7] | Wu S, Xu J, Tai Y, et al. Deep high dynamic range imaging with large foreground motions[C]// IEEE European Conference on Computer Vision (ECCV). 2018: 117-132. |
| [8] | Li Z, Yang J, Liu Z, et al. Feedback network for image super-resolution[C]// IEEE CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2019: 3862-3871. |
| [9] | Endo Y, Kanamori Y, Mitani J. Deep reverse tone mapping[J]. ACM Transactions on Graphics, 2017, 36(6): 177. |
| [10] | Marnerides D, Bashford-Rogers T, Hatchett J, et al. ExpandNet: a deep convolutional neural network for high dynamic range expansion from low dynamic range content[J]. Computer Graphics Forum, 2017, 37(2): 37-49. |
| [11] | Lian J, Wang Y F, Wang C. Dual-streams global guided learning for high dynamic range image reconstruction[C]// IEEE Visual Communications and Image Processing (VCIP). 2019: 1-4. |
| [12] | Khan Z, Khanna M, Raman S. FHDR: HDR image reconstruction from a single LDR image using feedback network[C]// IEEE Global Conference on Signal and Information Processing (GlobalSIP). 2019: 1-5. |
| [13] | Yan Q, Gong D, Shi Q. Attention-guided network for ghost-free high dynamic range imaging[C]// IEEE CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2019: 1751-1760. |
| [14] | Kalantari N K, Ramamoorthi R. Deep high dynamic range imaging of dynamic scenes[J]. ACM Transactions on Graphics, 2017, 36(4): 144-. |
| [15] | Mantiuk R, Kim K J, Rempel A G, et al. HDR-VDP-2: a calibrated visual metric for visibility and quality predictions in all luminance conditions[J]. ACM Transactions on Graph, 2011, 30(4): 40. |
| [16] | Sen P, Kalantari K N, Maziaraziar Y, et al. Robust patch-based HDR reconstruction of dynamic scenes[J]. ACM Transactions on Graphics, 2012, 31(6): 203. |
/
| 〈 |
|
〉 |