基于注意力和反馈机制的 HDR 视频重建

doi:10.12066/j.issn.1007-2861.2307

Abstract

Abstract:

In the study, we developed a high dynamic range (HDR) reconstruction method based on attention and feedback mechanism. First, three continuous frames with cyclic exposure were captured as the input of the network. The attention image was generated by introducing the attention module, and the acquired features were weighted adaptively to optimise the feature extraction of the network and reduce ghost phenomenon occurrence. Subsequently, the feedback mechanism was introduced into the network to improve the efficiency of feature information further and optimise the network performance in feature fusion and reconstruction. Finally, based on the L1 loss function, the proposed network added colour similarity and VGG loss functions to enhance the colour similarity and reconstructed HDR image details. The experimental results show that the proposed HDR reconstruction method based on attention and feedback mechanism can achieve better subjective and objective reconstruction quality and is superior to the existing mainstream algorithm.

Key words: high dynamic range (HDR) reconstruction, deep learning, attention mechanism, feedback mechanism, loss function

CLC Number:

TP751.1

YANG Yingjie, WANG Yongfang, ZHANG Han. HDR video reconstruction based on attention and feedback mechanism[J]. Journal of Shanghai University（Natural Science Edition）, 2023, 29(1): 56-67.

Figures/Tables 11

Fig. 1

Fig. 2

Fig. 3

Fig. 4

Table 1

Table 2

Fig. 5

Table 3

Table 4

Table 5

Fig. 6

References 16

[1]	范劲松, 范彦斌, 裴继刚. 高动态范围图像(HDRI) 编码及色调映射技术研究[J]. 图学学报, 2010, 31(1): 124-128.
[2]	Nayar S K, Mitsunaga T. High dynamic range imaging: spatially varying pixel exposures[C]// IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2000: 472-479.
[3]	Kang S B, Uyttendaele M, Winder S, et al. High dynamic range video[J]. ACM Transactions on Graphics, 2003, 22(3): 319-325.
[4]	Mangiat S, Gibson J. High dynamic range video with ghost removal[C]// SPIE Applications of Digital Image Processing ⅩⅩⅩⅢ. 2010, 7798: 307-314.
[5]	周飞燕, 金林鹏, 董军. 卷积神经网络研究综述[J]. 计算机学报, 2017, 40(6): 1229-1251.
[6]	Eilertsen G, Krinader J, Denes G, et al. HDR image reconstruction from a single exposure using deep CNNs[J]. ACM Transactions on Graphics, 2017, 36(6): 178.
[7]	Wu S, Xu J, Tai Y, et al. Deep high dynamic range imaging with large foreground motions[C]// IEEE European Conference on Computer Vision (ECCV). 2018: 117-132.
[8]	Li Z, Yang J, Liu Z, et al. Feedback network for image super-resolution[C]// IEEE CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2019: 3862-3871.
[9]	Endo Y, Kanamori Y, Mitani J. Deep reverse tone mapping[J]. ACM Transactions on Graphics, 2017, 36(6): 177.
[10]	Marnerides D, Bashford-Rogers T, Hatchett J, et al. ExpandNet: a deep convolutional neural network for high dynamic range expansion from low dynamic range content[J]. Computer Graphics Forum, 2017, 37(2): 37-49.
[11]	Lian J, Wang Y F, Wang C. Dual-streams global guided learning for high dynamic range image reconstruction[C]// IEEE Visual Communications and Image Processing (VCIP). 2019: 1-4.
[12]	Khan Z, Khanna M, Raman S. FHDR: HDR image reconstruction from a single LDR image using feedback network[C]// IEEE Global Conference on Signal and Information Processing (GlobalSIP). 2019: 1-5.
[13]	Yan Q, Gong D, Shi Q. Attention-guided network for ghost-free high dynamic range imaging[C]// IEEE CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2019: 1751-1760.
[14]	Kalantari N K, Ramamoorthi R. Deep high dynamic range imaging of dynamic scenes[J]. ACM Transactions on Graphics, 2017, 36(4): 144-.
[15]	Mantiuk R, Kim K J, Rempel A G, et al. HDR-VDP-2: a calibrated visual metric for visibility and quality predictions in all luminance conditions[J]. ACM Transactions on Graph, 2011, 30(4): 40.
[16]	Sen P, Kalantari K N, Maziaraziar Y, et al. Robust patch-based HDR reconstruction of dynamic scenes[J]. ACM Transactions on Graphics, 2012, 31(6): 203.

HDR video reconstruction based on attention and feedback mechanism

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 11

References 16

Related Articles 12

Recommended Articles

Metrics

Comments

[1]	LI Zhongyu, SUN Haodong, LI Jiao. Design of real-time basketball referee gesture recognition system based on loss weighting [J]. Journal of Shanghai University（Natural Science Edition）, 2023, 29(1): 68-81.
[2]	ZHU Yonghua, FENG Tianyu, ZHANG Meixian, ZHANG Wenjun. Convolutional speech emotion recognition network based on incremental method [J]. Journal of Shanghai University（Natural Science Edition）, 2023, 29(1): 24-40.
[3]	WANG Rongrong, XU Shugong, HUANG Jianbo. Image matting based on deep learning [J]. Journal of Shanghai University（Natural Science Edition）, 2022, 28(2): 261-269.
[4]	LI Chengfan, ZHAO Junjuan. Improved approach to detect small sample target based on remote sensing image [J]. Journal of Shanghai University（Natural Science Edition）, 2022, 28(2): 314-323.
[5]	XU Genglin, RAN Feng, DENG Liang, SHI Huakang, GUO Aiying. Application of lightweight neural network and Hash tracking algorithm in embedded face capture system [J]. Journal of Shanghai University（Natural Science Edition）, 2021, 27(6): 1018-1028.
[6]	XING Yixue, ZHU Yonghua, GAO Haiyan, ZHOU Jin, ZHANG Ke. Distant supervision for relation extraction via attention CNNs [J]. Journal of Shanghai University（Natural Science Edition）, 2021, 27(5): 983-992.
[7]	SHAO Huixiang, ZENG Dan. Classification and recognition of underwater small targets based on improved YOLOv3 algorithm [J]. Journal of Shanghai University（Natural Science Edition）, 2021, 27(3): 481-491.
[8]	CHEN Yu, DING Youdong, YU Bing, XU Min. Video colourisation based on voxel flow [J]. Journal of Shanghai University（Natural Science Edition）, 2021, 27(1): 18-27.
[9]	Jiacheng HU, Xiangyang WANG, Han LIU. Defect detection of continuous casting slabs based on deep learning [J]. Journal of Shanghai University（Natural Science Edition）, 2019, 25(4): 445-452.
[10]	XIE Zhifeng, YE Guanhua, YAN Shuqi, HE Shaorong, DING Youdong. HDR image style transfer technique based on generative adversarial networks [J]. Journal of Shanghai University（Natural Science Edition）, 2018, 24(4): 524-534.
[11]	LANG Yue1, ZHOU Jiting1, LIANG Xiaolong2, ZHANG Wenjun1. Automatic annotation for film and Television drama shots and recut system based on face identification [J]. Journal of Shanghai University（Natural Science Edition）, 2017, 23(3): 353-363.
[12]	SHEN Wenfeng, ZHANG Jianlei, ZHOU Dingqian, CHEN Shengbo, QIU Feng. Recognition of Chinese characters on license plates based on big data [J]. Journal of Shanghai University（Natural Science Edition）, 2016, 22(1): 88-96.