基于注意力和反馈机制的 HDR 视频重建

doi:10.12066/j.issn.1007-2861.2307

Abstract

Abstract:

In the study, we developed a high dynamic range (HDR) reconstruction method based on attention and feedback mechanism. First, three continuous frames with cyclic exposure were captured as the input of the network. The attention image was generated by introducing the attention module, and the acquired features were weighted adaptively to optimise the feature extraction of the network and reduce ghost phenomenon occurrence. Subsequently, the feedback mechanism was introduced into the network to improve the efficiency of feature information further and optimise the network performance in feature fusion and reconstruction. Finally, based on the L1 loss function, the proposed network added colour similarity and VGG loss functions to enhance the colour similarity and reconstructed HDR image details. The experimental results show that the proposed HDR reconstruction method based on attention and feedback mechanism can achieve better subjective and objective reconstruction quality and is superior to the existing mainstream algorithm.

Key words: high dynamic range (HDR) reconstruction, deep learning, attention mechanism, feedback mechanism, loss function

CLC Number:

TP751.1

YANG Yingjie, WANG Yongfang, ZHANG Han. HDR video reconstruction based on attention and feedback mechanism[J]. Journal of Shanghai University（Natural Science Edition）, 2023, 29(1): 56-67.

Add to citation manager EndNote|Reference Manager|ProCite|BibTeX|RefWorks

URL: https://www.journal.shu.edu.cn/EN/10.12066/j.issn.1007-2861.2307

https://www.journal.shu.edu.cn/EN/Y2023/V29/I1/56

Figures/Tables 11

Fig. 1

Fig. 2

Fig. 3

Fig. 4

Table 1

Table 2

Fig. 5

Table 3

Table 4

Table 5

Fig. 6

[1]	范劲松, 范彦斌, 裴继刚. 高动态范围图像(HDRI) 编码及色调映射技术研究[J]. 图学学报, 2010, 31(1): 124-128.
[2]	Nayar S K, Mitsunaga T. High dynamic range imaging: spatially varying pixel exposures[C]// IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2000: 472-479.
[3]	Kang S B, Uyttendaele M, Winder S, et al. High dynamic range video[J]. ACM Transactions on Graphics, 2003, 22(3): 319-325.
[4]	Mangiat S, Gibson J. High dynamic range video with ghost removal[C]// SPIE Applications of Digital Image Processing ⅩⅩⅩⅢ. 2010, 7798: 307-314.
[5]	周飞燕, 金林鹏, 董军. 卷积神经网络研究综述[J]. 计算机学报, 2017, 40(6): 1229-1251.
[6]	Eilertsen G, Krinader J, Denes G, et al. HDR image reconstruction from a single exposure using deep CNNs[J]. ACM Transactions on Graphics, 2017, 36(6): 178.
[7]	Wu S, Xu J, Tai Y, et al. Deep high dynamic range imaging with large foreground motions[C]// IEEE European Conference on Computer Vision (ECCV). 2018: 117-132.
[8]	Li Z, Yang J, Liu Z, et al. Feedback network for image super-resolution[C]// IEEE CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2019: 3862-3871.
[9]	Endo Y, Kanamori Y, Mitani J. Deep reverse tone mapping[J]. ACM Transactions on Graphics, 2017, 36(6): 177.
[10]	Marnerides D, Bashford-Rogers T, Hatchett J, et al. ExpandNet: a deep convolutional neural network for high dynamic range expansion from low dynamic range content[J]. Computer Graphics Forum, 2017, 37(2): 37-49.
[11]	Lian J, Wang Y F, Wang C. Dual-streams global guided learning for high dynamic range image reconstruction[C]// IEEE Visual Communications and Image Processing (VCIP). 2019: 1-4.
[12]	Khan Z, Khanna M, Raman S. FHDR: HDR image reconstruction from a single LDR image using feedback network[C]// IEEE Global Conference on Signal and Information Processing (GlobalSIP). 2019: 1-5.
[13]	Yan Q, Gong D, Shi Q. Attention-guided network for ghost-free high dynamic range imaging[C]// IEEE CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2019: 1751-1760.
[14]	Kalantari N K, Ramamoorthi R. Deep high dynamic range imaging of dynamic scenes[J]. ACM Transactions on Graphics, 2017, 36(4): 144-.
[15]	Mantiuk R, Kim K J, Rempel A G, et al. HDR-VDP-2: a calibrated visual metric for visibility and quality predictions in all luminance conditions[J]. ACM Transactions on Graph, 2011, 30(4): 40.
[16]	Sen P, Kalantari K N, Maziaraziar Y, et al. Robust patch-based HDR reconstruction of dynamic scenes[J]. ACM Transactions on Graphics, 2012, 31(6): 203.

HDR video reconstruction based on attention and feedback mechanism

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 11

Related Articles 15

Recommended Articles

Metrics

Comments

[1]	ZHAO Yaning, YAN Limin. Improved model for robust image watermarking based on generative adversarial networks [J]. Journal of Shanghai University（Natural Science Edition）, 2026, 32(2): 226-239.
[2]	CHU Qingyuan, CHU Haijian. A deep learning-based non-destructive testing method by combining ultrasonic phased array sector scanning and total focusing method [J]. Journal of Shanghai University（Natural Science Edition）, 2026, 32(2): 240-250.
[3]	ZHU Shuo, WANG Yongfang, LI Zixuan, CHEN Wei. Real-time reflective vest detection in operational environments based on YOLO [J]. Journal of Shanghai University（Natural Science Edition）, 2025, 31(4): 691-703.
[4]	FAN Shenwei, LI Guoping, WANG Guozhong. Lightweight image compression algorithm based on deep learning [J]. Journal of Shanghai University（Natural Science Edition）, 2024, 30(3): 466-475.
[5]	ZHANG Heng, SHU Zhan. Parametric identiﬁcation method for fabricated components based on point cloud deep learning [J]. Journal of Shanghai University（Natural Science Edition）, 2023, 29(3): 502-.
[6]	LI Zhongyu, SUN Haodong, LI Jiao. Design of real-time basketball referee gesture recognition system based on loss weighting [J]. Journal of Shanghai University（Natural Science Edition）, 2023, 29(1): 68-81.
[7]	ZHU Yonghua, FENG Tianyu, ZHANG Meixian, ZHANG Wenjun. Convolutional speech emotion recognition network based on incremental method [J]. Journal of Shanghai University（Natural Science Edition）, 2023, 29(1): 24-40.
[8]	LI Chengfan, ZHAO Junjuan. Improved approach to detect small sample target based on remote sensing image [J]. Journal of Shanghai University（Natural Science Edition）, 2022, 28(2): 314-323.
[9]	WANG Rongrong, XU Shugong, HUANG Jianbo. Image matting based on deep learning [J]. Journal of Shanghai University（Natural Science Edition）, 2022, 28(2): 261-269.
[10]	XU Genglin, RAN Feng, DENG Liang, SHI Huakang, GUO Aiying. Application of lightweight neural network and Hash tracking algorithm in embedded face capture system [J]. Journal of Shanghai University（Natural Science Edition）, 2021, 27(6): 1018-1028.
[11]	XING Yixue, ZHU Yonghua, GAO Haiyan, ZHOU Jin, ZHANG Ke. Distant supervision for relation extraction via attention CNNs [J]. Journal of Shanghai University（Natural Science Edition）, 2021, 27(5): 983-992.
[12]	SHAO Huixiang, ZENG Dan. Classification and recognition of underwater small targets based on improved YOLOv3 algorithm [J]. Journal of Shanghai University（Natural Science Edition）, 2021, 27(3): 481-491.
[13]	CHEN Yu, DING Youdong, YU Bing, XU Min. Video colourisation based on voxel flow [J]. Journal of Shanghai University（Natural Science Edition）, 2021, 27(1): 18-27.
[14]	Jiacheng HU, Xiangyang WANG, Han LIU. Defect detection of continuous casting slabs based on deep learning [J]. Journal of Shanghai University（Natural Science Edition）, 2019, 25(4): 445-452.
[15]	XIE Zhifeng, YE Guanhua, YAN Shuqi, HE Shaorong, DING Youdong. HDR image style transfer technique based on generative adversarial networks [J]. Journal of Shanghai University（Natural Science Edition）, 2018, 24(4): 524-534.