A deep feature fusion method was proposed for low-light images rain streak removal in digital twin applications. First, the dynamic line convolution（DLConv） was created to learn the prior knowledge of rain streaks, enhancing the ability to extract rain streak features to address the artifact problem. Second, by combining prior knowledge with the coordinate attention（CA） mechanism, the U-shaped model was improved to generate deep features of rain streaks, thereby addressing the issue of detail loss. Finally, the brightness information of the images was utilized to further refine and enhance the deep features and obtain rain-free images at the current stage, where the features of the rainfree images were fused between the stages, and multiscale features were transferred through cross-stage feature fusion, achieving rain streak removal for low-light images. Experimental results on the latest low-light rainy image dataset showed that the proposed method achieved peak signal-to-noise ratio（PSNR）、structural similarity（SSIM）、learned perceptual image patch similarity（LPIPS） and natural image quality evaluator（NIQE） values of 38.522 9 dB, 0.974 5, 0.061 9 and 4.207 2, respectively, outperforming existing mainstream methods. This demonstrated the effectiveness of the proposed approach in alleviating artifacts and detail loss, thereby significantly improving the usability of low-light images.
HAN Dazhuan
,
ZHANG Jinyi
,
JIANG Yuxi
. Deep feature fusion for low-light images rain streak removal in digital twin applications[J]. Journal of Shanghai University, 2025
, 31(3)
: 543
-560
.
DOI: 10.12066/j.issn.1007-2861.2666
[1] Zhang Z, Wei Y, Zhang H, et al. Data-driven single image deraining:a comprehensive review and new perspectives[J]. Pattern Recognition, 2023, 143:109740.
[2] Ren D, Zuo W, Hu Q, et al. Progressive image deraining networks:a better and simpler baseline[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019:3937-3946.
[3] 柳长源, 王琪, 毕晓君. 基于多通道多尺度卷积神经网络的单幅图像去雨方法[J]. 电子与信息学报, 2020, 42(9):2285-2292.
[4] 张学锋, 李金晶. 基于双注意力残差循环单幅图像去雨集成网络[J]. 软件学报, 2021, 32(10):3283-3292.
[5] Zamir S W, Arora A, Khan S, et al. Multi-stage progressive image restoration[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021:14821-14831.
[6] Ba Y, Zhang H, Yang E, et al. Not just streaks:towards ground truth for single image deraining[C]//European Conference on Computer Vision Cham. 2022:723-740.
[7] Liang Y, Anwar S, Liu Y. DRT:a lightweight single image deraining recursive transformer[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022:589-598.
[8] Chen X, Li H, Li M, et al. Learning a sparse transformer network for efiective image deraining[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023:5896-5905.
[9] Zhang F, You S, Li Y, et al. Learning rain location prior for nighttime deraining[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023:13148-13157.
[10] 汤红忠, 熊珮全, 王蔚, 等. 基于双阶段特征解耦网络的单幅图像去雨方法[J]. 计算机辅助设计与图形学学报, 2024, 36(2):273-282.
[11] Chen X, Pan J, Dong J. Bidirectional multi-scale implicit neural representations for image deraining[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024:25627-25636.
[12] Dai J, Qi H, Xiong Y, et al. Deformable convolutional networks[C]//Proceedings of the IEEE International Conference on Computer Vision. 2017:764-773.
[13] Qi Y, He Y, Qi X, et al. Dynamic snake convolution based on topological geometric constraints for tubular structure segmentation[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023:6070-6079.
[14] Hu J, Shen L, Sun G. Squeeze-and-excitation net-works[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018:7132-7141.
[15] Woo S, Park J, Lee J Y, et al. CBAM:convolutional block attention module[C]//Proceedings of the European Conference on Computer Vision. 2018:3-19.
[16] Hou Q, Zhou D, Feng J. Coordinate attention for ecient mobile network design[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021:13713-13722.
[17] Jiang K, Wang Z, Yi P, et al. Multi-scale progressive fusion network for single image deraining[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020:8346-8355.
[18] Yang W, Tan R T, Feng J, et al. Deep joint rain detection and removal from a single image[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017:1357-1366.
