上海大学学报(自然科学版) ›› 2025, Vol. 31 ›› Issue (3): 543-560.doi: 10.12066/j.issn.1007-2861.2666

• 通信工程 • 上一篇    下一篇

面向数字孪生的深层特征融合低照度图像去雨线

韩大专1,2, 张金艺1,2, 姜玉稀3   

  1. 1. 上海大学 特种光纤与光接入网重点实验室, 上海 200444;
    2. 上海大学 特种光纤与先进通信国际合作联合实验室, 上海 200444;
    3. 上海三思电子工程有限公司, 上海 201100
  • 收稿日期:2024-12-17 出版日期:2025-06-30 发布日期:2025-07-22
  • 通讯作者: 张金艺(1965-),男,研究员,博士生导师,博士,研究方向为伴随性机器人、计算机视觉. E-mail:zhangjinyi@shu.edu.cn
  • 基金资助:
    高等学校学科创新引智计划(111计划)资助项目(D20031);上海市闵行区重大产业技术攻关计划资助项目(2022MH-ZD19)

Deep feature fusion for low-light images rain streak removal in digital twin applications

HAN Dazhuan1,2, ZHANG Jinyi1,2, JIANG Yuxi3   

  1. 1. Key laboratory of Specialty Fiber Optics and Optical Access Networks, Shanghai University, Shanghai 200444, China;
    2. Joint International Research Laboratory of Specialty Fiber Optics and Advanced Communication, Shanghai University, Shanghai 200444, China;
    3. Shanghai Sansi Institute for System Integration, Shanghai 201100, China
  • Received:2024-12-17 Online:2025-06-30 Published:2025-07-22

摘要: 提出了一种面向数字孪生的深层特征融合低照度图像去雨线方法.首先,创建动态线性卷积(dynamic line convolution,DLConv),学习雨线的先验知识,提高对雨线的特征提取能力,解决伪影问题.其次,结合先验知识和坐标注意力(coordinate attention,CA)机制,改进U形模型,生成雨线的深层特征,解决细节丢失问题.最后,利用图像的亮度信息进一步细化加强深层特征,得到每一级的无雨图像.在此基础上,各级之间融合无雨图像的特征,并通过跨级特征融合(cross-stage feature fusion,CSFF)传递多尺度特征,实现低照度图像去雨线.在最新的夜间雨天图像数据集上进行实验,结果表明,本方法在峰值信噪比(peak signal-to-noise ratio,PSNR)、结构相似性(structural similarity,SSIM)、学习感知图像块相似度(learned perceptual image patch similarity,LPIPS)和自然图像质量评估器(natural image quality evaluator,NIQE)指标上分别达到了38.522 9 dB、0.974 5、0.061 9和4.207 2,优于现有的主流方法,证明了本方法能有效改善伪影和细节丢失问题,大大提升了低照度图像的可用性.

关键词: 数字孪生, 低照度图像, 图像去雨线, 深层特征, 坐标注意力机制

Abstract: A deep feature fusion method was proposed for low-light images rain streak removal in digital twin applications. First, the dynamic line convolution(DLConv) was created to learn the prior knowledge of rain streaks, enhancing the ability to extract rain streak features to address the artifact problem. Second, by combining prior knowledge with the coordinate attention(CA) mechanism, the U-shaped model was improved to generate deep features of rain streaks, thereby addressing the issue of detail loss. Finally, the brightness information of the images was utilized to further refine and enhance the deep features and obtain rain-free images at the current stage, where the features of the rainfree images were fused between the stages, and multiscale features were transferred through cross-stage feature fusion, achieving rain streak removal for low-light images. Experimental results on the latest low-light rainy image dataset showed that the proposed method achieved peak signal-to-noise ratio(PSNR)、structural similarity(SSIM)、learned perceptual image patch similarity(LPIPS) and natural image quality evaluator(NIQE) values of 38.522 9 dB, 0.974 5, 0.061 9 and 4.207 2, respectively, outperforming existing mainstream methods. This demonstrated the effectiveness of the proposed approach in alleviating artifacts and detail loss, thereby significantly improving the usability of low-light images.

Key words: digital twin, low-light image, image rain streak removal, deep feature, coordinate attention(CA) mechanism

中图分类号: