To fully exploit the inherent correlation between light field epipolar plane images（EPI） and strengthen the effective capture of spatial information, this study proposed a dense light field decoupling reconstruction method based on multiscale EPI information fusion. This method utilized the spatial and epipolar plane dimensions to enable a better capture of the angular correlations between subaperture views. By decoupling and fusing multiple types of information, the accuracy and effectiveness of light field reconstruction could be enhanced. First, based on four-dimensional light field data, additional dense spatial dimensions were introduced to improve the generalization capability of the network and enhanced its understanding of local structures and texture details in images. Second, to better complement and enhance the mutual information among epipolar planes, an epipolar plane fusion module was designed along with a novel multiscale convolutional attention mechanism to integrate feature information. This attention mechanism effectively captured angular correlations through multiscale feature extraction and a global attention mechanism, thereby enhancing the expression of critical features while suppressing redundant content. Finally, experiments conducted on light field datasets, such as HCInew, HCIold,and Stanford, demonstrated that the proposed method outperformed existing state-of-theart（SOTA） approaches in terms of evaluation metrics including the peak signal-to-noise ratio（PSNR） and structural similarity（SSIM）. The proposed method achieved superior reconstruction performance in most test scenarios.
[1] Cui Z, Sheng H, Yang D, et al. Light field depth estimation for non-lambertian objects via adaptive cross operator[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2024, 34(2):1199-1211.
[2] Chao W, Wang X, Wang Y, et al. Learning sub-pixel disparity distribution for light field depth estimation[J]. IEEE Transactions on Computational Imaging, 2023, 9:1126-1138.
[3] Yang D, Cui Z, Sheng H, et al. An occlusion and noise-aware stereo framework based on light field imaging for robust disparity estimation[J]. IEEE Transactions on Computers, 2023, 73(3):764-777.
[4] Chen Y, Jiang G, Yu M, et al. Learning zero-shot dense light field reconstruction from heterogeneous imaging[J]. Information Fusion, 2024, 103:102088.
[5] Kwon K H, Erdenebat M U, Kim N, et al. Image quality enhancement of 4D light field microscopy via reference impge propagation-based one-shot learning[J]. Applied Intelligence, 2023, 53(20):23834-23852.
[6] Piao Y, Jiang Y, Zhang M, et al. PANet:patch-aware network for light field salient object detection[J]. IEEE Transactions on Cybernetics, 2021, 53(1):379-391.
[7] Zhang M, Ji W, Piao Y, et al. LFNet:light field fusion network for salient object detection[J]. IEEE Transactions on Image Processing, 2020, 29:6276-6287.
[8] Ma J, Li Z, Cheng J, et al. Light field image super-resolution based on dual learning and deep Fourier channel attention[J]. Optics Letters, 2024, 49(11):2886-2889.
[9] Zhou S, Hu L, Wang Y, et al. AIF-LFNet:all-in-focus light field super-resolution method considering the depth-varying defocus[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2023, 33(8):3976-3988.
[10] Wang Y, Wang L, Wu G, et al. Disentangling light fields for super-resolution and disparity estimation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 45(1):425-443.
[11] Zhang S, Chang S, Lin Y. End-to-end light field spatial super-resolution network using multiple epipolar geometry[J]. IEEE Transactions on Image Processing, 2021, 30:5956-5968.
[12] Liu G, Yue H, Wu J, et al. Efficient light field angular super-resolution with sub-aperture feature learning and macro-pixel upsampling[J]. IEEE Transactions on Multimedia, 2022, 25:6588-6600.
[13] Jin J, Hou J, Yuan H, et al. Learning light field angular super-resolution via a geometry-aware network[C]//Proceedings of the AAAI Conference on Artificial Intelligence. 2020:11141-11148.
[14] Wanner S, Goldluecke B. Variational light field analysis for disparity estimation and superresolution[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 36(3):606-619.
[15] Kalantari N K, Wang T C, Ramamoorthi R. Learning-based view synthesis for light field cameras[J]. ACM Transactions on Graphics, 2016, 35(6):1-10.
[16] Shi J, Jiang X, Guillemot C. Learning fused pixel and feature-based view reconstructions for light fields[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020:2555-2564.
[17] Meng N, Li K, Liu J, et al. Light field view synthesis via aperture disparity and warping confidence map[J]. IEEE Transactions on Image Processing, 2021, 30:3908-3921.
[18] Liu G, Yue H, Li K, et al. Adaptive pixel aggregation for joint spatial and angular superresolution of light field images[J]. Information Fusion, 2024, 104:102183.
[19] Shi L, Hassanieh H, Davis A, et al. Light field reconstruction using sparsity in the continuous Fourier domain[J]. ACM Transactions on Graphics, 2014, 34(1):12-24.
[20] Wu G, Zhao M, Wang L, et al. Light field reconstruction using deep convolutional network on EPI[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017:6319-6327.
[21] Yeung H W F, Hou J, Chen J, et al. Fast light field reconstruction with deep coarse-to-fine modeling of spatial-angular clues[C]//Proceedings of the European Conference on Computer Vision. 2018:137-152.
[22] Wu G, Liu Y, Dai Q, et al. Learning sheared EPI structure for light field reconstruction[J]. IEEE Transactions on Image Processing, 2019, 28(7):3261-3273.
[23] Honauer K, Johannsen O, Kondermann D. A dataset and evaluation methodology for depth estimation on 4D light fields[M]. Berlin:Springer International Publishing, 2017:19-34.
[24] Wanner S, Meister S, Goldluecke B. Datasets and benchmarks for densely sampled 4D light fields[J]. Vision, Modeling & Visualization, 2013, 13:225-226.
[25] Wang Y, Liu F, Wang Z, et al. End-to-end view synthesis for light field imaging with pseudo 4DCNN[C]//Proceedings of the European Conference on Computer Vision. 2018:333-348.
[26] Jin J, Hou J, Chen J, et al. Deep coarse-to-fine dense light field reconstruction with flexible sampling and geometry-aware fusion[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 44(4):1819-1836.
[27] Wu G, Liu Y, Fang L, et al. Light field reconstruction using convolutional network on EPI and extended applications[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 41(7):1681-1694.
[28] Wang Y, Liu F, Zhang K, et al. High-fidelity view synthesis for light field imaging with extended pseudo 4DCNN[J]. IEEE Transactions on Computational Imaging, 2020, 6:830-842.
[29] Zhong L, Zong B, Wang Q, et al. Implicit epipolar geometric function based light field continuous angular representation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023:3463-3472.
[30] Xia P, Lu Y, Zhang S, et al. Revisiting large kernel convolution for light field image angular super-resolution[C]//2024 IEEE International Conference on Multimedia and Expo. 2024:1-6.
