Journal of Shanghai University(Natural Science Edition) ›› 2024, Vol. 30 ›› Issue (4): 669-681.doi: 10.12066/j.issn.1007-2861.2546

Previous Articles     Next Articles

Light field image coding based on multi-feature fusion and geometry-aware networks

LIU Faguo1, BAI Xiaoqi2, ZHANG Qian1, WANG Bin1, SI Wen3   

  1. 1. College of Information and Electromechanical Engineering, Shanghai Normal University, Shanghai 201418, China;2. Yangpu District Education Institute, Shanghai 200092, China;3. Shanghai Business School, Shanghai 201400, China
  • Received:2023-09-24 Online:2024-08-30 Published:2024-09-13

Abstract: To address the inherent limitations of light field image coding methods relying on single parallax synthesis, which hinders the recovery of texture details from occluded regions, an innovative light field image coding approach grounded in multi-feature fusion and geometric perception networks is introduced in this research. The primary objective is to further enhance the compression performance of light-field images captured in occluded scenes. The methodology begins by employing sparse sampling of the dense light field, followed by the application of versatile video coding (VVC) to effectively compress the resulting sparse light field. Subsequently, two pivotal branch modules, the parallax estimation module and the spatial angle joint convolution module, were deployed during the decoding process. These branches collectively capture the global geometric attributes of the optical field image and ensure a more comprehensive recovery of the features, particularly from regions characterized by dense textures and occlusions. To exploit the structural information embedded within the fused features originating from these two branches fully, a stack structure featuring bidirectional views was constructed. Furthermore, a refinement network with geometric perception capabilities was used to reconstruct high-quality dense light fields. The experimental results demonstrate the significant advantages of our pro-posed method over current international light-field image coding techniques.

Key words: light ?eld image coding, occlusion region, multi-feature fusion, geometry-aware

CLC Number: