中文

Journal of Shanghai University >

2019 , Vol. 25 >Issue 5: 679 - 691

DOI: https://doi.org/10.12066/j.issn.1007-2861.1962

Research Paper

Video coding for 3D-HEVC based on saliency information and view synthesis prediction

Expand
  • Key Laboratory of Advanced Display and System Applications of Ministry of Education, Shanghai University,Shanghai 200444, China

Received date: 2017-01-12

  Online published: 2019-10-31

Fold

Abstract

To improve rate distortion performance, all standards of video coding emphasize reduction of information redundancy. However influences of human visual system (HVS) perception is often ignored. This paper proposes a video coding method for the latest standard 3D-high efficiency video coding (HEVC) based on human visual features. In this method, a 3D saliency model is first constructed. Different regions are coded according to their saliency information. The original view synthesis prediction (VSP) algorithm is improved to avoid boundary effects between depth blocks. The compressed video with new view-points is then generated using a proper rendering tool. Experimental results show that the proposed method can reduce the BD-rate by as much as 10% while maintaining the subjective quality. Peak signal to noise ratio (PSNR) of the obtained video is raised by 0.1 dB thus improving coding efficiency.

Key words： 3D-high efficiency video coding (HEVC); saliency information; view synthesis prediction; rendering

Cite this article

Fang YU, Ping AN, Xule YAN . Video coding for 3D-HEVC based on saliency information and view synthesis prediction[J]. Journal of Shanghai University, 2019 , 25(5) : 679 -691 . DOI: 10.12066/j.issn.1007-2861.1962

References

[1] Sullivan G J. Ohm J, Han W J , et al. Overview of the high efficiency video coding (HEVC) standard[J]. IEEE Trans Circuits and Systems for Video Technology, 2012,22(12), 1649-1667.
[2] Shen L, Liu Z, Zhang X , et al. An effective CU size decision method for HEVC encoders[J]. IEEE Trans Multimedia, 2013,15(2):465-470.
[3] Lin Y H, Wu J L . A depth information based fast mode decision algorithm for color plus depth-map 3D videos[J]. IEEE Trans Broadcasting, 2011,57(2):542-550.
[4] Sanchez G, Saldanha M, Balota G , et al. A complexity reduction algorithm for depth maps intra prediction on the 3D-HEVC[C]// Proc Visual Communications and Image Processing (VCIP). 2014: 137-140.
[5] Lee J Y, Hocheon W, Dusik P . A fast and efficient multiview depth image coding method based on temporal and inter-view correlations of texture images[J]. IEEE Trans Circuits and Systems for Video Technology, 2011,21(12):1859-1868.
[6] 马然, 史楠迪, 李攀攀 , 等. 3D-HEVC 帧间编码的低复杂度算法[J]. 电视技术, 2015,39(13):7-13.
[7] Li Y M, Liao W H, Huang J M , et al. Saliency based perceptual HEVC[C]// Proc Multimedia and Expo Workshops (ICMEW). 2014: 1-5.
[8] Xu Z R, Yu M, Fang S Q , et al. A new saliency based video coding method with HEVC[C]// Proc Intelligent Systems Research and Mechatronics Engineering (ISRME). 2015: 672-679.
[9] Noor N U M, Karim H A, Arif N A M , et al. Multi-view video plus depth representation with saliency depth video[C]// Proc Multimedia and Broadcasting (APMediaCast). 2015: 1-6.
[10] Yu F, An P, Yang C . Video coding for 3D-HEVC based on saliency information[C]// Proc Real-time Photonic Measurements, Data Management, and Processing II. 2016
[11] Julesz B, Sehumer R A . Early visual perception[J]. Annual Review of Psychology, 1981,32:575-627.
[12] Koch C, Ullman S . Shifts in selection in visual attention: toward the underlying neural circuitry[J]. Human Neurobiology, 1985,4(4):219-227.
[13] Tsotsos J K, Culhane S M, Wai W Y K , et al. Modeling visual attention via selectivetuning[J]. Artificial Intelligence, 1995,78(1):507-545.
[14] Walther D . Interactions of visual attention and object recognition: computational modeling, algorithms, and psychophysics[D]. California: California Institute of Technology, 2006.
[15] Itti L, Koch C . Computational modeling of visual attention[J]. Nature Reviews Neuroscience, 2001,2(3):194-203.
[16] Liu Y, Li Z G, Soh Y C . Region-of-interest based resource allocation for conversational video communication of H.264/AVC[J]. IEEE Trans Circuits and Systems for Video Technology, 2008,18(1):134-139.
[17] Itti L, Koch C, Niebur E . A model of saliency-based visual attention for rapid sceneanalysis[J]. IEEE Trans Pattern Analysis and Machine Intelligence, 1998,20(11):1254-1259.
[18] Sch?lkopf B, Platt J, Hofmann T . Graph-based visual saliency[C]// Proc Advances in Neural Information Processing Systems. 2007: 545-552.
[19] Zhang L, Gu Z Y, Li H Y . SDSP: a novel saliency detection method by combining simplepriors[C]// Proc Image Processing (ICIP). 2013: 171-175.
[20] Nobuyuki O . A threshold selection method from gray-level histograms[J]. IEEE Trans Sys Man Cyber, 1979,9(1):62-66.
[21] Deng X, Xu M, Jiang L , et al. Subjective-driven complexity control approach for HEVC[J]. IEEE Trans Circuits and Systems for Video Technology, 2016,26(1):91-106.
[22] Shen L Q, Zhang Z Y, Liu Z . Adaptive inter-mode decision for HEVC jointly utilizing inter-level and spatiotemporal correlations[J]. IEEE Trans on Circuits and Systems for Video Technology, 2014,24(10):1709-1722.
Options
Outlines

/

〈 〉