一种车载多模式全景融合拼接算法

doi:10.12066/j.issn.1007-2861.2098

上海大学学报(自然科学版) ›› 2020, Vol. 26 ›› Issue (6): 909-920.doi: 10.12066/j.issn.1007-2861.2098

一种车载多模式全景融合拼接算法

刘畅¹, 徐美华¹(), 郭爱英²

1. 上海大学机电工程与自动化学院, 上海 200444
2. 上海大学新型显示与系统集成教育部重点实验室, 上海 200444

收稿日期:2018-07-13 出版日期:2020-12-31 发布日期:2020-12-29
通讯作者: 徐美华 E-mail:mhxu@shu.edu.cn
作者简介:徐美华(1957—), 女, 教授, 博士生导师, 博士, 研究方向为图像处理、汽车电子与IC设计. E-mail: mhxu@shu.edu.cn
基金资助:
国家自然科学基金资助项目(61376028);国家自然科学基金资助项目(61674100)

Stitching algorithm of multi-mode fusional vehicle panorama

LIU Chang¹, XU Meihua¹(), GUO Aiying²

1. School of Mechatronic Engineering and Automation, Shanghai University, Shanghai 200444, China
2. Key Laboratory of Advanced Display and System Applications, Ministry of Education, Shanghai University, Shanghai 200444, China

Received:2018-07-13 Online:2020-12-31 Published:2020-12-29
Contact: XU Meihua E-mail:mhxu@shu.edu.cn

摘要/Abstract

摘要：

由于传统的辅助驾驶系统的视角呈现较为单一, 故使用情况受限. 提出了一种可自动切换的多模式(包含行车和倒车)全景拼接算法,为驾驶员提供了更灵活的观察视角. 首先,将来自摄像头的输入图像进行畸变校正; 然后, 提取相邻帧图像的特征点,通过位移量判断行车方向; 其次, 根据行车方向进行视角变换,前进方向采用柱面投影, 后退方向采用透视变换; 最后,根据图像特征点进行配准, 采用分段拼接的方法对传统拼接方法进行改进,并对重合区域采用图像融合算法, 完成拼接. 实验结果表明,车载多模式全景融合拼接模型可根据行车方向自动切换模式,对每个摄像头图像还原度更高, 且柱面投影算法所耗时间减少了75%.

关键词: 透视变换, 柱面投影, 特征点配准, 全景融合, 多模式

Abstract:

As the traditional driver assistant system provides only one viewpoint, its usage is limited. A panoramic stitching algorithm, which can automatically switch between forward and backward modes, is proposed. This algorithm can provide a more flexible viewpoint for drivers. First, the distortion of images input by fish-eye cameras is corrected. Second, the driving direction is deduced by the displacements of the feature points detected between adjacent images. Third, the viewpoint transformation is chosen according to the driving direction, using cylindrical projection for the forward direction and perspective transformation for the backward direction. Finally, images are matched using piecewise stitching in place of the traditional stitching method and overlapped regions are fused. The model can automatically switch the mode according to the driving direction. The images of the cameras are highly reserved. The time cost for the cylindrical projection of this model is 75% lower than the traditional value.

Key words: perspective transformation, cylindrical projection, feature registration, fusion panorama, multi-mode

中图分类号:

TP312

刘畅, 徐美华, 郭爱英. 一种车载多模式全景融合拼接算法[J]. 上海大学学报(自然科学版), 2020, 26(6): 909-920.

LIU Chang, XU Meihua, GUO Aiying. Stitching algorithm of multi-mode fusional vehicle panorama[J]. Journal of Shanghai University（Natural Science Edition）, 2020, 26(6): 909-920.

图/表 17

图1

图2

图3

图4

图5

图6

图7

图8

图9

图10

图11

图12

图13

图14

表1

表2

表3

参考文献 19

[1]	蔡丽欢, 廖英豪, 郭东辉. 图像拼接方法及其关键技术研究[J]. 计算机技术与发展, 2008,18(3):1-4.
[2]	Brown M, Lowe D G. Automatic panoramic image stitching using invariant features[J]. International Journal of Computer Vision, 2007,74(1):59-73. doi: 10.1007/s11263-006-0002-3
[3]	Lowe D G. Object recognition from local scale-invariant features[C]// The Proceedings of the Seventh IEEE International Conference on Computer Vision. 2002: 1150.
[4]	Lowe D G. Local feature view clustering for 3D object recognition[C]// 2001 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2001: 682-688.
[5]	Bay H, Ess A, Tuytelaars T, et al. Speeded-up robust features[J]. Computer Vision and Image Understanding, 2008,110(3):404-417.
[6]	Joo K, Kim N, Oh T H, et al. Line meets as-projective-as-possible image stitching with moving DLT[C]// IEEE International Conference on Image Processing. 2015. DOI: 10.1109/ICIP.2015.7350985.
[7]	李晓辉, 周荫清, 王祖林. 基于曲面拼接的球面全景生成算法[J]. 北京航空航天大学学报, 2007,33(6):668-671.
[8]	刘冬, 秦瑞, 陈曦, 等. 3D车载环视全景生成方法[J]. 计算机科学, 2017,44(4):302-305.
[9]	杨刚, 臧春华, 李仲年. 基于参考点配准法的全景泊车系统研究[J]. 传感器与微系统, 2017,36(10):41-44.
[10]	李彬. 全景视频实时拼接技术研究[D]. 西安: 西安工业大学, 2016.
[11]	杨镇豪. 基于柱面变换的图像拼接技术[J]. 现代计算机(专业版), 2018(10):11-14.
[12]	张建伟, 雷霖. 基于透视投影的垂直视角投影算法研究[J]. 成都大学学报(自然科学版), 2017,36(1):47-50.
[13]	Fischler M A. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography[J]. Readings in Computer Vision, 1987,24(6):726-740.
[14]	Zhang Z. Parameter estimation techniques: a tutorial with application to conic fitting[J]. Image Vision Comput, 1997,15(1):59-76. doi: 10.1016/S0262-8856(96)01112-2
[15]	Mokhtarian F, Suomela R. Robust image corner detection through curvature scale space[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1999,20(12):1376-1381. doi: 10.1109/34.735812
[16]	Zhang Z Y. A flexible new technique for camera calibration[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2000,22(11):1330-1334. doi: 10.1109/34.888718
[17]	Budil D E, Lee S, Saxena S, et al. Nonlinear-least-squares analysis of slow-motion EPR spectra in one and two dimensions using a modified levenberg—marquardt algorithm[J]. Journal of Magnetic Resonance, 1996,120(2):155-189.
[18]	Chang C H, Sato Y, Chuang Y Y. Shape-preserving half-projective warps for imagestitching[C]// 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2014: 3254-3261.
[19]	厉丹, 肖理庆, 田隽, 等. 基于改进相位相关与特征点配准的多图拼接算法[J]. 计算机科学, 2018,45(1):313-319.

一种车载多模式全景融合拼接算法

Stitching algorithm of multi-mode fusional vehicle panorama

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 17

参考文献 19

相关文章 3

编辑推荐

Metrics

本文评价

[1]	陈娟, 荆昊, 方宇杰. 基于多种群协同进化算法的混合交通流信号优化[J]. 上海大学学报(自然科学版), 2020, 26(6): 1001-1014.
[2]	胡燕伟, 徐美华, 郭爱英. 基于 BING-casDPM 的快速行人检测算法[J]. 上海大学学报(自然科学版), 2019, 25(5): 712-721.
[3]	龚露鸣, 徐美华, 刘冬军, 张发宇. 基于混合高斯和HOG+SVM的行人检测模型[J]. 上海大学学报(自然科学版), 2018, 24(3): 341-351.