基于卡尔曼滤波的含噪声小样本数据处理方法

doi:10.12066/j.issn.1007-2861.2379

上海大学学报(自然科学版) ›› 2022, Vol. 28 ›› Issue (3): 427-439.doi: 10.12066/j.issn.1007-2861.2379

• 数据采集、数据库和数据处理 • 上一篇下一篇

基于卡尔曼滤波的含噪声小样本数据处理方法

刘芬¹, 范洪强², 吕涛³, 李谦²^,⁴, 钱权¹^,³^,⁵()

1.上海大学计算机工程与科学学院, 上海 200444
2.上海大学材料科学与工程学院, 上海 200444
3.上海大学材料基因组工程研究院材料信息与数据科学中心, 上海 200444
4.重庆大学材料科学与工程学院国家镁合金材料工程技术研究中心, 重庆 400044
5.之江实验室, 浙江杭州 311100

收稿日期:2022-03-15 出版日期:2022-06-30 发布日期:2022-05-27
通讯作者: 钱权 E-mail:qqian@shu.edu.cn
作者简介:钱权(1972—), 男, 研究员, 博士生导师, 博士, 研究方向为材料信息学、机器学习、网络安全等. E-mail: qqian@shu.edu.cn
基金资助:
国家重点研发计划资助项目(2018YFB0704400);云南省重大科技专项资助项目(202102AB080019-3);云南省重大科技专项资助项目(202002AB080001-2);之江实验室科研攻关资助项目(2021PE0AC02);上海张江国家自主创新示范区专项发展资金重大资助项目(ZJ2021-ZD-006)

Kalman filter based method for processing small noisy sample data

LIU Fen¹, FAN Hongqiang², LÜ Tao³, LI Qian²^,⁴, QIAN Quan¹^,³^,⁵()

1. School of Computer Engineering and Science, Shanghai University, Shanghai 200444, China
2. School of Materials Science and Engineering, Shanghai University, Shanghai 200444, China
3. Center of Materials Informatics and Data Science, Materials Genome Institute, Shanghai University, Shanghai 200444, China
4. National Engineering Research Center for Magnesium Alloys, School of Materials Science and Engineering, Chongqing University, Chongqing 400044, China
5. Zhejiang Laboratory, Hangzhou 311100, Zhejiang, China

Received:2022-03-15 Online:2022-06-30 Published:2022-05-27
Contact: QIAN Quan E-mail:qqian@shu.edu.cn

摘要/Abstract

摘要：

提出一种基于卡尔曼滤波和扩展卡尔曼滤波的小样本噪声数据处理方法. 首先, 通过物理模型或经验公式建立系统模型. 然后, 利用系统模型预测模型数据. 最后, 采用观测数据修正模型数据, 达到平滑数据噪声的效果. 实验结果表明, 对于BC500耐候钢腐蚀增重数据, 用差分整合移动平均自回归(autoregressive integrated moving average, ARIMA)模型和随机森林(random forest, RF)模型进行腐蚀增重预测时, 经卡尔曼滤波降噪后, 决定系数$R^2$平均提升6.4%, 而经扩展卡尔曼滤波降噪后, $R^2$平均提升4.9%, 验证了本方法的有效性.

关键词: 卡尔曼滤波, 扩展卡尔曼滤波, 数据降噪, 小样本, 腐蚀数据

Abstract:

A small sample noisy data processing method based on Kalman filter and extended Kalman filter has been proposed. The core idea was to establish a system model using physical models or empirical formula, then used the system model to predict the model data, and finally used the observation data to correct the model data and achieve the effect of smoothing data noise. Experimental results showed that when using the autoregressive integrated moving average (ARIMA) model and random forest (RF) model to predict the corrosion weight gain of weather steel BC500, the coefficient of determination $R^{2}$ was increased by an average of 6.4% after Kalman filter denoising, while the $R^{2}$ was increased by an average of 4.9% after extended Kalman filter. These results verified the effectiveness of the proposed methods.

Key words: Kalman filter, extended Kalman filter, data denoising, small samples, corrosion data

中图分类号:

TP 391

刘芬, 范洪强, 吕涛, 李谦, 钱权. 基于卡尔曼滤波的含噪声小样本数据处理方法[J]. 上海大学学报(自然科学版), 2022, 28(3): 427-439.

LIU Fen, FAN Hongqiang, LÜ Tao, LI Qian, QIAN Quan. Kalman filter based method for processing small noisy sample data[J]. Journal of Shanghai University（Natural Science Edition）, 2022, 28(3): 427-439.

图/表 8

图1

图2

图3

表1

图4

图5

表2

表3

参考文献 31

[1]	王石, 李玉忱, 刘乃丽, 等. 在属性级别上处理噪声数据的数据清洗算法[J]. 计算机工程, 2005, 31(9): 3.
[2]	Kerber R. ChiMerge: discretization of numeric attributes[C]// Proceedings of the 10th National Conference on Artificial Intelligence. 1992: 123-128.
[3]	Catlett J. On changing continuous attributes into ordered discrete attributes[M]. New York: Springer-Verlag, 1991.
[4]	傅涛, 孙文静, 孙亚民, 等. 基于分箱统计的 FCM 算法及其在网络入侵检测中的应用[J]. 计算机科学, 2008, 35(4): 4.
[5]	Box G, Jenkins G M, Reinsel G C. Time series analysis forecasting and control[J]. Journal of Time, 1976, 31(2): 238-242.
[6]	Li D. Support vector regression based image denoising[J]. Image and Vision Computing, 2009, 27(6): 623-627. doi: 10.1016/j.imavis.2008.06.006
[7]	López-Rubio E, Florentín-Núñez M N. Kernel regression based feature extraction for 3D MR image denoising[J]. Medical Image Analysis, 2011, 15(4): 498-513. doi: 10.1016/j.media.2011.02.006 pmid: 21414834
[8]	Huang P, Yang Z J, Wang W B, et al. Denoising low-rank discrimination based least squares regression for image classification[J]. Information Sciences, 2022, 587: 247-264. doi: 10.1016/j.ins.2021.12.031
[9]	Bai X R, Peng X. Radar image series denoising of space targets based on Gaussian process regression[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(7): 4659-4669. doi: 10.1109/TGRS.2019.2892183
[10]	孙添, 何国良. 基于 K-均值聚类和泰森多边形的异常值检测方法[J]. 重庆文理学院学报(社会科学版), 2016, 35(2): 4.
[11]	Dong W, Xin L, Lei Z, et al. Sparsity-based image denoising via dictionary learning and structural clustering[C]// 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2011: 457-464.
[12]	胡姣姣. 基于深度学习的飞行器遥测时间序列数据异常检测与预测方法研究[D]. 西安: 西安理工大学, 2019.
[13]	Zhang S, Salari E. Image denoising using a neural network based non-linear filter in wavelet domain[C]// IEEE International Conference on Acoustics, Speech, and Signal Processing. 2005: 855947.
[14]	Mozhde F P, Mansour V, Mohammad A G. Lung sound signal denoising using discrete wavelet transform and artificial neural network[J]. Biomedical Signal Processing and Control, 2022, 72: 103329. doi: 10.1016/j.bspc.2021.103329
[15]	Yao C, Tang Y B, Sun J, et al. Multiscale residual fusion network for image denoising[J]. IET Image Process, 2022, 16(3): 878-887. doi: 10.1049/ipr2.12394
[16]	Basar T. A new approach to linear filtering and prediction problems[M]. New Jersey: Wiley-IEEE Press, 2009.
[17]	Kalman R E. New results in linear filtering and prediction theory[J]. Journal of Fluids Engineering, 1961, 83(1): 95-108.
[18]	Huang J F, Sun I W. Nonanomalous electrodeposition of zinc-iron alloys in an acidic zinc chloride-1-ethyl-3-methylimidazolium chloride ionic liquid[J]. Journal of the Electrochemical Society, 2004, 151(1): C8-C14. doi: 10.1149/1.1628235
[19]	Bajat J B, Miškovivić-Stanković V B, Kačrević-Popović Z. The influence of steel surface modification by electrodeposited Zn-Fe alloys on the protective behaviour of an epoxy coating[J]. Progress in Organic Coatings, 2003, 47(1): 49-54. doi: 10.1016/S0300-9440(03)00020-1
[20]	Bajat J B, Miskovic-Stankovic V B, Md M, et al. Electrochemical deposition and characterization of Zn-Fe alloys[J]. Journal of the Serbian Chemical Society, 2004, 69(10): 807-815. doi: 10.2298/JSC0410807B
[21]	杨仲年. 耐候钢和 Zn 与 Zn-Fe 合金镀层的腐蚀电化学行为研究[D]. 杭州: 浙江大学, 2006.
[22]	Shuichi H, Takayuki K, Hideaki M, et al. Taxonomy for protective ability of rust layer using its composition formed on weathering steel bridge[J]. Corrosion Science, 2007, 49(3): 1131-1142. doi: 10.1016/j.corsci.2006.06.016
[23]	梁彩凤, 侯文泰. 合金元素对碳钢和低合金钢在大气中耐腐蚀性的影响[J]. 中国腐蚀与防护学报, 1997, 17(2): 6.
[24]	刘国超, 董俊华, 韩恩厚, 等. 耐候钢锈层研究进展[J]. 腐蚀科学与防护技术, 2006, 18(4): 5.
[25]	Townsend H E. Effects of alloying elements on the corrosion of steel in industrial atmospheres[J]. Corrosion, 2001, 57: 497-501. doi: 10.5006/1.3290374
[26]	Ma Y, Ying L, Wang F. The atmospheric corrosion kinetics of low carbon steel in a tropical marine environment[J]. Corrosion Science, 2010, 52(5): 1796-1800. doi: 10.1016/j.corsci.2010.01.022
[27]	Hao L, Zhang S, Dong J, et al. Atmospheric corrosion resistance of MnCuP weathering steel in simulated environments[J]. Corrosion Science, 2011, 53(12): 4187-4192. doi: 10.1016/j.corsci.2011.08.028
[28]	Lan T, Thoa N, Nishimura R, et al. Atmospheric corrosion of carbon steel under field exposure in the southern part of Vietnam[J]. Corrosion Science, 2006, 48(1): 179-192. doi: 10.1016/j.corsci.2004.11.018
[29]	Rajamani M R. Data-based techniques to improve state estimation in model predictive control[D]. Madison: The University of Wisconsin, 2007.
[30]	Murali R R, Rawlings J B. Estimation of the disturbance structure from data using semidefinite programming and optimal weighting[J]. Automatica, 2009, 45(1): 142-148. doi: 10.1016/j.automatica.2008.05.032
[31]	Bania P, Baranowski J. Field Kalman filter and its approximation[C]// Decision & Control. 2016: 2875-2880.

基于卡尔曼滤波的含噪声小样本数据处理方法

Kalman filter based method for processing small noisy sample data

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 8

参考文献 31

相关文章 6

编辑推荐

Metrics

本文评价

[1]	姚煜, 胡涛, 付建勋, 胡顺波. 小样本分散数据的回归建模和多目标优化[J]. 上海大学学报(自然科学版), 2022, 28(3): 451-462.
[2]	李成范, 赵俊娟. 面向遥感图像的小样本目标检测改进算法研究[J]. 上海大学学报(自然科学版), 2022, 28(2): 314-323.
[3]	高文凯, 严利民, 孙叠. 一种变参数模型平方根 UKF 锂离子电池SOC 估计方法[J]. 上海大学学报(自然科学版), 2020, 26(3): 413-424.
[4]	张金艺1,2,3, 徐德政2, 李若涵1, 陈兴秀1, 徐秦乐2. 9轴MEMS-IMU实时姿态估算算法[J]. 上海大学学报(自然科学版), 2015, 21(5): 547-559.
[5]	沙晓萍;陈惠民;鲍敏祺;韩逸昕. 基于船舶的GPS/单陀螺航向融合算法[J]. 上海大学学报(自然科学版), 2008, 14(1): 7-9 .
[6]	张晶晶;何幼桦;王巧兰;忻莉莉. 改进的二维最优信息扩散方法[J]. 上海大学学报(自然科学版), 2006, 12(4): 371-375 .