基于卡尔曼滤波的含噪声小样本数据处理方法

doi:10.12066/j.issn.1007-2861.2379

Abstract

Abstract:

A small sample noisy data processing method based on Kalman filter and extended Kalman filter has been proposed. The core idea was to establish a system model using physical models or empirical formula, then used the system model to predict the model data, and finally used the observation data to correct the model data and achieve the effect of smoothing data noise. Experimental results showed that when using the autoregressive integrated moving average (ARIMA) model and random forest (RF) model to predict the corrosion weight gain of weather steel BC500, the coefficient of determination $R^{2}$ was increased by an average of 6.4% after Kalman filter denoising, while the $R^{2}$ was increased by an average of 4.9% after extended Kalman filter. These results verified the effectiveness of the proposed methods.

Key words: Kalman filter, extended Kalman filter, data denoising, small samples, corrosion data

CLC Number:

TP 391

LIU Fen, FAN Hongqiang, LÜ Tao, LI Qian, QIAN Quan. Kalman filter based method for processing small noisy sample data[J]. Journal of Shanghai University（Natural Science Edition）, 2022, 28(3): 427-439.

Figures/Tables 8

Fig.1

Fig.2

Fig.3

Table 1

Fig. 4

Fig.5

Table 2

Table 3

References 31

[1]	王石, 李玉忱, 刘乃丽, 等. 在属性级别上处理噪声数据的数据清洗算法[J]. 计算机工程, 2005, 31(9): 3.
[2]	Kerber R. ChiMerge: discretization of numeric attributes[C]// Proceedings of the 10th National Conference on Artificial Intelligence. 1992: 123-128.
[3]	Catlett J. On changing continuous attributes into ordered discrete attributes[M]. New York: Springer-Verlag, 1991.
[4]	傅涛, 孙文静, 孙亚民, 等. 基于分箱统计的 FCM 算法及其在网络入侵检测中的应用[J]. 计算机科学, 2008, 35(4): 4.
[5]	Box G, Jenkins G M, Reinsel G C. Time series analysis forecasting and control[J]. Journal of Time, 1976, 31(2): 238-242.
[6]	Li D. Support vector regression based image denoising[J]. Image and Vision Computing, 2009, 27(6): 623-627. doi: 10.1016/j.imavis.2008.06.006
[7]	López-Rubio E, Florentín-Núñez M N. Kernel regression based feature extraction for 3D MR image denoising[J]. Medical Image Analysis, 2011, 15(4): 498-513. doi: 10.1016/j.media.2011.02.006 pmid: 21414834
[8]	Huang P, Yang Z J, Wang W B, et al. Denoising low-rank discrimination based least squares regression for image classification[J]. Information Sciences, 2022, 587: 247-264. doi: 10.1016/j.ins.2021.12.031
[9]	Bai X R, Peng X. Radar image series denoising of space targets based on Gaussian process regression[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(7): 4659-4669. doi: 10.1109/TGRS.2019.2892183
[10]	孙添, 何国良. 基于 K-均值聚类和泰森多边形的异常值检测方法[J]. 重庆文理学院学报(社会科学版), 2016, 35(2): 4.
[11]	Dong W, Xin L, Lei Z, et al. Sparsity-based image denoising via dictionary learning and structural clustering[C]// 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2011: 457-464.
[12]	胡姣姣. 基于深度学习的飞行器遥测时间序列数据异常检测与预测方法研究[D]. 西安: 西安理工大学, 2019.
[13]	Zhang S, Salari E. Image denoising using a neural network based non-linear filter in wavelet domain[C]// IEEE International Conference on Acoustics, Speech, and Signal Processing. 2005: 855947.
[14]	Mozhde F P, Mansour V, Mohammad A G. Lung sound signal denoising using discrete wavelet transform and artificial neural network[J]. Biomedical Signal Processing and Control, 2022, 72: 103329. doi: 10.1016/j.bspc.2021.103329
[15]	Yao C, Tang Y B, Sun J, et al. Multiscale residual fusion network for image denoising[J]. IET Image Process, 2022, 16(3): 878-887. doi: 10.1049/ipr2.12394
[16]	Basar T. A new approach to linear filtering and prediction problems[M]. New Jersey: Wiley-IEEE Press, 2009.
[17]	Kalman R E. New results in linear filtering and prediction theory[J]. Journal of Fluids Engineering, 1961, 83(1): 95-108.
[18]	Huang J F, Sun I W. Nonanomalous electrodeposition of zinc-iron alloys in an acidic zinc chloride-1-ethyl-3-methylimidazolium chloride ionic liquid[J]. Journal of the Electrochemical Society, 2004, 151(1): C8-C14. doi: 10.1149/1.1628235
[19]	Bajat J B, Miškovivić-Stanković V B, Kačrević-Popović Z. The influence of steel surface modification by electrodeposited Zn-Fe alloys on the protective behaviour of an epoxy coating[J]. Progress in Organic Coatings, 2003, 47(1): 49-54. doi: 10.1016/S0300-9440(03)00020-1
[20]	Bajat J B, Miskovic-Stankovic V B, Md M, et al. Electrochemical deposition and characterization of Zn-Fe alloys[J]. Journal of the Serbian Chemical Society, 2004, 69(10): 807-815. doi: 10.2298/JSC0410807B
[21]	杨仲年. 耐候钢和 Zn 与 Zn-Fe 合金镀层的腐蚀电化学行为研究[D]. 杭州: 浙江大学, 2006.
[22]	Shuichi H, Takayuki K, Hideaki M, et al. Taxonomy for protective ability of rust layer using its composition formed on weathering steel bridge[J]. Corrosion Science, 2007, 49(3): 1131-1142. doi: 10.1016/j.corsci.2006.06.016
[23]	梁彩凤, 侯文泰. 合金元素对碳钢和低合金钢在大气中耐腐蚀性的影响[J]. 中国腐蚀与防护学报, 1997, 17(2): 6.
[24]	刘国超, 董俊华, 韩恩厚, 等. 耐候钢锈层研究进展[J]. 腐蚀科学与防护技术, 2006, 18(4): 5.
[25]	Townsend H E. Effects of alloying elements on the corrosion of steel in industrial atmospheres[J]. Corrosion, 2001, 57: 497-501. doi: 10.5006/1.3290374
[26]	Ma Y, Ying L, Wang F. The atmospheric corrosion kinetics of low carbon steel in a tropical marine environment[J]. Corrosion Science, 2010, 52(5): 1796-1800. doi: 10.1016/j.corsci.2010.01.022
[27]	Hao L, Zhang S, Dong J, et al. Atmospheric corrosion resistance of MnCuP weathering steel in simulated environments[J]. Corrosion Science, 2011, 53(12): 4187-4192. doi: 10.1016/j.corsci.2011.08.028
[28]	Lan T, Thoa N, Nishimura R, et al. Atmospheric corrosion of carbon steel under field exposure in the southern part of Vietnam[J]. Corrosion Science, 2006, 48(1): 179-192. doi: 10.1016/j.corsci.2004.11.018
[29]	Rajamani M R. Data-based techniques to improve state estimation in model predictive control[D]. Madison: The University of Wisconsin, 2007.
[30]	Murali R R, Rawlings J B. Estimation of the disturbance structure from data using semidefinite programming and optimal weighting[J]. Automatica, 2009, 45(1): 142-148. doi: 10.1016/j.automatica.2008.05.032
[31]	Bania P, Baranowski J. Field Kalman filter and its approximation[C]// Decision & Control. 2016: 2875-2880.

Kalman filter based method for processing small noisy sample data

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 8

References 31

Related Articles 3

Recommended Articles

Metrics

Comments

[1]	GAO Wenkai, YAN Limin, SUN Die. A square root UKF with variable-parameter model for Li-ion batteries SOC estimation method [J]. Journal of Shanghai University（Natural Science Edition）, 2020, 26(3): 413-424.
[2]	ZHANG Jin-yi1,2,3, XU De-zheng2, LI Ruo-han1,CHEN Xing-xiu1, XU Qin-le2. 9-axis MEMS-IMU real-time data fusion algorithm for attitude estimation [J]. Journal of Shanghai University（Natural Science Edition）, 2015, 21(5): 547-559.
[3]	SHA Xiao-ping;CHEN Hui-min;BAO Min-qi;HAN Yi-xin. A GPS/Gyro Attitude Fusion Algorithm for Marine Application [J]. Journal of Shanghai University（Natural Science Edition）, 2008, 14(1): 7-9 .