收稿日期: 2022-03-15
网络出版日期: 2022-05-27
基金资助
国家重点研发计划资助项目(2018YFB0704400);云南省重大科技专项资助项目(202102AB080019-3);云南省重大科技专项资助项目(202002AB080001-2);之江实验室科研攻关资助项目(2021PE0AC02);上海张江国家自主创新示范区专项发展资金重大资助项目(ZJ2021-ZD-006)
Kalman filter based method for processing small noisy sample data
Received date: 2022-03-15
Online published: 2022-05-27
提出一种基于卡尔曼滤波和扩展卡尔曼滤波的小样本噪声数据处理方法. 首先, 通过物理模型或经验公式建立系统模型. 然后, 利用系统模型预测模型数据. 最后, 采用观测数据修正模型数据, 达到平滑数据噪声的效果. 实验结果表明, 对于BC500耐候钢腐蚀增重数据, 用差分整合移动平均自回归(autoregressive integrated moving average, ARIMA)模型和随机森林(random forest, RF)模型进行腐蚀增重预测时, 经卡尔曼滤波降噪后, 决定系数$R^2$平均提升6.4%, 而经扩展卡尔曼滤波降噪后, $R^2$平均提升4.9%, 验证了本方法的有效性.
刘芬, 范洪强, 吕涛, 李谦, 钱权 . 基于卡尔曼滤波的含噪声小样本数据处理方法[J]. 上海大学学报(自然科学版), 2022 , 28(3) : 427 -439 . DOI: 10.12066/j.issn.1007-2861.2379
A small sample noisy data processing method based on Kalman filter and extended Kalman filter has been proposed. The core idea was to establish a system model using physical models or empirical formula, then used the system model to predict the model data, and finally used the observation data to correct the model data and achieve the effect of smoothing data noise. Experimental results showed that when using the autoregressive integrated moving average (ARIMA) model and random forest (RF) model to predict the corrosion weight gain of weather steel BC500, the coefficient of determination $R^{2}$ was increased by an average of 6.4% after Kalman filter denoising, while the $R^{2}$ was increased by an average of 4.9% after extended Kalman filter. These results verified the effectiveness of the proposed methods.
Key words： Kalman filter; extended Kalman filter; data denoising; small samples; corrosion data
| [1] | 王石, 李玉忱, 刘乃丽, 等. 在属性级别上处理噪声数据的数据清洗算法[J]. 计算机工程, 2005, 31(9): 3. |
| [2] | Kerber R. ChiMerge: discretization of numeric attributes[C]// Proceedings of the 10th National Conference on Artificial Intelligence. 1992: 123-128. |
| [3] | Catlett J. On changing continuous attributes into ordered discrete attributes[M]. New York: Springer-Verlag, 1991. |
| [4] | 傅涛, 孙文静, 孙亚民, 等. 基于分箱统计的 FCM 算法及其在网络入侵检测中的应用[J]. 计算机科学, 2008, 35(4): 4. |
| [5] | Box G, Jenkins G M, Reinsel G C. Time series analysis forecasting and control[J]. Journal of Time, 1976, 31(2): 238-242. |
| [6] | Li D. Support vector regression based image denoising[J]. Image and Vision Computing, 2009, 27(6): 623-627. |
| [7] | López-Rubio E, Florentín-Núñez M N. Kernel regression based feature extraction for 3D MR image denoising[J]. Medical Image Analysis, 2011, 15(4): 498-513. |
| [8] | Huang P, Yang Z J, Wang W B, et al. Denoising low-rank discrimination based least squares regression for image classification[J]. Information Sciences, 2022, 587: 247-264. |
| [9] | Bai X R, Peng X. Radar image series denoising of space targets based on Gaussian process regression[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(7): 4659-4669. |
| [10] | 孙添, 何国良. 基于 K-均值聚类和泰森多边形的异常值检测方法[J]. 重庆文理学院学报(社会科学版), 2016, 35(2): 4. |
| [11] | Dong W, Xin L, Lei Z, et al. Sparsity-based image denoising via dictionary learning and structural clustering[C]// 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2011: 457-464. |
| [12] | 胡姣姣. 基于深度学习的飞行器遥测时间序列数据异常检测与预测方法研究[D]. 西安: 西安理工大学, 2019. |
| [13] | Zhang S, Salari E. Image denoising using a neural network based non-linear filter in wavelet domain[C]// IEEE International Conference on Acoustics, Speech, and Signal Processing. 2005: 855947. |
| [14] | Mozhde F P, Mansour V, Mohammad A G. Lung sound signal denoising using discrete wavelet transform and artificial neural network[J]. Biomedical Signal Processing and Control, 2022, 72: 103329. |
| [15] | Yao C, Tang Y B, Sun J, et al. Multiscale residual fusion network for image denoising[J]. IET Image Process, 2022, 16(3): 878-887. |
| [16] | Basar T. A new approach to linear filtering and prediction problems[M]. New Jersey: Wiley-IEEE Press, 2009. |
| [17] | Kalman R E. New results in linear filtering and prediction theory[J]. Journal of Fluids Engineering, 1961, 83(1): 95-108. |
| [18] | Huang J F, Sun I W. Nonanomalous electrodeposition of zinc-iron alloys in an acidic zinc chloride-1-ethyl-3-methylimidazolium chloride ionic liquid[J]. Journal of the Electrochemical Society, 2004, 151(1): C8-C14. |
| [19] | Bajat J B, Miškovivić-Stanković V B, Kačrević-Popović Z. The influence of steel surface modification by electrodeposited Zn-Fe alloys on the protective behaviour of an epoxy coating[J]. Progress in Organic Coatings, 2003, 47(1): 49-54. |
| [20] | Bajat J B, Miskovic-Stankovic V B, Md M, et al. Electrochemical deposition and characterization of Zn-Fe alloys[J]. Journal of the Serbian Chemical Society, 2004, 69(10): 807-815. |
| [21] | 杨仲年. 耐候钢和 Zn 与 Zn-Fe 合金镀层的腐蚀电化学行为研究[D]. 杭州: 浙江大学, 2006. |
| [22] | Shuichi H, Takayuki K, Hideaki M, et al. Taxonomy for protective ability of rust layer using its composition formed on weathering steel bridge[J]. Corrosion Science, 2007, 49(3): 1131-1142. |
| [23] | 梁彩凤, 侯文泰. 合金元素对碳钢和低合金钢在大气中耐腐蚀性的影响[J]. 中国腐蚀与防护学报, 1997, 17(2): 6. |
| [24] | 刘国超, 董俊华, 韩恩厚, 等. 耐候钢锈层研究进展[J]. 腐蚀科学与防护技术, 2006, 18(4): 5. |
| [25] | Townsend H E. Effects of alloying elements on the corrosion of steel in industrial atmospheres[J]. Corrosion, 2001, 57: 497-501. |
| [26] | Ma Y, Ying L, Wang F. The atmospheric corrosion kinetics of low carbon steel in a tropical marine environment[J]. Corrosion Science, 2010, 52(5): 1796-1800. |
| [27] | Hao L, Zhang S, Dong J, et al. Atmospheric corrosion resistance of MnCuP weathering steel in simulated environments[J]. Corrosion Science, 2011, 53(12): 4187-4192. |
| [28] | Lan T, Thoa N, Nishimura R, et al. Atmospheric corrosion of carbon steel under field exposure in the southern part of Vietnam[J]. Corrosion Science, 2006, 48(1): 179-192. |
| [29] | Rajamani M R. Data-based techniques to improve state estimation in model predictive control[D]. Madison: The University of Wisconsin, 2007. |
| [30] | Murali R R, Rawlings J B. Estimation of the disturbance structure from data using semidefinite programming and optimal weighting[J]. Automatica, 2009, 45(1): 142-148. |
| [31] | Bania P, Baranowski J. Field Kalman filter and its approximation[C]// Decision & Control. 2016: 2875-2880. |
/
| 〈 |
|
〉 |
