中文

Journal of Shanghai University >

2022 , Vol. 28 >Issue 3: 492 - 503

DOI: https://doi.org/10.12066/j.issn.1007-2861.2383

Machine Learning

Multi-modal data representation learning for ceramic coating materials

Expand
  • 1. School of Computer Engineering and Science, Shanghai University, Shanghai 200444, China
    2. Zhejiang Laboratory, Hangzhou 311100, Zhejiang, China
    3. Center of Materials Informatics and Data Science, Materials Genome Institute, Shanghai University, Shanghai 200444, China
    4. College of Sciences, Shanghai University, Shanghai 200444, China

Received date: 2022-03-28

  Online published: 2022-05-27

Fold

Abstract

Ceramic coatings have excellent temperature resistance, corrosion resistance, and wear resistance, among other advantages. Their thermal expansion coefficient and thermal conductivity are two properties directly related to their performance. To address the issues of high experimental costs and challenging test conditions, we propose a method to predict the performance of ceramic coating materials based on multimodal data representation learning. To enlarge the data set, this method uses the Gaussian mixture model virtual sample generation (GMMVSG) algorithm to generate samples that match the real ceramic-coating data distribution. The method extracts micro-structural image data's features using the very deep convolutional neural network VGG16, extracts structured data's features using TabNet, and fuses the features of the extracted image data with those of the structured data. the final prediction models based on three machine learning algorithms-K-nearest neighbor (KNN), support-vector-machine regression (SVR), and multi-layer perceptron（MLP）—are established by using multimodal data representation to predict the thermal expansion coefficient and thermal conductivity of the performance index of ceramic coatings. The experimental results show that the proposed multimodal-data representation-learning model has a better prediction performance than that of the single-modal-data machine-learning model, and that the former model based on the MLP can most accurately predict ceramic coating performance. In the test set, the mean absolute and mean square errors for the prediction of the thermal expansion coefficient are 0.026 6 and 0.001 7, respectively, and the mean absolute and mean square errors for the prediction of thermal conductivity are 0.017 9 and 0.000 7, respectively. Our proposed learning method for multimodal data representation of ceramic coating materials effectively combines structured and unstructured data to learn both types of modal data with potentially shared information and successfully improves the pred.

Key words： ceramic coatings; Gaussian mixture models; multimodal data representation; machine learning algorithm

Cite this article

WU Xing, HU Mingtao, DING Peng . Multi-modal data representation learning for ceramic coating materials[J]. Journal of Shanghai University, 2022 , 28(3) : 492 -503 . DOI: 10.12066/j.issn.1007-2861.2383

References

[1] 倪嘉, 史昆, 薛松海, 等. 航空发动机用热障涂层陶瓷材料的发展现状及展望[J]. 材料导报, 2021, 35(S1): 163-168.
[2] 刘丹丹, 樊自拴. 超高温陶瓷涂层的研究进展[J]. 材料保护, 2020, 53(5): 105-110.
[3] 魏晨光. 陶瓷涂层物理性能评价的相对法模型及验证[D]. 北京: 中国建筑材料科学研究总院, 2015.
[4] Lu Z, Jiang T, Min Z, et al. Review of thermal properties of graphite: coefficient of thermal expansion and thermal conductivity[J]. New Carbon Materials, 2022, 37(3): 1-12.
[5] Liu B, Vu-Bac N, Zhuang X, et al. Stochastic full-range multiscale modeling of thermal conductivity of polymeric carbon nanotubes composites: a machine learning approach[J]. Composite Structures, 2022, 289: 115393.
[6] Baltrušaitis T, Ahuja C, Morency L P. Multimodal machine learning: a survey and taxonomy[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 41(2): 423-443.
[7] 银正强. 基于多模态表征学习的源代码搜索研究[D]. 西安: 电子科技大学, 2021.
[8] 宋云峰, 任鸽, 杨勇, 等. 基于注意力的多层次混合融合的多任务多模态情感分析[J]. 计算机应用研究, 2022, 39(3): 716-720.
[9] 田彦涛, 黄兴, 卢辉遒, 等. 基于注意力与深度交互的周车多模态行为轨迹预测[J]. 吉林大学学报(工学版), 2022(1): 1-9.
[10] 薛景瑜. 基于深度学习的多模态阿尔兹海默症预测方法研究[D]. 济南: 齐鲁工业大学, 2021.
[11] Maimaitijiang M, Sagan V, Sidike P, et al. Soybean yield prediction from UAV using multimodal data fusion and deep learning[J]. Remote Sensing of Environment, 2020, 237: 111599.
[12] Pakdamanian E, Sheng S, Baee S, et al. Deeptake: prediction of driver takeover behavior using multimodal data[C]// Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 2021: 1-14.
[13] Liu T, Huang J, Liao T, et al. A hybrid deep learning model for predicting molecular subtypes of human breast cancer using multimodal data[J]. IRBM, 2022, 43(1): 62-74.
[14] McClenny L, Haile M, Attari V, et al. Deep multimodal transfer-learned regression in data-poor domains[DB/OL]. [2022-02-15]. https://doi.org/10.48550/arXiv.2006.09310.
[15] Li L, Damarla S K, Wang Y, et al. A Gaussian mixture model based virtual sample generation approach for small datasets in industrial processes[J]. Information Sciences, 2021, 581: 262-277.
[16] Sha'abani M, Fuad N, Jamal N, et al. KNN and SVM classification for EEG: a review[M]. New York: Springer, 2020: 555-565.
[17] Awad M, Khanna R. Support vector regression[M]. Berkeley: Efficient Learning Machines, 2015: 67-80.
[18] Taud H, Mas J F. Multilayer perceptron (MLP)[M]. New York: Springer, 2018: 451-455.
[19] Krizhevsky A, Sutskever I, Hinton G E. ImageNet classification with deep convolutional neural networks[DB/OL]. [2022-02-15]. https://web.cs.ucdavis.edu/-yjlee/teaching/ecs289g-winter2018/alexnet.pdf.
[20] Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition[DB/OL]. [2022-02-15]. https://ui.adsabs.harvard.edu/abs/2014arXiv1409.1556S/abstract.
[21] Arik S O, Pfister T. Tabnet: attentive interpretable tabular learning[C]// AAAI. 2021: 6679-6687.
[22] Sharma K, Giannakos M. Multimodal data capabilities for learning: what can multimodal data tell us about learning?[J]. British Journal of Educational Technology, 2020, 51(5): 1450-1484.
Options
Outlines

/

〈 〉