To solve the low accuracy of model translation words owing to the lack of parallel corpora and variable meanings in the current ancient Chinese machine translation model, a Transformer-based pre-trained feature center algorithm model is proposed. This method first trains the language model to extract pretrained feature centers to improve generalization ability when parallel corpora are insufficient. Second, the Transformer generates queries in the feature center space and determines the generation based on the distance between the queries in the feature center to improve the model’s understanding of polysemy. The experimental results indicate that the generation accuracy of the proposed method is generally higher than that of the benchmark model. In the XiHan Datasets, BLEU 1~4 scores were 6.1, 4.9, 3.7, and 2.8 higher than those of the Transformers. In the WuDaiShiGuo Datasets, the BLEU 1~4 scores were 4.2, 4.2, 3.3, and 2.6 higher than those of the Transformers. In the Tang Datasets, the BLEU 1~4 scores were the best. Simultaneously, compared with the Transformer, the parameter count of the proposed method decreased by 5.73 M, demonstrating its advantages.
JIN Yanliang
,
GAO Zhifeng
,
GAO Yuan
. Translating classical Chinese with pre-trained center embedding model[J]. Journal of Shanghai University, 2026
, 32(4)
: 665
-677
.
DOI: 10.12066/j.issn.1007-2861.2555
[1] Kalchbrenner N, Blunsom P. Recurrent continuous translation models [C]// Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing. 2013: 1700-1709.
[2] Cho K, van Merrienboer B, Gulcehre C, et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation [C]// Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. 2014: 1724-1734.
[3] Sutskever I, Vinyals O, Le Q V. Sequence to sequence learning with neural networks [J]. Advances in Neural Information Processing Systems, 2014, 27: 3104-3122.
[4] Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need [J]. Advances in Neural Information Processing Systems, 2017, 30: 5998-6008.
[5] Mnih V, Heess N, Graves A. Recurrent models of visual attention [J]. Advances in Neural Information Processing Systems, 2014, 27: 2204-2212.
[6] Larochelle H, Erhan D, Bengio Y. Zero-data learning of new tasks [C]// AAAI. 2008: 646-651.
[7] Palatucci M, Pomerleau D, Hinton G E, et al. Zero-shot learning with semantic output codes [J]. Advances in Neural Information Processing Systems, 2009, 22: 1410-1418.
[8] Kenton J D M W C, Toutanova L K. BERT: pre-training of deep bidirectional transformers for language understanding [C]// Proceedings of NAACL-HLT. 2019: 4171-4186.
[9] Brown T, Mann B, Ryder N, et al. Language models are few-shot learners [J]. Advances in Neural Information Processing Systems, 2020, 33: 1877-1901.
[10] Rajpurkar P, Zhang J, Lopyrev K, et al. SQuAD: 100000+ questions for machine comprehension of text [C]// Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. 2016: 2383-2392.
[11] Zhang X, Lapata M. Chinese poetry generation with recurrent neural networks [C]// Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. 2014: 670-680.
[12] Wang D Z, He W, Wu H, et al. Chinese poetry generation with planning based neural network [C]// The 26th International Conference on Computational Linguistics: Technical Papers. 2016: 1051-1060.
[13] Yi X Y, Sun M S, Li R Y, et al. Chinese poetry generation with a working memory model [C]// Proceedings of the 27th International Joint Conference on Artificial Intelligence. 2018: 4553- 4559.
[14] Ghazvininejad M, Shi X, Choi Y, et al. Generating topical poetry [C]// Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. 2016: 1183-1191.
[15] 韩芳, 杨天心, 宋继华. 基于句本位句法体系的古汉语机器翻译研究[J]. 中文信息学报, 2015, 29(2): 103-110; 117.
[16] Yan R. I. Poet: automatic poetry composition through recurrent neural networks with iterative polishing schema [C]// IJCAI 2016. 2016: 2238-2244.
[17] Yang Z C, Cai P S, Feng Y S, et al. Generating classical Chinese poems from vernacular Chinese [C]// EMNLP-IJCNLP 2019. 2019: 6155-6164.
[18] Jin P, Wang H L, Ma L M, et al. Translating classical Chinese poetry into modern Chinese with transformer [C]// 22nd Chinese Lexical Semantics Workshop (CLSW). 2022: 474-480.
[19] Holtzman A, Buys J, Du L, et al. The curious case of neural text degeneration [C]// ICLR. 2020: 1-14.
[20] Papineni K, Roukos S, Ward T, et al. BLEU: a method for automatic evaluation of machine translation [C]// Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics. 2002: 311-318.
[21] Ott M, Edunov S, Baevski A, et al. Fairseq: a fast, extensible toolkit for sequence modeling [C]// Proceedings of the 2019 Conference of the North. 2019: 48-53.
[22] Ramachandran P, Liu P J, Le Q. Unsupervised pretraining for sequence to sequence learning [C]// Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. 2017: 383-391.
