基于预训练特征中心的古汉语翻译模型

doi:10.12066/j.issn.1007-2861.2555

上海大学学报(自然科学版) ›› 2026, Vol. 32 ›› Issue (4): 665-677.doi: 10.12066/j.issn.1007-2861.2555

• 通信与信息工程 • 上一篇

基于预训练特征中心的古汉语翻译模型

金彦亮^1,2, 高至锋^1,2, 高塬^1,2

1. 上海大学通信与信息工程学院, 上海 200444;
2. 上海大学上海先进通信与数据科学研究院, 上海 200444

收稿日期:2023-12-14 发布日期:2026-09-04
通讯作者: 金彦亮(1973—),男,副教授,博士生导师,博士,研究方向为宽带无线通信、SG/下一代无线接入等.E-mail:jinyanliang@staff.shu.edu.cn E-mail:jinyanliang@staff.shu.edu.cn

Translating classical Chinese with pre-trained center embedding model

JIN Yanliang^1,2, GAO Zhifeng^1,2, GAO Yuan^1,2

1. School of Communication and Information Engineering, Shanghai University, Shanghai 200444, China;
2. Shanghai Institute for Advanced Communication and Data Science, Shanghai University, Shanghai 200444, China

Received:2023-12-14 Published:2026-09-04

摘要/Abstract

摘要： 针对当前古汉语机器翻译任务中平行语料较少、因词义多变产生的模型翻译用词精确度低的问题,提出了一种基于Transformer模型的预训练特征中心算法模型.首先,训练语言模型和提取预训练特征中心,以提高在平行语料不足时的泛化能力;其次,Transformer模型在特征中心空间生成查询点,依据查询点确定生成在特征中心中的距离,以提高模型对一词多义的理解能力.实验结果表明,该方法在翻译准确率上总体高于基准模型,在西汉时期的数据集中BLEU 1~4分数分别高出Transformer模型6.1、4.9、3.7和2.8,在五代十国数据集中BLEU 1~4分数分别高出Transformer模型4.2、4.2、3.3和2.6,在唐朝数据集中BLEU 1~4分数表现最好.与此同时,相较于Transformer模型,该方法参数量下降了5.73 M,充分证明该算法模型的优势.

关键词: 自然语言处理, Transformer, 机器翻译, 预训练特征中心, 古汉语翻译

Abstract: To solve the low accuracy of model translation words owing to the lack of parallel corpora and variable meanings in the current ancient Chinese machine translation model, a Transformer-based pre-trained feature center algorithm model is proposed. This method first trains the language model to extract pretrained feature centers to improve generalization ability when parallel corpora are insufficient. Second, the Transformer generates queries in the feature center space and determines the generation based on the distance between the queries in the feature center to improve the model’s understanding of polysemy. The experimental results indicate that the generation accuracy of the proposed method is generally higher than that of the benchmark model. In the XiHan Datasets, BLEU 1~4 scores were 6.1, 4.9, 3.7, and 2.8 higher than those of the Transformers. In the WuDaiShiGuo Datasets, the BLEU 1~4 scores were 4.2, 4.2, 3.3, and 2.6 higher than those of the Transformers. In the Tang Datasets, the BLEU 1~4 scores were the best. Simultaneously, compared with the Transformer, the parameter count of the proposed method decreased by 5.73 M, demonstrating its advantages.

Key words: natural language processing, Transformer, machine translation, pre-trained feature center, classical Chinese translation

中图分类号:

TP391.1

金彦亮, 高至锋, 高塬. 基于预训练特征中心的古汉语翻译模型[J]. 上海大学学报(自然科学版), 2026, 32(4): 665-677.

JIN Yanliang, GAO Zhifeng, GAO Yuan. Translating classical Chinese with pre-trained center embedding model[J]. Journal of Shanghai University（Natural Science Edition）, 2026, 32(4): 665-677.

[1] Kalchbrenner N, Blunsom P. Recurrent continuous translation models [C]// Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing. 2013: 1700-1709.
[2] Cho K, van Merrienboer B, Gulcehre C, et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation [C]// Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. 2014: 1724-1734.
[3] Sutskever I, Vinyals O, Le Q V. Sequence to sequence learning with neural networks [J]. Advances in Neural Information Processing Systems, 2014, 27: 3104-3122.
[4] Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need [J]. Advances in Neural Information Processing Systems, 2017, 30: 5998-6008.
[5] Mnih V, Heess N, Graves A. Recurrent models of visual attention [J]. Advances in Neural Information Processing Systems, 2014, 27: 2204-2212.
[6] Larochelle H, Erhan D, Bengio Y. Zero-data learning of new tasks [C]// AAAI. 2008: 646-651.
[7] Palatucci M, Pomerleau D, Hinton G E, et al. Zero-shot learning with semantic output codes [J]. Advances in Neural Information Processing Systems, 2009, 22: 1410-1418.
[8] Kenton J D M W C, Toutanova L K. BERT: pre-training of deep bidirectional transformers for language understanding [C]// Proceedings of NAACL-HLT. 2019: 4171-4186.
[9] Brown T, Mann B, Ryder N, et al. Language models are few-shot learners [J]. Advances in Neural Information Processing Systems, 2020, 33: 1877-1901.
[10] Rajpurkar P, Zhang J, Lopyrev K, et al. SQuAD: 100000+ questions for machine comprehension of text [C]// Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. 2016: 2383-2392.
[11] Zhang X, Lapata M. Chinese poetry generation with recurrent neural networks [C]// Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. 2014: 670-680.
[12] Wang D Z, He W, Wu H, et al. Chinese poetry generation with planning based neural network [C]// The 26th International Conference on Computational Linguistics: Technical Papers. 2016: 1051-1060.
[13] Yi X Y, Sun M S, Li R Y, et al. Chinese poetry generation with a working memory model [C]// Proceedings of the 27th International Joint Conference on Artificial Intelligence. 2018: 4553- 4559.
[14] Ghazvininejad M, Shi X, Choi Y, et al. Generating topical poetry [C]// Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. 2016: 1183-1191.
[15] 韩芳, 杨天心, 宋继华. 基于句本位句法体系的古汉语机器翻译研究[J]. 中文信息学报, 2015, 29(2): 103-110; 117.
[16] Yan R. I. Poet: automatic poetry composition through recurrent neural networks with iterative polishing schema [C]// IJCAI 2016. 2016: 2238-2244.
[17] Yang Z C, Cai P S, Feng Y S, et al. Generating classical Chinese poems from vernacular Chinese [C]// EMNLP-IJCNLP 2019. 2019: 6155-6164.
[18] Jin P, Wang H L, Ma L M, et al. Translating classical Chinese poetry into modern Chinese with transformer [C]// 22nd Chinese Lexical Semantics Workshop (CLSW). 2022: 474-480.
[19] Holtzman A, Buys J, Du L, et al. The curious case of neural text degeneration [C]// ICLR. 2020: 1-14.
[20] Papineni K, Roukos S, Ward T, et al. BLEU: a method for automatic evaluation of machine translation [C]// Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics. 2002: 311-318.
[21] Ott M, Edunov S, Baevski A, et al. Fairseq: a fast, extensible toolkit for sequence modeling [C]// Proceedings of the 2019 Conference of the North. 2019: 48-53.
[22] Ramachandran P, Liu P J, Le Q. Unsupervised pretraining for sequence to sequence learning [C]// Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. 2017: 383-391.

基于预训练特征中心的古汉语翻译模型

Translating classical Chinese with pre-trained center embedding model

PDF

可视化

摘要/Abstract

引用本文

使用本文

相关文章 3

编辑推荐

Metrics

本文评价

[1]	魏晓, 王晓鑫, 陈永琪, 张惠然. 基于自然语言处理的材料领域知识图谱构建方法[J]. 上海大学学报(自然科学版), 2022, 28(3): 386-398.
[2]	赵倩, 吴悦, 刘宗田. 基于关键 $n$-grams 和门控循环神经网络的文本分类模型[J]. 上海大学学报(自然科学版), 2021, 27(3): 544-552.
[3]	周金, 朱永华, 张铁男, 邢毅雪, 张克. 基于图的联合特征实体链接方法[J]. 上海大学学报(自然科学版), 2020, 26(5): 747-755.