Journal of Shanghai University >
Text classification model based on essential $n$-grams and gated recurrent neural network
Received date: 2019-03-27
Online published: 2019-10-28
An effective text classification model based on $n$-grams and a gated recurrent neural network is proposed in this paper. First, we adopt a simpler and more efficient pooling layer to replace the traditional convolutional layer to extract the essential $n$-grams as important semantic features. Second, a bidirectional gated recurrent unit (GRU) is constructed to obtain the global dependency features of the input text. Finally, we apply the fusion model of the two features to the text classification task. We evaluate the quality of our model on sentiment and topic categorization tasks over multiple public datasets. Experimental results show that the proposed method can improve text classification effectiveness compared with the traditional model. On accuracy, it approaches an improvement of 1.95% on the 20newsgroup and 1.55% on the Rotten Tomatoes corpus.
ZHAO Qian, WU Yue, LIU Zongtian . Text classification model based on essential $n$-grams and gated recurrent neural network[J]. Journal of Shanghai University, 2021 , 27(3) : 544 -552 . DOI: 10.12066/j.issn.1007-2861.2158
| [1] | Joulin A, Grave E, Bojanowski P, et al. Bag of tricks for efficient text classification[C]// 15th Conference of the European Chapter of the Association for Computational Linguistics. 2017: 427-431. |
| [2] | Li B F, Zhao Z, Liu T, et al. Weighted neural bag-of-$n$-grams model: new baselines for text classification[C]// Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics. 2016: 1591-1600. |
| [3] | Kim Y. Convolutional neural networks for sentence classification[C]// 2014 Conference on Empirial Methods in Natural Language Processing. 2014: 1746-1751. |
| [4] | Conneau A, Schwenk H, Barrault L, et al. Very deep convolutional networks for text classification[C]// 15th Conference of the European Chapter of the Association for Computational Linguistics. 2017: 1107-1116. |
| [5] | Kolen J, Kremer S. Gradient flow in recurrent nets: the difficulty of learning long term dependencies [M]. Piscataway: Wiley-IEEE Press, 2001. |
| [6] | Tang D, Qin B, Liu T. Document modeling with gated recurrent neural network for sentiment classification[C]// Conference on Empirical Methods in Natural Language Processing. 2015: 1422-1432. |
| [7] | Abreu J, Fred L, David M, et al. Hierarchical attentional hybrid neural networks for document classification[C]// International Conference on Artificial Neural Networks. 2019: 396-402. |
| [8] | Hsu S T, Moon C, Jones P, et al. A hybrid CNN-RNN alignment model for phrase-aware sentence classification[C]// 15th Conference of the European Chapter of the Association for Computational Linguistics. 2017: 443-449. |
| [9] | Pang B, Lee L. Seeing stars: exploiting class relationships for sentiment categorization with respect to rating scales[C]// 43rd Annual Meeting of the Association for Computational Linguistics. 2005: 115-124. |
| [10] | Maas A L, Daly R E, Pham P T, et al. Learning word vectors for sentiment analysis[C]// 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies. 2011: 142-150. |
| [11] | Johnson R, Zhang T. Effective use of word order for text categorization with convolutional neural networks[C]// 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2015: 103-112. |
| [12] | Mcauley J, Leskovec J. Hidden factors and hidden topics: understanding rating dimensions with review text[C]// ACM Conference on Recommender Systems. 2013: 165-172. |
| [13] | Dai A M, Le Q V. Semi-supervised sequence learning[J]. Advances in Neural Information Processing Systems, 2015,2015:3079-3087. |
| [14] | Kingma D P, Ba J. Adam: a method for stochastic optimization[C]// 3rd International Conference for Learning Representations. 2015. |
/
| 〈 |
|
〉 |
