A Short Text Sentiment Analysis Model Combined with Attention Mechanism of Bidirectional Affective Words

Authors

  • Yong Xu
  • Xiaoyu Li
  • Hengna Wang
  • Hao Chang

DOI:

https://doi.org/10.54097/jceim.v11i1.9474

Keywords:

Convolutional Neural Network, Bidirectional affective word vectors, Attention mechanism, Social media, Sentiment analysis

Abstract

Short text sentiment in social media platforms has important research value. Due to the convolution kernel parameter sharing and pooling operation, convolutional neural network training speed is fast, and the effect is also good. However, most work such as max-pooling, average-pooling, more or less discard some of the secondary features, which makes the final expression of emotion possibly biased. So, in this paper, the attention mechanism applies to the pooling layer of CNN, and a Convolutional Neural Network based on double sentiment word attention pooling (DSA-CNN) is proposed. Each feature has its weight to be calculated. We also note that in the attention mechanism of text classification, the attention query vector is usually randomly initialized during network training, rather than using an existing vector--the semantic information of the previous moment, as in machine translation. Therefore, we use the emotion dictionary and emotion corpus to train bidirectional affective word vectors, so that the emotion-related features can interact with them in the attention. The experiment shows that DSA-CNN has achieved better performance than the classical classification model in a dataset of Weibo nCoV Data, NLPCC 2014, and yf_dianping. The accuracy of the model is 2.18% higher than that of the second-best model. Besides, the convergence rate of DSA-CNN is also significantly improved. The highest accuracy was achieved only in the third epochs, while CNN using the max-pooling needed 14 epochs.

References

Alsmadi I, Hoon G K. Term weighting scheme for short-text classification: Twitter corpuses[J]. Neural Computing and Applications, 2019, 31(8): 3819-3831.

Cao J X, Xu S, Chen G J, et al. Regional topic discovery in online social networks [J]. Chinese Journal of Computers, 2017, 40(07): 1530-1542.

Yin H, Yang S, Li J. Detecting Topic and Sentiment Dynamics Due to COVID-19 Pandemic Using Social Media[C]. Advanced Data Mining and Applications, 2020: 610-623.

Yaqub U, Chun S A, Atluri V, et al. Analysis of political discourse on twitter in the context of the 2016 US presidential elections[J]. Government Information Quarterly, 2017, 34(4): 613-626.

Siering M, Deokar A V, Janze C. Disentangling consumer recommendations: Explaining and predicting airline recommendations based on online reviews[J]. Decision Support Systems, 2018, 107: 52-63.

Ren R, Wu D D, Liu T. Forecasting Stock Market Movement Direction Using Sentiment Analysis and Support Vector Machine[J]. IEEE Systems Journal, 2019, 13(1): 760-770.

Ekman P. An argument for basic emotions[J]. Cognition and Emotion, 1992, 6(3-4): 169-200.

Simonyan K, Zisserman A. Very Deep Convolutional Networks for Large-Scale Image Recognition[J]. arXiv e-prints, 2014: arXiv:1409.1556.

Devlin J, Chang M-W, Lee K, et al. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding[J]. arXiv e-prints, 2018: arXiv:1810.04805.

Asghar M Z, Subhan F, Ahmad H, et al. Senti‐eSystem: A sentiment‐based eSystem‐using hybridized fuzzy and deep neural network for measuring customer satisfaction[J]. Software: Practice and Experience, 2021, 51(3): 571-594.

He Y X, Sun S T, Niu F F, et al. A deep learning model with enhanced emotional semantics for microblog sentiment analysis[J]. Chinese Journal of Computers, 2017, 40(04): 773-790.

Kim Y. Convolutional Neural Networks for Sentence Classification[C]. EMNLP, 2014.

Yang Z, Yang D, Dyer C, et al. Hierarchical Attention Networks for Document Classification[C]. Proceedings of the 2016 conference of the North American chapter of the association for computational linguistics: human language technologies, 2016: 1480-1489.

Zhou P, Shi W, Tian J, et al. Attention-Based Bidirectional Long Short-Term Memory Networks for Relation Classification[C]. Proceedings of the 54th annual meeting of the association for computational linguistics 2016: 207-212.

Xu G, Yu Z, Yao H, et al. Chinese Text Sentiment Analysis Based on Extended Sentiment Dictionary[J]. IEEE Access, 2019, 7: 43749-43762.

Kaity M, Balakrishnan V. An integrated semi-automated framework for domain-based polarity words extraction from an unannotated non-English corpus[J]. The Journal of Supercomputing, 2020, 76(12): 9772-9799.

Wu F, Huang Y, Song Y, et al. Towards building a high-quality microblog-specific Chinese sentiment lexicon[J]. Decision Support Systems, 2016, 87: 39-49.

Zhang X, Zhao J, Lecun Y. Character-level convolutional networks for text classification[C]. Proceedings of the 28th International Conference on Neural Information Processing Systems - Volume 1, 2015: 649–657.

Yan L, Han J, Yue Y, et al. Sentiment Analysis of Short Texts Based on Parallel DenseNet[J]. Computers, Materials & Continua, 2021, 69(1): 51--65.

Gan C, Wang L, Zhang Z, et al. Sparse attention based separable dilated convolutional neural network for targeted sentiment analysis[J]. Knowledge-Based Systems, 2020, 188: 104827.

Zhou M, Liu D, Zheng Y, et al. A text sentiment classification model using double word embedding methods[J]. Multimedia Tools and Applications, 2020: 1-20.

Wang X, Liu Y, Sun C, et al. Predicting Polarities of Tweets by Composing Word Embeddings with Long Short-Term Memory[C]. Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), 2015: 1343-1353.

Lai S, Xu L, Liu K, et al. Recurrent convolutional neural networks for text classification[C]. Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence, 2015: 2267–2273.

Zuo E, Zhao H, Chen B, et al. Context-Specific Heterogeneous Graph Convolutional Network for Implicit Sentiment Analysis[J]. IEEE Access, 2020, 8: 37967-37975.

Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need[C]. Proceedings of the 31st International Conference on Neural Information Processing Systems, 2017: 6000–6010.

Yin R, Li P, Wang B. Sentiment Lexical-Augmented Convolutional Neural Networks for Sentiment Analysis[C]. 2017 IEEE Second International Conference on Data Science in Cyberspace (DSC), 2017: 630-635.

Luo L-X. Network text sentiment analysis method combining LDA text representation and GRU-CNN[J]. Personal and Ubiquitous Computing, 2019, 23(3): 405-412.

Alharbi A S M, De Doncker E. Twitter sentiment analysis with a deep neural network: An enhanced approach using user behavioral information[J]. Cognitive Systems Research, 2019, 54: 50-61.

Bahdanau D, Cho K, Bengio Y. Neural Machine Translation by Jointly Learning to Align and Translate[J]. CoRR, 2015, abs/1409.0473.

Cheng Y, Yao L, Xiang G, et al. Text Sentiment Orientation Analysis Based on Multi-Channel CNN and Bidirectional GRU With Attention Mechanism[J]. IEEE Access, 2020, 8: 134964-134975.

Chen K, Liang B, Ke W D, et al. Sentiment analysis of Chinese microblog based on multi-channel convolutional Neural Network[J]. Journal of Computer Research and Development, 2018, 55(05): 945-957.

Downloads

Published

10-07-2023

Issue

Section

Articles

How to Cite

Xu, Y., Li, X., Wang, H., & Chang, H. (2023). A Short Text Sentiment Analysis Model Combined with Attention Mechanism of Bidirectional Affective Words. Journal of Computing and Electronic Information Management, 11(1), 16-27. https://doi.org/10.54097/jceim.v11i1.9474

Similar Articles

1-10 of 113

You may also start an advanced similarity search for this article.