Aspect-based Sentiment Analysis based on Feature Extraction and Attention

: Aspect-based Sentiment Analysis (ABSA) is a fine-grained sentiment analysis task whose goal is to analyze the sentiment polarity of specific aspects of a text. Currently, Aspect-based Sentiment Analysis is widely used in opinion analysis and content recommendation. In this paper, we provide an overview of Aspect-based Sentiment Analysis methods such as Sentiment Dictionary, Machine Learning, and Deep Learning, introduce their latest research results, and analyze the advantages and disadvantages of using different methods. The current status of research on Aspect-based Sentiment Analysis at home and abroad is analyzed in depth by combing this class of methods, and the future development and trends of textual Aspect-based Sentiment Analysis are proposed.


Introduction
The internet is growing by leaps and bounds in the digital age and internet users are increasing at a breakneck pace, with around 900,000 people going online for the first time every day since 2018.Currently, the Internet has long been used by more than 5.3 billion people worldwide, equivalent to 66% of the world's population, and this number is expected to top 6.54 billion in 2025.Along with the increase in Internet usage comes data: IDC expects to generate 159.2 zettabytes of data in 2024 and 384.6 zettabytes in one year by 2028.This huge amount of data has greatly accelerated the development of fields such as Natural Language Processing (NLP) and Computer Vision (CV).
Natural Language Processing (NLP) is a branch of Artificial Intelligence that deals with the design and implementation of systems and algorithms capable of interacting through human language [1].In the field of NLP, there has been continuous research on sentimen analysis.Sentiment Analysis (SA) is mainly categorized into Chapterlevel Sentiment Analysis, Sentence-level Sentiment Analysis and Aspect-based Sentiment Analysis (ABSA) [2].Chapterlevel sentiment analysis analyzes the whole text in general, while sentence-level sentiment analysis analyzes the whole sentence as a basic unit, and both of them can only analyze the overall sentiment of a text or sentence, not the sentiment of multiple aspects, so they are classified as coarse-grained sentiment analysis.
Aspect-based Sentiment Analysis is a fine-grained sentiment analysis [3] that analyzes the sentiment of specific aspects of a sentence rather than the overall sentiment.For example, "This video card works great, but the video memory is too small."(Fig. 1) This sentence has two opposite sentiment polarities, positive and negative, with the sentiment polarity of "graphics card" being positive and the sentiment polarity of "video memory" being negative.Aspect-based Sentiment Analysis and Sentence-level Sentiment Analysis are difficult to accurately analyze the sentiments of the text.Therefore, traditional coarse-grained sentiment analysis methods can no longer meet the needs of sentiment analysis in today's complex semantic environment.Finer-grained sentiment analysis methods have become a key research issue in natural language processing [4].

Development of Aspect-based Sentiment Analysis
Aspect-based Sentiment Analysis (ABSA) is a fine-grained sentiment analysis task that can deal with texts that contain many sentiment polarities and are complex, and it contains several subtasks such as Aspect Sentiment Classification and Aspect Extraction [5][6] [7].The aspect sentiment classification task refers to classifying the sentiment polarity of target aspect words in the text.For example, "graphics card" and "memory" in Fig. 1 are the aspect words in this line of text, and in this sentence, the polarity of these two words is judged as positive and negative respectively, while the aspect extraction task is a part of the aspect sentiment classification task.task is the basis of the aspect sentiment categorization task, and all it has to do is to extract all the aspect words in the given text.Aspect-based Sentiment Analysis subtasks and examples are shown in Fig. 2.This paper focuses on the task of aspectual sentiment categorization.
Early research on Aspect-based Sentiment Analysis of text mainly used methods such as Machine Learning-based and Sentiment Dictionary-based because of their ability to capture feature representations with relative ease [8] [9].However, the input text must undergo extensive preprocessing as well as complex feature engineering before using these methods, and the effectiveness of the results they obtain often depends on the quality of the manually labeled data, which requires a large investment in labor costs when dealing with the massive amount of text data nowadays.In addition, these models do not or less take into account the semantic relations within the utterance, contextual relations and other elements.With the development of deep learning, neural networks are increasingly used in natural language processing (NLP) tasks [10].Deep learning based neural networks are able to convert text into entity vectors using word embeddings thus overcoming the problem of traditional machine learning that relies on manual extraction of text features.Recurrent Neural Networks (RNN) and Attention Mechanisms have been widely introduced for Aspect-based Sentiment Analysis, these methods have the advantage of being able to capture sequential and contextual semantic information of the text [11] and are therefore suitable for sentiment analysis.However, when multiple sentiment information is in the same text, Aspect-based Sentiment Analysis requires deeper and more subtle sentiment information than sentence-level or chapterlevel sentiment analysis [12].In this case, such models, which are difficult to fully utilize contextual and inter-word syntactic dependencies in sentences, are less adequate.
In this context, using pre-training models to enhance the semantic representation and thus deepen the model's learning of the syntactic dependencies of the text is a good approach.The "Embeddings from Language models (Elmo)" [13] and "Generative pre-training (GPT)" [14] models, which exhibit very good performance, and the subsequent emergence of the BERT model [15] also performs well, having topped the list on various natural language processing tasks.
Karimi et al. [16] used the standard BERT model to complete the ABSA processing task and found that the performance of the BERT model can significantly outperform the general neural network model, which can show that BERT is capable of performing the ABSA task to a certain extent.Although the BERT model has excellent performance, it has too many parameters, which leads to slow convergence of the model during training, and it is very easy to overfitting when the data set is not large enough.For this kind of problem, some researchers start at the pre-training stage, so that the model is first pre-trained on a larger sentence set sentiment analysis dataset, and then fine-tuned on the dataset of the ABSA task, so that the model performance can be improved [17][18].

Aspect-based Sentiment Analysis Approaches
Sentiment analysis is a technical means to probe deeply into the sentiment connotation of text, aiming to reveal the hidden emotions or attitudinal tendencies behind the text.The technique is widely used in several fields, such as public opinion monitoring, product review analysis, event monitoring, etc. Aspect-based Sentiment Analysis can be categorized into Document-level, Sentence-level and Aspectbased Sentiment Analysis according to the different granularity of analysis.Among them, document-level and sentence-level belong to coarse-grained sentiment analysis, while aspect-based Sentiment Analysis belongs to finegrained sentiment analysis.Document-level sentiment analysis is the earliest form of sentiment analysis, which targets the whole document, extracts all the sentiment words and performs sentiment classification statistics to determine the sentiment tendency of the whole document.Sentencelevel sentiment analysis is similar to the methodology of document-level sentiment analysis, but the object of analysis is narrowed down from the whole document to a single sentence.Aspect-based Sentiment Analysis is a sub-domain of sentiment analysis whose task is to mine multiple aspects of a given entity in a given text sentence.By extracting the sentiment words contained in these aspects, Aspect-based Sentiment Analysis is able to determine the sentiment polarity of that entity.The techniques used for different levels of sentiment analysis vary due to the granularity of the analysis.
Over the past years, the field of sentiment analysis has witnessed the emergence of multiple solution approaches.In the early stages of artificial intelligence, researchers commonly used sentiment dictionaries or traditional machine learning methods for sentiment analysis.However, with the advancement of AI technology, a single solution strategy can no longer fully satisfy the needs of sentiment analysis tasks.As a result, researchers have begun to explore ways to combine sentiment dictionaries with machine learning methods to optimize model performance by introducing specific constraint rules, with a view to obtaining more accurate sentiment analysis results.In recent years, the rise of deep learning has gradually replaced the traditional manual feature design and selection process as a mainstream technique in sentiment analysis tasks.Currently, there are diverse implementation strategies for sentiment analysis, including but not limited to: sentiment lexicon-based approaches, traditional machine learning techniques, and deep learning techniques.Each of these approaches has its own advantages, and together they have contributed to the advancement of sentiment analysis technology and the expansion of its application.

Sentiment Dictionary Based Approaches
In the early stages of sentiment analysis, researchers relied heavily on rules to construct sentiment dictionaries, which infer the sentiment tendency of a text by counting the frequency of sentiment words or phrases in the text.However, the effectiveness of this approach largely depends on the completeness of the sentiment dictionary.As an important resource reflecting the unstructured features of texts, sentiment dictionaries play a key role in sentiment analysis of texts.With the in-depth study of sentiment dictionaries by scholars at home and abroad, many sentiment dictionaries have been constructed artificially.Internationally, one of the earliest English sentiment dictionaries is SentiWordNet.in addition, there are some other well-known sentiment dictionaries, such as The General Inquirer (GI), Linguistic Inquiry and Word Count (LIWC), MPQA Subjectivity Cues Lexicon, etc.In China, the main representatives of Chinese sentiment dictionaries include NTUSD Dictionary, Zhiwang Dictionary HowNet and Sentiment Vocabulary Ontology Library.The construction and application of these dictionaries have greatly promoted the development of the field of sentiment analysis.
With the booming development of the social media industry, more and more scholars began to pay attention to the social sentiment in social media events, and this has given rise to numerous novel analysis methods based on sentiment dictionaries.Jia K et al [19] combined sentiment dictionaries and semantic rules to improve the method of calculating the sentiment intensity of sentiment words with different sentiment categories, and enhanced the effectiveness of sentiment classification.Alharthi K A et al [ 20] efficiently constructed a reusable sentiment dictionary on a large-scale system log and utilized the dictionary to detect errors and identify faulty components.In view of the colloquial, irregular and diverse nature of English social media texts, Rao L et al [21] proposed an innovative approach to text sentiment analysis that incorporates dictionary sentiment strength, expression patterns, and optimized semantic contextual features and constructs a sentiment analysis model based on these features.Based on word embedding, sentiment dictionary and fuzzy inference, Yan R et al [22] proposed a low-dimensional hybrid feature model and a sentimentenhanced inference model.
Although the sentiment dictionary approach is widely used in sentiment analysis, its effectiveness depends largely on the quality of the sentiment dictionary.Since the construction of sentiment dictionaries often requires significant human input and specialized knowledge, there are limitations to the process of their creation.In addition, with the development of society and the evolution of language, the existing sentiment dictionaries may not be able to cover specific sentiment expressions in emerging domains, and there are domain adaptation limitations.The rapid development of the Internet has led to the emergence of new words, and the sentiment tendencies of these new words may not be reflected in sentiment dictionaries in a timely manner, thus affecting the accuracy of sentiment analysis.The updating speed of the sentiment dictionary cannot keep up with the development of language, resulting in some new sentiment expressions not finding corresponding entries in the dictionary, which makes these expressions neglected or misjudged in the process of sentiment analysis.
Another problem is that sentiment dictionary-based approaches usually do not consider the contextual connection of words, resulting in the sentiment value of each word remaining static in different articles or sentences, and not being able to dynamically adapt to contextual changes.This approach has weak generalization ability and poor system migration, making it difficult to adapt to the needs of sentiment analysis in different domains and contexts.In view of the above limitations, more and more scholars are turning to machine learning-based sentiment analysis methods.Machine learning methods are able to automatically learn sentiment features in text by training on a large amount of labeled data, thus improving the accuracy and adaptability of sentiment analysis.These methods are usually able to better handle contextual information, dynamically adjust the sentiment values of words, and improve the generalization ability and system migration of sentiment analysis.

Machine Learning Based Approaches
The process of sentiment analysis based on traditional machine learning methods usually involves the following steps: first, sentiment words are identified and selected as features from the text, which are the basic units for expressing sentiment and are crucial for sentiment recognition and classification.Next, classical machine learning algorithms such as logistic regression, plain Bayes, and support vector machines are used to train on the already labeled feature dataset.These algorithms are able to learn the complex relationship between sentiment words and sentiment tendencies and construct a sentiment classification model.Finally, the trained models are utilized to predict sentiment tendencies on the text data within the test set.This prediction process relies on the quality of the selection of the text data used in the training phase as well as the accuracy of the sentiment labeling.Therefore, in order to obtain the desired classification results, not only do we need to carefully select the training texts to ensure their representativeness, but we also need to ensure the accuracy of the sentiment labeling.These two factors together determine the performance of the final sentiment classification model.
Pang B et al [23] pioneered the application of machine learning in the field of sentiment analysis by using machine learning methods to classify the sentiment of movie reviews.Aiming at the problem of over-reliance on sentiment word statistics for text sentiment analysis, Han K X et al [24] proposed an improved support vector machine (FK-SVM) method based on Fisher kernel function to fully consider lexical and potential semantic information in text.Without relying on manual labeling, Wang Y et al [25] proposed a new Aspect-based Sentiment Analysis method based on an incremental machine learning paradigm to achieve accurate machine labeling.Kurniawan C et al [26], on the other hand, used a Support Vector Machine approach for sentiment analysis, which was able to categorize student responses based on positive and negative polarities.These studies demonstrate the versatility and effectiveness of machine learning methods in the field of sentiment analysis.
The effectiveness of machine learning in sentiment analysis is highly dependent on the quality of feature representation, and the complexity of feature engineering itself often leads to difficulties in obtaining ideal classification results.In order to overcome the limitations of a single machine learning approach, many researchers and scholars have begun to explore the combination of sentiment dictionaries and machine learning in order to construct novel sentiment analysis models.
Li Z et al [27] constructed a sentiment dictionary for popup comments and proposed a new sentiment analysis method that combines a sentiment dictionary and a plain Bayesian model to analyze pop-up comments.Li H et al [28] proposed a new method that combines machine learning and dictionary techniques to construct a new dictionary by learning the labels of the classifiers of the training data and uses the new dictionary to correct the classifier's 's erroneous prediction results.
Compared to the traditional sentiment dictionary analysis methods, machine learning has demonstrated higher convenience and accuracy in the field of sentiment analysis.
It not only has excellent scalability and repeatability, but also achieves more efficient classification performance.However, the performance of machine learning algorithms is highly dependent on well-labeled training datasets.Obtaining largescale and high-quality training data usually requires expensive labor costs, and the subjectivity of human labeling may have some impact on the classification results.
Although combining machine learning with sentiment dictionaries is an improvement over a single approach, this approach is still susceptible to the quality of sentiment dictionaries and the labeling of text features.In addition, machine learning may fall into inefficient work when dealing with large-scale data, resulting in a sluggish model response.Therefore, in practical applications, more and more scholars have shifted the research focus of sentiment analysis to deep learning.Deep learning is able to reduce the reliance on manually labeled data by automatically extracting features and end-to-end model training, and usually provides more efficient performance when dealing with large-scale data.

Deep Learning Based Approaches
Traditional sentiment analysis methods are no longer able to meet the demand for efficient and high-quality sentiment classification tasks in the era of big data in terms of labor cost and time efficiency.Thanks to the rise of neural network technology, the field of sentiment analysis has been able to convert text sequences into word vectors through word embedding models, and then feed the vector sequences as inputs into neural network models, thus making the features rich in semantic information.With the successful application of attention and memory mechanisms in natural language processing tasks, deep learning-based sentiment analysis methods are able to achieve even better performance.The main advantage of this approach is that it does not need to define the features manually, but simulates the working mechanism of the human brain through deep learning, parses the text and extracts key features in a hierarchical way, and automatically adjusts the model parameters to optimize the output results.In addition, the neural network is able to learn the sentiment information embedded in the text on its own, significantly improving the overall accuracy of sentiment analysis.
In recent years, Recurrent Neural Networks (RNN) represent an end-to-end neural network model that has attracted attention in the field of Aspect-based Sentiment Analysis with its excellent automatic feature extraction capability and good classification results [29].However, RNN faces the problem of vanishing or exploding gradients when dealing with long sequence data, which limits its ability to capture long-term dependencies in sequences.In addition, RNN's limitations in memory affect its performance in processing complex sequence data, as it struggles to retain and utilize early history information.To overcome these challenges, researchers introduced gating mechanisms and proposed the Long Short-Term Memory Network (LSTM) model [30].The LSTM model is able to accurately filter and retain critical historical information through the synergistic action of forgetting gates, input gates, and output gates, effectively solving the problems of gradient vanishing and long-term dependencies, and thus demonstrating excellent performance when processing long sequence data.Therefore, LSTM models are widely used in aspect-based Sentiment Analysis tasks to improve the correctness and efficiency of sentiment analysis.
Based on the LSTM model Li et al [31] developed a "Semi-Supervised and Multi-Task Learning" (SEML) framework that uses embedding and LSTM layers to transfer sentiment information to document-level data in an ABSA model.Chen and Qian [32] developed a "Transmission Capsule Network Model (TransCap)" for sharing prior knowledge at the document level to an aspect-based sentiment analysis task."Transmission Capsule Network Model (TransCap)" for sharing document-level prior knowledge to Aspect-based Sentiment Analysis tasks.Su et al [33] discussed an ABSA model based on capsule networks and XLNet, which first obtains the relationship between sequences and aspects, and then generates aspect-specific representations The global relationship between aspects facilitates aspect awareness and ensures that pre-training of XLNet ultimately improves task uncertainty.Dong et al. [34] describe an Adaptive Recurrent Neural Network (AdaRNN) model for categorizing sentiment in Twitter comments.
However, all these models lose more or less implicit features computed in the middle part when using recurrent neural networks, while the structure of Elmo's [13] model can effectively retain the implicit features computed by each layer of the recurrent neural network and enable the model to learn deep contextualized word representations.However, the Elmo [13] model merges the implicit features of each layer in such a way that the key features of some layers do not receive enough attention.Therefore, we improve the feature merging approach of the Elmo [13] model merging by first highlighting the focus of each layer using the multi-head selfattention mechanism, and then merging the data of each layer using convolution, which allows the model to retain the important implicit features and reduces the influence of noise, and deepens the model's understanding of the input text.
In the field of text data processing, the attention mechanism is favored for its significant advantages, which can effectively obtain the importance of individual words in a sequence, so often researchers combine the attention mechanism with recurrent neural networks to solve the problem.Aiming at the problem of small dataset size for aspect-based sentiment analysis, Xu G et al [35] adopted the structure of Bidirectional Long and Short-Term Memory Network (Bi-LSTM) that combines the attention mechanism and introduced the transfer learning technique in order to realize more effective Aspectbased Sentiment Analysis.Although the attention mechanism has made significant progress in aspect-level sentiment categorization, the problem of information loss may still exist when dealing with longer or contextual sentences containing multiple sentiments.For this reason, Liao W et al [36] proposed a fine-grained attention-based phrase-aware neural network to address the problem of ignoring the importance of key phrases in a sentence.These studies show that deep learning models incorporating attention mechanisms have strong potential and application prospects in sentiment analysis tasks.
Graph Convolution Network (GCN) has attracted a lot of attention in the research field due to its remarkable ability to capture the structure and relationships between graph nodes as well as syntactic features between words.Numerous researchers have applied it to dependency tree analysis to account for syntactic constraints and long-term lexical dependencies.Zhang C et al [37] proposed to construct a graph convolution network on the dependency tree of a sentence to mine syntactic information and lexical dependencies.In order to enhance the model's focus on local syntactic context, Xiao L et al [38] constructed a context dynamic weighting mechanism incorporating syntactic awareness.Meng F et al [39] designed a weighted graph convolutional network (WGCN) based on feature combinations to mine rich syntactic information, which solved the problem of failing to make full use of the dependency analysis gained from dependency analysis when using a graph convolutional network on a dependency tree of a sentence.of syntactic information.Du J et al [40] designed an enhancement module that dynamically updates dependency edges based on dependency and contextual information.In order to fully analyze implicit knowledge and learn critical fine-grained implicit distance information through soft pruning strategies, Xiao L et al [41] designed an innovative Aspect-based Sentiment Analysis framework (SDGCN) that incorporates graph convolutional networks and enhances the model performance through distanceattention guidance.
With the emergence of the dependency-based dual graph (LD2G) model [42], the Interactive Dual Graph Convolutional Network (Inter-DGCN) model [43], the Syntactically and Semantically Enhanced Graph Convolutional Network (SSEGCN) model [44], and the combination of Local Global Contextual Guided Networks (LGCG) model [45], the research community has mitigated the noisy information introduced during the use of dependency trees problem and provides rich relational information between words.The emergence of these models has further advanced the field of sentiment analysis and provided new perspectives for understanding complex textual data.
And with the advent of the Transformer [46], the number of models that use attention mechanisms as their primary structure has increased dramatically.Zhao et al. [47] describe an "Attention Transfer Network (ATN)" model that incorporates both attention-guided and fused models.The former uses the attention weights of a document-level sentiment analysis model as learning signals for guiding the ABSA model during the training phase.Besides, some pretrained language models such as GPT [14] and BERT [15] shine on various text processing tasks and show extremely good performance.Currently most researchers tend to optimize or improve auto-coding based BERT models for ABSA tasks.Zhao et al [48] argued that the two subtasks of aspect extraction and aspect sentiment classification in ABSA are intrinsically linked, and also they found that the performance of aspect sentiment classification task can be further improved by utilizing the dependency syntax information of graph neural networks, so this research team utilized the Multi-head Attention to associate dependency trees with aspect extraction and use BERT to process the original text and aspect features proposed a multi-task learning model which improved the overall performance of the model on the ABSA task.Liu et al [49] found that severe domain shifting leads to poor results when using self-coding pre-training models such as BERT to process downstream tasks of ABSA.To address this problem Liu et al. used a stepwise training approach.First, the team selected instances related to the target domain from a large-scale pre-training dataset to align the instances between the pre-training and target domains, and used these data for the first stage of training.Then, a knowledge-based guidance strategy was introduced in the second phase of training to further bridge the domain gap at the knowledge level.Finally, in the third phase, the learner model is fine-tuned to better adapt its learned knowledge to the target dataset.

Conclusion
Aspect-based Sentiment Analysis is an important research area in the field of Natural Language Processing, which provides data support for decision making by identifying and evaluating the sentiment tendencies of specific aspects of a text.Research methods in this area have evolved from traditional approaches based on sentiment dictionaries, to machine learning algorithms, to deep learning models.Each of these approaches has its own characteristics, with sentiment dictionary approaches relying on pre-constructed sentiment vocabulary resources, machine learning approaches learning features automatically through algorithms, and deep learning approaches utilizing neural networks to capture complex text features.
With the increase in computational power and data volume, large models, various Transformers and their variants are increasingly used in Aspect-based Sentiment Analysis, which are capable of handling more complex tasks and providing more accurate results.In addition, multimodal Aspect-based Sentiment Analysis combines visual and auditory information other than text, such as videos, images, and speech, to provide a richer context for sentiment analysis and enhance the accuracy and usefulness of the analysis.

Fig 1 .
Fig 1. Example of a multifaceted, multilevel sentence

Fig 2 .
Fig 2. Aspect Extraction and Aspect Sentiment Classification