Sentiment analysis based on deep learning
Abstract:
The computational analysis of people’s views, sentiments, emotions, and attitudes toward entities
is known as sentiment analysis. Public opinion studies can provide us all with useful information.
Deep learning is a powerful machine learning technology that learns numerous layers of data
representations or features and generates cutting-edge prediction results. Deep learning has been
employed in sentiment analysis in recent years, in addition to its success in several application
domains. This study presents an overview of deep learning before doing a complete survey of its
existing uses in sentiment analysis.
Introduction:
Sentiment analysis (also known as opinion mining or emotion Artificial intelligence(AI) is
the use of biometrics, natural language processing, computational linguistics ,text analysis to
systematically identify, extract, quantify, and study affective states and subjective information.
The computational examination of people's opinions, attitudes, and feelings toward an entity is
known as sentiment analysis (SA) or opinion mining (OM). The computational study of people's
beliefs, attitudes, and emotions regarding an entity is known as SA or OM. The thing can stand
in for people, things, or things. Reviews are more likely to cover these subjects. The terms SA
and OM can both be used in the same sentence [1]. Sentiment analysis is necessary since it stores
data in an effective, affordable manner. Sentiment analysis helps you to resolve all real-world
problems and real-world circumstances. Sentiment analysis provides the comprehension
information related to public views, as it analyzes different tweets and reviews. It is a verified
tool for the prediction of many significant events such as the box office performance of movies
and general elections. The need for analyzing and structuring hidden information from social
media in the form of unstructured data has increased, which has increased the demand for
sentiment analysis [2]. Sentiment Analysis (SA), has recently drawn the attention of many
researchers because it is advantageous and necessary to analyze online text for market research,
psychological and sociological surveys, political polls, business intelligence, the improvement of
online shopping infrastructures, etc. Sentiment analysis, which examines several tweets and
reviews, provides [1]comprehension information on public opinions. Public reviews, which can
be found on various websites like Amazon and Yelp, are used to assess a certain entity, such as a
person, product, or location. One can classify opinions as either unfavorable, good, or neutral. To
automatically as certain the expressive orientation of user reviews, sentiment analysis is used [3].
Social media is a potent tool for people to communicate with one another and express their
feelings in the form of thoughts and viewpoints about any subject or piece of content, which
generates a tone of unstructured data. To research data and generate business insights, business
organizations must process and examine these feelings. Therefore, several machine learning and
NLP-based algorithms have been employed in the past to analyze these sentiments. However,
deep learning-based approaches are gaining a lot of traction because of their excellent
performance in recent years [4].
A) Features of Sentiment Analysis: By using polarities and combinations, emotions can contain a
wide range of featured values, such as trigrams and bigrams. As a result, sentiments are
evaluated as both positive and negative elements utilizing training strategies for the various
support vector machines. In sentiment analysis, neural networks are used to calculate labels
belongingness. The conditional dependencies between several edges and nodes of an acyclic
graph run by Bayesian networks are utilized to aid in data extraction at the context level. On
social media platforms, learning and data accuracy can be achieved by optimizing words and
sentences. Data tokenization produces the negative and positive characteristics of data at the
word root level. Techniques are being used to reduce sentiment analysis errors in order to
increase the precision of the data for social media [5].
B) Sentiment analysis is an interdisciplinary field: Since it draws from many other disciplines,
including computational linguistics, information retrieval, semantics, natural language
processing, artificial intelligence, and machine learning [6]. Three extraction levels are available
for categorizing sentiment analysis methodologies. three levels: (1) feature or aspect (2)
document (3) sentence [5].
C) Techniques for Sentiment Analysis: Two different types of methodologies, lexicon-based and
machine learning-based, are used in sentiment analysis [5].
a) Machine learning based techniques: The sentences and aspect levels are extracted when
using this kind of method. Parts of Speech (POS) tags, n-grams, bi-grams, unigrams, and
bag-of-words are among the features. There are three varieties of machine learning,
namely Nave Bayes, Support Vector Machine (SVM), and Maximum Entropy.
b) Lexicon based or corpus-based techniques: These methods for categorizing sentiments are
based on decision trees and include k-Nearest Neighbors (k-NN), Conditional Random Field
(CRF), Hidden Markov Model (HMM), Single Dimensional Classification (SDC), and Sequential
Minimal Optimization (SMO).
Three types of machine learning approaches exist: supervised, semi-supervised, and
unsupervised. Since this method can be automated and handle enormous amounts of data, it
is ideal for sentiment analysis [6].
Deep learning:
Deep Learning, which refers to Deep Neural Networks, was first proposed by G.E. Hinton in
2006 and is a step in the machine learning process [7]. The most popular topic in the field of
artificial intelligence nowadays is deep learning. Artificial neural networks are essentially a
field of machine learning techniques that were motivated by the composition and operation
of the brain. Deep neural networks gradually extract higher-level information from the raw
input using numerous layers of artificial neurons. Convolutional neural networks (CNN),
recurrent neural networks (RNN), recursive neural networks (RNC), deep belief networks
(DBN), and many more are among the networks used in deep learning. Text creation, vector
representation, word representation estimation, sentence classification, phrase modeling, and
feature display are all areas where neural networks are particularly helpful [8]. Deep learning
models have been widely used in the field of NLP and have shown great promise. The main
deep learning architectures and related techniques that have been applied to NLP tasks are
briefly described in the following sections. The scientific community lost interest in neural
networks in the late 1990s, mostly because they were thought to be only practical for
"shallow" neural networks (neural networks with one or two layers), which are difficult and
computationally expensive. However, in the last ten years, deep learning has achieved
breakthroughs and delivered state-of-the-art results in a wide range of application areas,
beginning with computer vision and progressing to speech recognition and, more recently,
NLP (Collobert et al., 2011; Goldberg, 2016) [9]. This section discusses many types of deep
learning approaches.
Figure 1. Differences between two classification approaches of sentiment polarity, machine learning (top),
and deep learning (bottom). Part of Speech (POS); Named Entity Recognition (NER); Term Frequency-Inverse
Document Frequency (TF-IDF).
Deep Neural Network (DNN):
A deep neural network is one that has more than two layers, some of which are buried. Deep
neural networks interpret input in a variety of ways using complex mathematical models. A
neural network is an adaptable model of outputs as functions of inputs that consists of several
layers: an input layer that includes input data; hidden layers that include processing nodes
known as neurons; and an output layer that includes one or more neurons, the outputs of
which are the network outputs [10]. A deep neural network (DNN), or deep net for short, is a
neural network with some amount of complexity, usually at least two layers. Deep nets use
advanced math modelling to process data in complex ways (Fig.1).
Convolutional Neural Network (CNN):
A convolutional neural network (CNN) is a sort of artificial neural network that analyses data
using perceptron’s, a machine learning unit technique. It is a sort of feedforward neural
network that was first used in computer vision. Its design is based on the human visual
cortex, which is a visual process in the animal brain. The visual cortex comprises many cells
that sense light in small and overlapping subregions of the visual fields known as receptive
fields. These cells function as local filters in the input space. CNN is made up of numerous
convolutional layers, each of which performs the function that the cells in the visual cortex
process [9]
Figure 2. Deep Neural Network (DNN)
Figure 3. A long short-term memory network. LSTM, long short-term memory
Recurrent Neural Networks (RNN):
Elman (1990) defined recurrent neural networks as neural networks in which the connections
between neurons form a directed loop. Unlike feedforward neural networks, RNNs can
process a sequence of inputs using their internal "memory," making them ideal for processing
sequential data. The term "memory" refers to the fact that RNN executes the same task for
each element of a sequence, with each output being dependent on all prior calculations, akin
to "remembering" information about what has been processed too far [11].
Figure 4. A long short-term memory network. LSTM, long short-term memory
Other Neural Networks:
A deep belief network (DBN) is one sort of deep neural network. It is made up of several
layers of a graphical model with both directed and undirected edges. Each network is made
up of numerous layers of hidden units, and each layer is linked to the next, but the units
inside a layer are not. A greedy layer-wise learning approach is used to train a DBN [12].
Literature Survey:
In order to increase ad relevance and user experience at the same time, Guang Qiu in year
2010 suggest a novel advertising method in this study termed Dissatisfaction-oriented
Advertising based on Sentiment Analysis (DASA). Guang Qiu use
“automotvieforums.com” dataset and Rule-Based algorithm for this study[13].
Ahmed Sulaiman M Alharbi proposes a neural network model that includes user
behavioural data within a given document (tweet). This paper employs a Convolutional
Neural Network as its neural network (CNN). The system is tested using two datasets
from the SemEval-2016 Workshop. Going beyond the content of a document (tweet) is
helpful in sentiment analysis because it provides the classifier with a detailed
understanding of the task, as proved by Naive Bayes and Support Vector Machines [14].
Pushpak Bhattacharyya can perform sentiment classification on any text genre with few
constraints and no task-specific or data-specific manual feature engineering. We describe
some of the various approaches used in sentiment analysis research in this survey. Deep
learning models, particularly CNN and LSTM variants, exceed on the large majority of
datasets [15].
ConvLstm is a neural network architecture proposed by Abdalraouf Hassan that employs
Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM) on top of
pre-trained word vectors. In our experiments, ConvLstm uses LSTM as a pooling layer
replacement in CNN to reduce the loss of detailed local information and capture long-term
dependencies in a sentence sequence. The proposed model was validated using two
sentiment datasets: IMDB and Stanford Sentiment Treebank [16].
Bas Heerschop proposes Pathos, a framework that performs (partially) sentiment analysis
on documents based on their discourse structure. While most sentiment analysis research
focuses on the main components of a sentiment classifier (e.g., word sentiment scoring,
topic classification, negation, and intensifiers), little research has been done on analysing
text discourse structure to identify text spans that are more important for the overall
sentiment in a document. Bas Heerschop compares three methods for categorising texts
into important and minor sections. One method is based on a word's position in a text. The
other two methods make use of discourse structure in natural language text by
distinguishing between (sentence-level) nuclei and satellites, or by identifying and
exploiting (sentence-level) RST relation types. The goal of this paper is to give insights on
how information can be extracted from structural aspects of content to advance the state-
of-the-art in sentiment analysis[17].
Yung-Ming Li proposes a framework for providing a concise numerical summary of
opinions expressed on such platforms. The proposed framework is intended to handle the
following tasks: detection of trending topics, opinion classification, credibility assessment,
and numeric summarization. To demonstrate the effectiveness of the proposed framework,
an experiment is conducted on Twitter, the largest microblog website[18].
In his study, Rodrigo Moraes presents an empirical comparison of SVM and ANN for
document-level sentiment analysis. We explain the prerequisites, models that result, and
scenarios where both approaches attain higher levels of classification accuracy. In a
classic bag-of-words model, we use a standard evaluation context with popular supervised
approaches for feature selection and weighting[19].
XueBai offers a heuristic search-enhanced Markov blanket model capable of capturing
word dependencies and providing a vocabulary suitable for sentiment extraction[20].
To boost online contextual advertising, Teng-KaiFan proposes employing text mining
techniques to determine bloggers' direct personal interests. The suggested Blogger-Centric
Contextual Advertising (BCCA) architecture intends to combine contextual advertising
matching with text mining to choose and rank adverts linked to personal interests
indicated in a blog[21].
Table Review:
Studies Year Dataset Algorithm
Dissatisfaction- 2010 “automotvieforums.com” Rule-Based
oriented algorithm
Advertising
based on
Sentiment
Analysis
Twitter 2018 SemEval 2016 workshop CNN
Sentiment
Analysis with a
Deep Neural
Network: An
Enhanced
Approach using
User
Behavioural
Information
A survey of 2016 sentiment Treebank dataset, CNN,
sentiment movie reviews, MPQA, and RNTN,
analysis and customer reviews RNN, LSTM
deep learning
Deep Learning 2017 Stanford Large Movie CNN,
Approach for Review, IMDB, Stanford LSTM, on
Sentiment Sentiment Treebank, SSTb top of
Analysis of Short pretrained
Texts. word vectors
Polarity Analysis 2011 IMDB Lexicon-
of Texts using based,
Discourse semantic
Structure
Deriving market 2013 twitter SVM
intelligence from
microblogs
Document-level 2013 amazon.com SVM,ANN
sentiment
classification: An
empirical
comparison
between SVM
and ANN
Predicting 2010 IMDB Markov
consumer Blanket,
sentiments from SVM, NB,
online text ME
Blogger-Centric 2011 amazon.com, epinions.com, SVM, Chi-
Contextual blogs, SNS square
Advertising
Conclusion:
In this paper, we introduced the backbone of deep learning models and related techniques
that have been applied to sentiment analysis for social network data. We first introduced
sentiments through their application, and then we applied them. Deep learning applied to
sentiment analysis has recently been a popular research topic. We introduced various deep
learning architectures and their applications in sentiment analysis in this paper. Many of
these deep learning algorithms have produced trimming sentiment analysis results. We
believe that as deep learning research and applications develop, there will be more
exciting deep learning research for sentiment analysis in the near term.
References:
[1] W. Medhat, A. Hassan, and H. Korashy, “Sentiment analysis algorithms and applications:
A survey,” Ain Shams Eng. J., vol. 5, no. 4, pp. 1093–1113, Dec. 2014, doi:
10.1016/j.asej.2014.04.011.
[2] Q. Tul et al., “Sentiment Analysis Using Deep Learning Techniques: A Review,” Int. J.
Adv. Comput. Sci. Appl., vol. 8, no. 6, 2017, doi: 10.14569/ijacsa.2017.080657.
[3] S. Bahrainian and A. Dengel, “Sentiment Analysis using Sentiment Features,” 2013, doi:
10.1109/WI-IAT.2013.145.
[4] A. Yadav and D. K. Vishwakarma, “Sentiment analysis using deep learning architectures:
a review,” Artif. Intell. Rev., vol. 53, no. 6, pp. 4335–4385, 2020, doi: 10.1007/s10462-
019-09794-5.
[5] J. Singh, G. Singh, and R. Singh, “A review of sentiment analysis techniques for
opinionated web text,” CSI Trans. ICT, vol. 4, no. 2–4, pp. 241–247, 2016, doi:
10.1007/s40012-016-0107-y.
[6] E. Aydo and M. A. Akcayol, “Aydogan2016.Pdf,” 2016.
[7] M. Y. Day and C. C. Lee, “Deep learning for financial sentiment analysis on finance news
providers,” Proc. 2016 IEEE/ACM Int. Conf. Adv. Soc. Networks Anal. Mining, ASONAM
2016, no. 1, pp. 1127–1134, 2016, doi: 10.1109/ASONAM.2016.7752381.
[8] Y. Zhang, M. J. Er, R. Venkatesan, N. Wang, and M. Pratama, “Sentiment classification
using Comprehensive Attention Recurrent models,” Proc. Int. Jt. Conf. Neural Networks,
vol. 2016-October, pp. 1562–1569, 2016, doi: 10.1109/IJCNN.2016.7727384.
[9] L. Zhang, S. Wang, and B. Liu, “Deep learning for sentiment analysis: A survey,” Wiley
Interdiscip. Rev. Data Min. Knowl. Discov., vol. 8, no. 4, pp. 1–25, 2018, doi:
10.1002/widm.1253.
[10] C. C. Aggarwal, Neural Networks and Deep Learning. 2018. doi: 10.1007/978-3-319-
94463-0.
[11] A. Chamekh, M. Mahfoudh, and G. Forestier, “Sentiment Analysis Based on Deep
Learning in E-Commerce,” Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif.
Intell. Lect. Notes Bioinformatics), vol. 13369 LNAI, pp. 498–507, 2022, doi:
10.1007/978-3-031-10986-7_40.
[12] P. Ruangkanokmas, T. Achalakul, and K. Akkarajitsakul, “Deep Belief Networks with
Feature Selection for Sentiment Classification,” Proc. - Int. Conf. Intell. Syst. Model.
Simulation, ISMS, vol. 0, pp. 9–14, 2016, doi: 10.1109/ISMS.2016.9.
[13] G. Qiu, X. He, F. Zhang, Y. Shi, J. Bu, and C. Chen, “DASA: Dissatisfaction-oriented
Advertising based on Sentiment Analysis,” Expert Syst. Appl., vol. 37, no. 9, pp. 6182–
6191, 2010, doi: 10.1016/j.eswa.2010.02.109.
[14] A. S. M. Alharbi and E. de Doncker, “Twitter sentiment analysis with a deep neural
network: An enhanced approach using user behavioral information,” Cogn. Syst. Res., vol.
54, pp. 50–61, 2019, doi: 10.1016/j.cogsys.2018.10.001.
[15] M. Indhraom Prabha and G. Umarani Srikanth, “Survey of Sentiment Analysis Using
Deep Learning Techniques,” Proc. 1st Int. Conf. Innov. Inf. Commun. Technol. ICIICT
2019, 2019, doi: 10.1109/ICIICT1.2019.8741438.
[16] A. Hassan and A. Mahmood, “Deep Learning approach for sentiment analysis of short
texts,” 2017 3rd Int. Conf. Control. Autom. Robot. ICCAR 2017, pp. 705–710, 2017, doi:
10.1109/ICCAR.2017.7942788.
[17] B. Heerschop, F. Goossen, A. Hogenboom, F. Frasincar, U. Kaymak, and F. De Jong,
“Polarity analysis of texts using discourse structure,” Int. Conf. Inf. Knowl. Manag. Proc.,
pp. 1061–1070, 2011, doi: 10.1145/2063576.2063730.
[18] Y. M. Li and T. Y. Li, “Deriving market intelligence from microblogs,” Decis. Support
Syst., vol. 55, no. 1, pp. 206–217, 2013, doi: 10.1016/j.dss.2013.01.023.
[19] R. Moraes, J. F. Valiati, and W. P. Gavião Neto, “Document-level sentiment
classification: An empirical comparison between SVM and ANN,” Expert Syst. Appl., vol.
40, no. 2, pp. 621–633, 2013, doi: 10.1016/j.eswa.2012.07.059.
[20] X. Bai, “Predicting consumer sentiments from online text,” Decis. Support Syst., vol. 50,
no. 4, pp. 732–742, 2011, doi: 10.1016/j.dss.2010.08.024.
[21] T. Fan and C.-H. Chang, “Blogger-Centric Contextual Advertising Categories and Subject
Descriptors,” Expert Syst. Appl., vol. 3, no. 38, pp. 1777–1788, 2011.
