Sentiment Analysis: A Definitive Guide Try MonkeyLearn

What Is Sentiment Analysis? Sentiment Analysis Examples How Does It Work? Sentiment An

Sentiment analysis (or opinion mining) is a natural language processing (NLP) technique
used to determine whether data is positive, negative or neutral. Sentiment analysis is
often performed on textual data to help businesses monitor brand and product
sentiment in customer feedback, and understand customer needs.

Start analyzing your text for sentiment TRY NOW

Learn more about how sentiment analysis works, its challenges, and how you can use
sentiment analysis to improve processes, decision-making, customer satisfaction and
more.

Once you’re familiar with the basics, get started with easy-to-use sentiment analysis tools
that are ready to use right o the bat.
1. What Is Sentiment Analysis?

2. Types of Sentiment Analysis Try MonkeyLearn

3. Why Is Sentiment Analysis Important?

4. Sentiment Analysis Examples & Break Down of Trustpilot Reviews

5. How Does Sentiment Analysis Work?

6. Sentiment Analysis Challenges

7. Sentiment Analysis Applications

8. Sentiment Analysis Tools & Tutorials

9. Sentiment Analysis Research & Courses

What Is Sentiment Analysis?

Sentiment analysis is the process of detecting positive or negative sentiment in text. It’s
often used by businesses to detect sentiment in social data, gauge brand reputation, and
understand customers.
Types of Sentiment Analysis Try MonkeyLearn

Sentiment analysis focuses on the polarity of a text (positive, negative, neutral) but it also
goes beyond polarity to detect speci c feelings and emotions (angry, happy, sad, etc),
urgency (urgent, not urgent) and even intentions (interested v. not interested).

Depending on how you want to interpret customer feedback and queries, you can de ne
and tailor your categories to meet your sentiment analysis needs. In the meantime, here
are some of the most popular types of sentiment analysis:

Graded Sentiment Analysis

If polarity precision is important to your business, you might consider expanding your
polarity categories to include di erent levels of positive and negative:

Very positive

Positive

Neutral

Negative

Very negative

This is usually referred to as graded or ne-grained sentiment analysis, and could be used
to interpret 5-star ratings in a review, for example:

Very Positive = 5 stars

Very Negative = 1 star

Emotion detection

Emotion detection sentiment analysis allows you to go beyond polarity to detect

emotions, like happiness, frustration, anger, and sadness.

Many emotion detection systems use lexicons (i.e. lists of words and the emotions they
convey) or complex machine learning algorithms.
One of the downsides of using lexicons is that people express emotions in di erent ways.
Some words that typically express anger, like bad or kill (e.g. your productTry
is so bad or your
MonkeyLearn
customer support is killing me) might also express happiness (e.g. this is bad ass or you are
killing it).

Aspect-based Sentiment Analysis

Usually, when analyzing sentiments of texts you’ll want to know which particular aspects
or features people are mentioning in a positive, neutral, or negative way.

That's where aspect-based sentiment analysis can help, for example in this product
review: "The battery life of this camera is too short", an aspect-based classi er would be
able to determine that the sentence expresses a negative opinion about the battery life of
the product in question.

Multilingual sentiment analysis

Multilingual sentiment analysis can be di cult. It involves a lot of preprocessing and

resources. Most of these resources are available online (e.g. sentiment lexicons), while
others need to be created (e.g. translated corpora or noise detection algorithms), but
you’ll need to know how to code to use them.

Alternatively, you could detect language in texts automatically with a language classi er,
then train a custom sentiment analysis model to classify texts in the language of your
choice.

Why Is Sentiment Analysis Important?

Since humans express their thoughts and feelings more openly than ever before,
sentiment analysis is fast becoming an essential tool to monitor and understand
sentiment in all types of data.

Automatically analyzing customer feedback, such as opinions in survey responses and

social media conversations, allows brands to learn what makes customers happy or
frustrated, so that they can tailor products and services to meet their customers’ needs.
For example, using sentiment analysis to automatically analyze 4,000+ open-ended
responses in your customer satisfaction surveys could help you discoverTry
why customers
MonkeyLearn
are happy or unhappy at each stage of the customer journey.

Maybe you want to track brand sentiment so you can detect disgruntled customers
immediately and respond as soon as possible. Maybe you want to compare sentiment
from one quarter to the next to see if you need to take action. Then you could dig deeper
into your qualitative data to see why sentiment is falling or rising.

The overall bene ts of sentiment analysis include:

Sorting Data at Scale

Can you imagine manually sorting through thousands of tweets, customer support
conversations, or surveys? There’s just too much business data to process manually.
Sentiment analysis helps businesses process huge amounts of unstructured data in an
e cient and cost-e ective way.

Real-Time Analysis

Sentiment analysis can identify critical issues in real-time, for example is a PR crisis on
social media escalating? Is an angry customer about to churn? Sentiment analysis models
can help you immediately identify these kinds of situations, so you can take action right
away.

Consistent criteria

It’s estimated that people only agree around 60-65% of the time when determining the
sentiment of a particular text. Tagging text by sentiment is highly subjective, in uenced
by personal experiences, thoughts, and beliefs.

By using a centralized sentiment analysis system, companies can apply the same criteria
to all of their data, helping them improve accuracy and gain better insights.

The applications of sentiment analysis are endless. So, to help you understand how
sentiment analysis could bene t your business, let’s take a look at some examples of
texts that you could analyze using sentiment analysis.
Then, we’ll jump into a real-world example of how Chewy, a pet supplies company, was
able to gain a much more nuanced (and useful!) understanding of their reviews through
Try MonkeyLearn
the application of sentiment analysis.

Sentiment Analysis Examples

To understand the goal and challenges of sentiment analysis, here are some examples:

Basic examples of sentiment analysis data

Net ix has the best selection of lms

Hulu has a great UI

I dislike like the new crime series

I hate waiting for the next series to come out

More challenging examples of sentiment analysis

I do not dislike horror movies. (phrase with negation)

Disliking horror movies is not uncommon. (negation, inverted word order)

Sometimes I really hate the show. (adverbial modi es the sentiment)

I love having to wait two months for the next series to come out! ( sarcasm)

The nal episode was surprising with a terrible twist at the end (negative term used in
a positive way)

The lm was easy to watch but I would not recommend it to my friends. (di cult to
categorize)

I LOL’d at the end of the cake scene (often hard to understand new terms)

Now, let’s take a look at some real reviews on Trustpilot and see how MonkeyLearn’s
sentiment analysis tools fare when it comes to recognizing and categorizing sentiment.

Case Study: Sentiment analysis on

TrustPilot Reviews
Chewy is a pet supplies company – an industry with no shortage of competition, so
providing a superior customer experience (CX) to their customers can beTry
a massive
MonkeyLearn
di erence maker.

For this reason, online reviews can be an extremely valuable source of information to
gain customer insights to improve their CX. Chewy has thousands of reviews in TrustPilot,
this is what their review archive looks like:

Via TrustPilot

It is easy to draw a general conclusion about Chewy’s relative success from this alone -
82% of responses being excellent is a great starting place.

But TrustPilot’s results alone fall short if Chewy’s goal is to improve its services. This
perfunctory overview fails to provide actionable insight, the cornerstone, and end goal, of
e ective sentiment analysis.

If Chewy wanted to unpack the what and why behind their reviews, in order to further
improve their services, they would need to analyze each and every negative review at a
granular level.

But with sentiment analysis tools, Chewy could plug in their 5,639 (at the time) TrustPilot
reviews to gain instant sentiment analysis insights.
We uploaded and analyzed Chewy’s reviews to MonkeyLearn’s all-in-one data analysis
and visualization studio to generate the following dashboard: Try MonkeyLearn

Chewy TrustPilot Reviews Sample

Feel free to click this link to peruse the results at your leisure - as this sample dashboard
is a public demo, you can click through and explore the inputs and lters at work yourself.

While there is a ton more to explore, in this breakdown we are going to focus on four
sentiment analysis data visualization results that the dashboard has visualized for us.

1. Overall Sentiment

2. Sentiment over Time

3. Sentiment by Rating

4. Sentiment by Topic

1. Overall sentiment
We’ll begin by pulling the relevant graphic from the above dashboard.
 Try MonkeyLearn

You’ll notice that these results are very di erent from TrustPilot’s overview (82%
excellent, etc). This is because MonkeyLearn’s sentiment analysis AI performs advanced
sentiment analysis, parsing through each review sentence by sentence, word by word.

What you are left with is an accurate assessment of everything customers have written,
rather than a simple tabulation of stars. This analysis can point you towards friction
points much more accurately and in much more detail.

Read up on the mechanics of how sentiment analysis works below.

2. Sentiment over time

Here’s our handy-dandy sentiment over time graph, blown up:

This data visualization sample is classic temporal datavis, a datavis type that tracks results
and plots them over a period of time.

This graph expands on our Overall Sentiment data - it tracks the overall proportion of
positive, neutral, and negative sentiment in the reviews from 2016 to 2021.

This graph informs the gradual change in the content of their written reviews over this
ve year period. For instance, negative responses went down from 2019-2020, then
jumped back up to previous levels in 2021.
3. Sentiment by rating
Try MonkeyLearn

Now we jump to something that anchors our text-based sentiment to TrustPilot’s earlier
results.

By taking each TrustPilot category from 1-Bad to 5-Excellent, and breaking down the text
of the written reviews from the scores you can derive the above graphic.

Looking at the results, and courtesy of taking a deeper look at the reviews via sentiment
analysis, we can draw a couple interesting conclusions right o the bat.

1. TrustPilots results aren’t useless - the better reviews have higher proportions of
positive sentiment and the worse reviews have more negative sentiment. But, all
reviews contain a little bit of all types of sentiment - we’ve learned that our reviews are
nuanced and thus likely have even more hidden insight for us!

2. Our reviews are polarized. They skew in amounts towards 5 and 1.

These quick takeaways point us towards goldmines for future analysis. Namely, the
positive sentiment sections of negative reviews and the negative section of positive ones,
and the 2 - 4 reviews (why do they feel the way they do, how could we improve their
scores?).

4. Sentiment by Topic
 Try MonkeyLearn

Finally, we can take a look at Sentiment by Topic to begin to illustrate how sentiment
analysis can take us even further into our data.

The above chart applies product-linked text classi cation in addition to sentiment
analysis to pair given sentiment to product/service speci c features, this is known as
aspect-based sentiment analysis.

This means we can know how our customers feel about what, helping us zero in and x
speci c pain points or issues.

These are all great jumping o points designed to visually demonstrate the value of
sentiment analysis - but they only scratch the surface of its true power.

Read on for a step-by-step walkthrough of how sentiment analysis works.

How Does Sentiment Analysis Work?

 Try MonkeyLearn

Sentiment analysis, otherwise known as opinion mining, works thanks to natural

language processing (NLP) and machine learning algorithms, to automatically determine
the emotional tone behind online conversations.

There are di erent algorithms you can implement in sentiment analysis models,
depending on how much data you need to analyze, and how accurate you need your
model to be. We’ll go over some of these in more detail, below.

Sentiment analysis algorithms fall into one of three buckets:

Rule-based: these systems automatically perform sentiment analysis based on a set

of manually crafted rules.

Automatic: systems rely on machine learning techniques to learn from data.

Hybrid systems combine both rule-based and automatic approaches.

Rule-based Approaches
Usually, a rule-based system uses a set of human-crafted rules to help identify
subjectivity, polarity, or the subject of an opinion.

These rules may include various NLP techniques developed in computational linguistics,
such as:

Stemming, tokenization, part-of-speech tagging and parsing.

Lexicons (i.e. lists of words and expressions).

Here’s a basic example of how a rule-based system works:

1. De nes two lists of polarized words (e.g. negative words such as bad, worst, ugly, etc
and positive words such as good, best, beautiful, etc).

2. Counts the number of positive and negative words that appear in a given text.
3. If the number of positive word appearances is greater than the number of negative
word appearances, the system returns a positive sentiment, and vice Try
versa. If the
MonkeyLearn
numbers are even, the system will return a neutral sentiment.

Rule-based systems are very naive since they don't take into account how words are
combined in a sequence. Of course, more advanced processing techniques can be used,
and new rules added to support new expressions and vocabulary. However, adding new
rules may a ect previous results, and the whole system can get very complex. Since rule-
based systems often require ne-tuning and maintenance, they’ll also need regular
investments.

Automatic Approaches
Automatic methods, contrary to rule-based systems, don't rely on manually crafted rules,
but on machine learning techniques. A sentiment analysis task is usually modeled as a
classi cation problem, whereby a classi er is fed a text and returns a category, e.g.
positive, negative, or neutral.

Here’s how a machine learning classi er can be implemented:

 Try MonkeyLearn

The Training and Prediction Processes

In the training process (a), our model learns to associate a particular input (i.e. a text) to
the corresponding output (tag) based on the test samples used for training. The feature
extractor transfers the text input into a feature vector. Pairs of feature vectors and tags
(e.g. positive, negative, or neutral) are fed into the machine learning algorithm to generate
a model.

In the prediction process (b), the feature extractor is used to transform unseen text
inputs into feature vectors. These feature vectors are then fed into the model, which
generates predicted tags (again, positive, negative, or neutral).

Feature Extraction from Text

The rst step in a machine learning text classi er is to transform the text extraction or
text vectorization, and the classical approach has been bag-of-words or bag-of-ngrams
with their frequency.

More recently, new feature extraction techniques have been applied based on word
embeddings (also known as word vectors). This kind of representations makes it possible
for words with similar meaning to have a similar representation, which can improve the
performance of classi ers.

Classi cation Algorithms

The classi cation step usually involves a statistical model like Naïve Bayes, Logistic
Regression, Support Vector Machines, or Neural Networks:

Naïve Bayes: a family of probabilistic algorithms that uses Bayes’s Theorem to predict
the category of a text.

Linear Regression: a very well-known algorithm in statistics used to predict some value
(Y) given a set of features (X).
 Support Vector Machines: a non-probabilistic model which uses a representation of
text examples as points in a multidimensional space. Examples of di Try
erent categories
MonkeyLearn
(sentiments) are mapped to distinct regions within that space. Then, new texts are
assigned a category based on similarities with existing texts and the regions they’re
mapped to.

Deep Learning: a diverse set of algorithms that attempt to mimic the human brain, by
employing arti cial neural networks to process data.

Hybrid Approaches
Hybrid systems combine the desirable elements of rule-based and automatic techniques
into one system. One huge bene t of these systems is that results are often more
accurate.

Sentiment Analysis Challenges

Sentiment analysis is one of the hardest tasks in natural language processing because
even humans struggle to analyze sentiments accurately.

Data scientists are getting better at creating more accurate sentiment classi ers, but
there’s still a long way to go. Let’s take a closer look at some of the main challenges of
machine-based sentiment analysis:

1. Subjectivity & Tone

2. Context & Polarity

3. Irony & Sarcasm

4. Comparisons

5. Emojis

6. De ning Neutral

7. Human Annotator Accuracy

Subjectivity and Tone

There are two types of text: subjective and objective. Objective texts do not contain
explicit sentiments, whereas subjective texts do. Say, for example, you intend to analyze
Try MonkeyLearn
the sentiment of the following two texts:

The package is nice.

The package is red.

Most people would say that sentiment is positive for the rst one and neutral for the
second one, right? All predicates (adjectives, verbs, and some nouns) should not be
treated the same with respect to how they create sentiment. In the examples above, nice
is more subjective than red.

Context and Polarity

All utterances are uttered at some point in time, in some place, by and to some people,
you get the point. All utterances are uttered in context. Analyzing sentiment without
context gets pretty di cult. However, machines cannot learn about contexts if they are
not mentioned explicitly. One of the problems that arise from context is changes in
polarity. Look at the following responses to a survey:

Everything about it.

Absolutely nothing!

Imagine the responses above come from answers to the question What did you like about
the event? The rst response would be positive and the second one would be negative,
right? Now, imagine the responses come from answers to the question What did you
DISlike about the event? The negative in the question will make sentiment analysis change
altogether.

A good deal of preprocessing or postprocessing will be needed if we are to take into

account at least part of the context in which texts were produced. However, how to
preprocess or postprocess data in order to capture the bits of context that will help
analyze sentiment is not straightforward.

Irony and Sarcasm

When it comes to irony and sarcasm, people express their negative sentiments using
positive words, which can be di cult for machines to detect without having a thorough
Try MonkeyLearn
understanding of the context of the situation in which a feeling was expressed.

For example, look at some possible answers to the question, Did you enjoy your shopping
experience with us?

Yeah, sure. So smooth!

Not one, but many!

What sentiment would you assign to the responses above? The rst response with an
exclamation mark could be negative, right? The problem is there is no textual cue that will
help a machine learn, or at least question that sentiment since yeah and sure often
belong to positive or neutral texts.

How about the second response? In this context, sentiment is positive, but we’re sure you
can come up with many di erent contexts in which the same response can express
negative sentiment.

Comparisons
How to treat comparisons in sentiment analysis is another challenge worth tackling. Look
at the texts below:

This product is second to none.

This is better than older tools.

This is better than nothing.

The rst comparison doesn’t need any contextual clues to be classi ed correctly. It’s clear
that it’s positive.

The second and third texts are a little more di cult to classify, though. Would you classify
them as neutral, positive, or even negative? Once again, context can make a di erence. For
example, if the ‘older tools’ in the second text were considered useless, then the second
text is pretty similar to the third text.
Emojis
Try MonkeyLearn
There are two types of emojis according to Guibon et al.. Western emojis (e.g. :D) are
encoded in only one or two characters, whereas Eastern emojis (e.g. ¯ \ (ツ) / ¯) are a
longer combination of characters of a vertical nature. Emojis play an important role in the
sentiment of texts, particularly in tweets.

You’ll need to pay special attention to character-level, as well as word-level, when

performing sentiment analysis on tweets. A lot of preprocessing might also be needed.
For example, you might want to preprocess social media content and transform both
Western and Eastern emojis into tokens and whitelist them (i.e. always take them as a
feature for classi cation purposes) in order to help improve sentiment analysis
performance.

Here’s a quite comprehensive list of emojis and their unicode characters that may come
in handy when preprocessing.

De ning Neutral
De ning what we mean by neutral is another challenge to tackle in order to perform
accurate sentiment analysis. As in all classi cation problems, de ning your categories -
and, in this case, the neutral tag- is one of the most important parts of the problem. What
you mean by neutral, positive, or negative does matter when you train sentiment analysis
models. Since tagging data requires that tagging criteria be consistent, a good de nition
of the problem is a must.

Here are some ideas to help you identify and de ne neutral texts:

1. Objective texts. So called objective texts do not contain explicit sentiments, so you
should include those texts into the neutral category.

2. Irrelevant information. If you haven’t preprocessed your data to lter out irrelevant
information, you can tag it neutral. However, be careful! Only do this if you know how
this could a ect overall performance. Sometimes, you will be adding noise to your
classi er and performance could get worse.

3. Texts containing wishes. Some wishes like, I wish the product had more integrations
are generally neutral. However, those including comparisons like, I wish the product
 were better are pretty di cult to categorize
Try MonkeyLearn

Human Annotator Accuracy

Sentiment analysis is a tremendously di cult task even for humans. On average, inter-
annotator agreement (a measure of how well two (or more) human labelers can make the
same annotation decision) is pretty low when it comes to sentiment analysis. And since
machines learn from labeled data, sentiment analysis classi ers might not be as precise
as other types of classi ers.

Still, sentiment analysis is worth the e ort, even if your sentiment analysis predictions are
wrong from time to time. By using MonkeyLearn’s sentiment analysis model, you can
expect correct predictions about 70-80% of the time you submit your texts for
classi cation.

If you are new to sentiment analysis, then you’ll quickly notice improvements. For typical
use cases, such as ticket routing, brand monitoring, and VoC analysis, you’ll save a lot of
time and money on tedious manual tasks.

Sentiment Analysis Use Cases &

Applications
 Try MonkeyLearn

The applications of sentiment analysis are endless and can be applied to any industry,
from nance and retail to hospitality and technology. Below, we’ve listed some of the
most popular ways that sentiment analysis is being used in business:

1. Social Media Monitoring

2. Brand Monitoring

3. Voice of customer (VoC)

4. Customer Service

5. Market Research

Social Media Monitoring

Sentiment analysis is used in social media monitoring, allowing businesses to gain
insights about how customers feel about certain topics, and detect urgent issues in real
time before they spiral out of control.

On the fateful evening of April 9th, 2017, United Airlines forcibly removed a passenger
from an overbooked ight. The nightmare-ish incident was lmed by other passengers on
their smartphones and posted immediately. One of the videos, posted to Facebook, was
shared more than 87,000 times and viewed 6.8 million times by 6pm on Monday, just 24
hours later.

The asco was only magni ed by the company’s dismissive response. On Monday
afternoon, United’s CEO tweeted a statement apologizing for “having to re-accommodate
customers.”

This is exactly the kind of PR catastrophe you can avoid with sentiment analysis. It’s an
example of why it’s important to care, not only about if people are talking about your
brand, but how they’re talking about it. More mentions don't equal positive mentions.
Brands of all shapes and sizes have meaningful interactions with customers, leads, even
their competition, all across social media. By monitoring these conversations you can
Try MonkeyLearn
understand customer sentiment in real time and over time, so you can detect disgruntled
customers immediately and respond as soon as possible.

Most marketing departments are already tuned into online mentions as far as volume –
they measure more chatter as more brand awareness. But businesses need to look
beyond the numbers for deeper insights.

Brand Monitoring
Not only do brands have a wealth of information available on social media, but across the
internet, on news sites, blogs, forums, product reviews, and more. Again, we can look at
not just the volume of mentions, but the individual and overall quality of those mentions.

In our United Airlines example, for instance, the are-up started on the social media
accounts of just a few passengers. Within hours, it was picked up by news sites and
spread like wild re across the US, then to China and Vietnam, as United was accused of
racial pro ling against a passenger of Chinese-Vietnamese descent. In China, the incident
became the number one trending topic on Weibo, a microblogging site with almost 500
million users.

And again, this is all happening within mere hours of the incident.

Brand monitoring o ers a wealth of insights from conversations happening about your
brand from all over the internet. Analyze news articles, blogs, forums, and more to gauge
brand sentiment, and target certain demographics or regions, as desired. Automatically
categorize the urgency of all brand mentions and route them instantly to designated
team members.

Get an understanding of customer feelings and opinions, beyond mere numbers and
statistics. Understand how your brand image evolves over time, and compare it to that of
your competition. You can tune into a speci c point in time to follow product releases,
marketing campaigns, IPO lings, etc., and compare them to past events.

Real-time sentiment analysis allows you to identify potential PR crises and take
immediate action before they become serious issues. Or identify positive comments and
respond directly, to use them to your bene t.
Try MonkeyLearn
Example: Expedia Canada

Around Christmas time, Expedia Canada ran a classic “escape winter” marketing
campaign. All was well, except for the screeching violin they chose as background music.
Understandably, people took to social media, blogs, and forums. Expedia noticed right
away and removed the ad.

Then, they created a series of follow-up spin-o videos: one showed the original actor
smashing the violin; another invited a real negative Twitter user to rip the violin out of the
actor’s hands on screen. Though their original campaign was a op, Expedia were able to
redeem themselves by listening to their customers and responding.

Sentiment analysis allows you to automatically monitor all chatter around your brand and
detect and address this type of potentially-explosive scenario while you still have time to
defuse it.

Voice of Customer (VoC)

Social media and brand monitoring o er us immediate, un ltered, and invaluable
information on customer sentiment, but you can also put this analysis to work on surveys
and customer support interactions.

Net Promoter Score (NPS) surveys are one of the most popular ways for businesses to
gain feedback with the simple question: Would you recommend this company, product,
and/or service to a friend or family member? These result in a single score on a number
scale.

Businesses use these scores to identify customers as promoters, passives, or detractors.

The goal is to identify overall customer experience, and nd ways to elevate all customers
to “promoter” level, where they, theoretically, will buy more, stay longer, and refer other
customers.

Numerical (quantitative) survey data is easily aggregated and assessed. But the next
question in NPS surveys, asking why survey participants left the score they did, seeks
open-ended responses, or qualitative data.
Open-ended survey responses were previously much more di cult to analyze, but with
sentiment analysis these texts can be classi ed into positive and negative (and
Try MonkeyLearn
everywhere in between) o ering further insights into the Voice of Customer (VoC).

Sentiment analysis can be used on any kind of survey – quantitative and qualitative – and
on customer support interactions, to understand the emotions and opinions of your
customers. Tracking customer sentiment over time adds depth to help understand why
NPS scores or sentiment toward individual aspects of your business may have changed.

You can use it on incoming surveys and support tickets to detect customers who are
‘strongly negative’ and target them immediately to improve their service. Zero in on
certain demographics to understand what works best and how you can improve.

Real-time analysis allows you to see shifts in VoC right away and understand the nuances
of the customer experience over time beyond statistics and percentages.

Discover how we analyzed the sentiment of thousands of Facebook reviews, and

transformed them into actionable insights.

Example: McKinsey City Voices project

In Brazil, federal public spending rose by 156% from 2007 to 2015, while satisfaction with
public services steadily decreased. Unhappy with this counterproductive progress, the
Urban Planning Department recruited McKinsey to help them focus on user experience,
or “citizen journeys,” when delivering services. This citizen-centric style of governance has
led to the rise of what we call Smart Cities.

McKinsey developed a tool called City Voices, which conducts citizen surveys across more
than 150 metrics, and then runs sentiment analysis to help leaders understand how
constituents live and what they need, in order to better inform public policy. By using this
tool, the Brazilian government was able to uncover the most urgent needs – a safer bus
system, for instance – and improve them rst.

If this can be successful on a national scale, imagine what it can do for your company.

Customer Service
We already looked at how we can use sentiment analysis in terms of the broader VoC, so
now we’ll dial in on customer service teams. Try MonkeyLearn

We all know the drill: stellar customer experiences means a higher rate of returning
customers. Leading companies know that how they deliver is just as, if not more,
important as what they deliver. Customers expect their experience with companies to be
immediate, intuitive, personal, and hassle-free. If not, they’ll leave and do business
elsewhere. Did you know that one in three customers will leave a brand after just one bad
experience?

You can use sentiment analysis and text classi cation to automatically organize incoming
support queries by topic and urgency to route them to the correct department and make
sure the most urgent are handled right away.

Analyze customer support interactions to ensure your employees are following

appropriate protocol. Increase e ciency, so customers aren’t left waiting for support.
Decrease churn rates; after all it’s less hassle to keep customers than acquire new ones.

Discover how we analyzed customer support interactions on Twitter.

Market Research
Sentiment analysis empowers all kinds of market research and competitive analysis.
Whether you’re exploring a new market, anticipating future trends, or seeking an edge on
the competition, sentiment analysis can make all the di erence.
You can analyze online reviews of your products and compare them to your competition.
Maybe your competitor released a new product that landed as a op. Find
Tryout what
MonkeyLearn
aspects of the product performed most negatively and use it to your advantage.

Follow your brand and your competition in real time on social media. Locate new markets
where your brand is likely to succeed. Uncover trends just as they emerge, or follow long-
term market leanings through analysis of formal market reports and business journals.

You’ll tap into new sources of information and be able to quantify otherwise qualitative
information. With social data analysis you can ll in gaps where public data is scarce, like
emerging markets.

Discover how to analyze the sentiment of hotel reviews on TripAdvisor or perform

sentiment analysis on Yelp restaurant reviews.

Sentiment Analysis Tools & Tutorials

Sentiment analysis is a vast topic, and it can be intimidating to get started. Luckily, there
are many useful resources, from helpful tutorials to all kinds of free online
Trytools, to help
MonkeyLearn
you take your rst steps.

Free Online Sentiment Analysis Tools

A good start to your journey is to simply play around with a sentiment analysis tool. A
little rst-hand experience will help you understand how it works

Next, to take your sentiment analysis further, you’ll want to try out MonkeyLearn’s
sentiment analysis and keyword template. First, you’ll need sign up, then walk through
the following steps:

1. Choose Keyword + Sentiment Analysis template

2. Upload your data

If you don't have a CSV, you can use our sample dataset.
Try MonkeyLearn
3. Match the CSV columns to the dashboard elds

In this template, there is only one eld: text. If you have more than one column in your
dataset, choose the column that has the text you would like to analyze.

4. Name your work ow

5. Wait for your data to import

 Try MonkeyLearn

6. Explore your dashboard!

You can:

Filter by sentiment or keyword.

Share via email with other coworkers.

Open Source vs SaaS (Software as a Service) Sentiment

Analysis Tools
When it comes to sentiment analysis (and text analysis in general), you have two choices:
build your own solution or buy a tool. Try MonkeyLearn

Open source libraries in languages like Python and Java are particularly well positioned to
build your own sentiment analysis solution because their communities lean more heavily
toward data science, like natural language processing and deep learning for sentiment
analysis. But you’ll need a team of data scientists and engineers on board, huge upfront
investments, and time to spare.

SaaS tools o er the option to implement pre-trained sentiment analysis models

immediately or custom-train your own, often in just a few steps. These tools are
recommended if you don’t have a data science or engineering team on board, since they
can be implemented with little or no code and can save months of work and money
(upwards of $100,000).

Another key advantage of SaaS tools is that you don't even need to know how to code;
they provide integrations with third-party apps, like MonkeyLearn’s Zendesk, Excel and
Zapier Integrations.

If you want to get started with these out-of-the-box tools, check out this guide to the best
SaaS tools for sentiment analysis, which also come with APIs for seamless integration
with your existing tools.

Or start learning how to perform sentiment analysis using MonkeyLearn’s API and the
pre-built sentiment analysis model, with just six lines of code. Then, train your own
custom sentiment analysis model using MonkeyLearn’s easy-to-use UI.

Tutorial on sentiment analysis in python using MonkeyLearn’s API.

If you’re still convinced that you need to build your own sentiment analysis solution,
check out these tools and tutorials in various programming languages:

Sentiment Analysis Python

Scikit-learn is the go-to library for machine learning and has useful tools for text
vectorization. Training a classi er on top of vectorizations, like frequency or tf-idf text
 vectorizers is quite straightforward. Scikit-learn has implementations for Support
Vector Machines, Naïve Bayes, and Logistic Regression, among others.
Try MonkeyLearn

NLTK has been the traditional NLP library for Python. It has an active community and
o ers the possibility to train machine learning classi ers.

SpaCy is an NLP library with a growing community. Like NLTK, it provides a strong set
of low-level functions for NLP and support for training text classi ers.

TensorFlow, developed by Google, provides a low-level set of tools to build and train
neural networks. There's also support for text vectorization, both on traditional word
frequency and on more advanced through-word embeddings.

Keras provides useful abstractions to work with multiple neural network types, like
recurrent neural networks (RNNs) and convolutional neural networks (CNNs) and
easily stack layers of neurons. Keras can be run on top of Tensor ow or Theano. It also
provides useful tools for text classi cation.

PyTorch is a recent deep learning framework backed by some prestigious

organizations like Facebook, Twitter, Nvidia, Salesforce, Stanford University, University
of Oxford, and Uber. It has quickly developed a strong community.

Tutorials to try out:

Python web scraping and sentiment analysis: this tutorial provides a step-by-step
guide on how to analyze the top 100 subreddits by sentiment. It explains how to use
Beautiful Soup, one of the most popular Python libraries for web scraping that collects
the names of the top subreddit web pages (subreddits like /r/funny, /r/AskReddit and
/r/todayilearned).
Using Praw library, it demonstrates how to interact with the Reddit API and extract the
comments from these subreddits. Then, learn how to use TextBlob to perform
sentiment analysis on the extracted comments. Code: https://github.com/jg-
sher/redditSentiment

Twitter sentiment analysis using Python and NLTK: This step-by-step guide shows you
how to train your rst sentiment classi er. The author uses Natural Language Toolkit
NLTK to train a classi er on tweets. Making Sentiment Analysis Easy with Scikit-learn:
This tutorial explains how to train a logistic regression model for sentiment analysis.

Making Sentiment Analysis Easy with Scikit-learn: This tutorial explains how to train a
logistic regression model for sentiment analysis.
Sentiment Analysis Javascript

Java is another programming language with a strong community aroundTry MonkeyLearn

data science
with remarkable data science libraries for NLP.

OpenNLP: a toolkit that supports the most common NLP tasks, such as tokenization,
sentence segmentation, part-of-speech tagging, named entity extraction, chunking,
parsing, language detection and coreference resolution.

Stanford CoreNLP: a Java suite of core NLP tools provided by The Stanford NLP Group.

Lingpipe: a Java toolkit for processing text using computational linguistics. LingPipe is
often used for text classi cation and entity extraction.

Weka: a set of tools created by The University of Waikato for data pre-processing,
classi cation, regression, clustering, association rules, and visualization.

Sentiment Analysis Research &

Courses
After learning the basics of sentiment analysis, and understanding how it can help you,
you might want to delve further into the topic: Try MonkeyLearn

Sentiment Analysis Papers

The literature around sentiment analysis is massive; there are more than 55,700 scholarly
articles, papers, theses, books, and abstracts out there.

The following are the most frequently cited and read papers in the sentiment analysis
community in general:

Opinion mining and sentiment analysis (Pang and Lee, 2008)

Recognizing contextual polarity in phrase-level sentiment analysis (Wilson, Wiebe and

Ho mann, 2005).

A survey of opinion mining and sentiment analysis (Liu and Zhang, 2012)

Sentiment analysis and opinion mining (Liu, 2012)

How to Perform Text Mining with Sentiment Analysis

Sentiment Analysis Books

Bing Liu is a thought leader in the eld of machine learning and has written a book about
sentiment analysis and opinion mining.

Useful for those starting research on sentiment analysis, Liu does a wonderful job of
explaining sentiment analysis in a way that is highly technical, yet understandable. In the
book, he covers di erent aspects of sentiment analysis including applications, research,
sentiment classi cation using supervised and unsupervised learning, sentence
subjectivity, aspect-based sentiment analysis, and more.

For those who want to learn about deep-learning based approaches for sentiment
analysis, a relatively new and fast-growing research area, take a look at Deep-Learning
Based Approaches for Sentiment Analysis.

Sentiment Analysis Courses and Lectures

Another good way to go deeper with sentiment analysis is mastering your knowledge and
skills in natural language processing (NLP), the computer science eld that
Tryfocuses on
MonkeyLearn
understanding ‘human’ language.

By combining machine learning, computational linguistics, and computer science, NLP

allows a machine to understand natural language including people's sentiments,
evaluations, attitudes, and emotions from written language.

There are a large number of courses, lectures, and resources available online, but the
essential NLP course is the Stanford Coursera course by Dan Jurafsky and Christopher
Manning. By taking this course, you will get a step-by-step introduction to the eld by two
of the most reputable names in the NLP community.

If you want a more hands-on course, you should enroll in the Data Science: Natural
Language Processing (NLP) in Python on Udemy. This course gives you a good
introduction to NLP and what it can do, but it will also make you build di erent projects in
Python, including a spam detector, a sentiment analyzer, and an article spinner. Most of
the lectures are really short (~5 minutes) and the course strikes the right balance
between practical and theoretical content.

Sentiment Analysis Datasets

The key part for mastering sentiment analysis is working on di erent datasets and
experimenting with di erent approaches. First, you’ll need to get your hands on data and
procure a dataset which you will use to carry out your experiments.

The following are some of our favorite sentiment analysis datasets for experimenting
with sentiment analysis and a machine learning approach. They’re open and free to
download:

Product reviews: this dataset consists of a few million Amazon customer reviews with
star ratings, super useful for training a sentiment analysis model.

Restaurant reviews: this dataset consists of 5,2 million Yelp reviews with star ratings.

Movie reviews: this dataset consists of 1,000 positive and 1,000 negative processed
reviews. It also provides 5,331 positive and 5,331 negative processed sentences /
snippets.
 Fine food reviews: this dataset consists of ~500,000 food reviews from Amazon. It
includes product and user information, ratings, and a plain text version
Tryof every
MonkeyLearn
review.

Twitter airline sentiment on Kaggle: this dataset consists of ~15,000 labeled tweets
(positive, neutral, and negative) about airlines.

First GOP Debate Twitter Sentiment: this dataset consists of ~14,000 labeled tweets
(positive, neutral, and negative) about the rst GOP debate in 2016.

If you are interested in rule-based approach, the following is a varied list of sentiment
analysis lexicons that will come in handy. These lexicons provide a set of dictionaries of
words with labels specifying their sentiments across di erent domains. The following
lexicons are really useful to identify the sentiment of texts:

Sentiment Lexicons for 81 Languages: this dataset contains both positive and negative
sentiment lexicons for 81 languages.

SentiWordNet: this dataset contains about 29,000 words with a sentiment score
between 0 and 1.

Opinion Lexicon for Sentiment Analysis: this dataset provides a list of 4,782 negative
words and 2,005 positive words in English.

Wordstat Sentiment Dictionary: this dataset includes ~4800 positive and ~9000
negative words.

Emoticon Sentiment Lexicon: this dataset contains a list of 477 emoticons labeled as
positive, neutral, or negative.

Parting words
Sentiment analysis can be applied to countless aspects of business, from brand
monitoring and product analytics, to customer service and market research. By
incorporating it into their existing systems and analytics, leading brands (not to mention
entire cities) are able to work faster, with more accuracy, toward more useful ends.

Sentiment analysis has moved beyond merely an interesting, high-tech whim, and will
soon become an indispensable tool for all companies of the modern age. Ultimately,
sentiment analysis enables us to glean new insights, better understand our customers,
and empower our own teams more e ectively so that they do better andTry
more
MonkeyLearn
productive work.

MonkeyLearn is an online platform that makes it easy to perform text analytics with
machine learning and data visualization tools.

If you need help building a sentiment analysis system for your business, visit
MonkeyLearn Studio and request a demo.

Related Posts
The Best Free Word Cloud to Visualize Your Data

Visualize Sentiments in a Word Cloud

Keyword Extraction: A Guide to Finding Keywords in Text

Start Using Sentiment Analysis Today!

Automate business processes and save hours of manual data processing.

SIGN UP FREE
 Try MonkeyLearn
RESOURCES GUIDES COMPANY LEGAL

Pricing Voice of Customer Core Values Privacy Policy

Help Data Cleaning Careers (we're Terms

hiring!)
API Docs Data Analysis Data Collection
Twitter

Blog Unstructured Data Responsible

Github
Disclosure

Security

MonkeyLearn Inc. All rights reserved 2022