What is sentiment analysis and how to use it?

The days of relying on a great product or service to do your branding are behind us, which is why acquiring all types of feedback is important.

Quantitative feedback like net promoter scores can provide a general pulse of your brand performance, but qualitative feedback in the form of text can provide insight into how people actually “feel” about your brand.

Sifting through textual data, however, can be impossibly time-consuming for some brands. Doing so manually just isn’t feasible and the nuances of brand sentiment could be difficult to capture. Whether analysing solicited feedback via channels such as surveys or examining unsolicited feedback found on social media, online forums and more, it’s impossible to comprehensively identify and integrate data on brand sentiment when relying solely on manual processes.

One solution to this problem? Sentiment analysis.

Let’s look at the importance of sentiment analysis and how it can be used to improve customer experience through direct and indirect interactions with your brand. Here’s an introduction to sentiment analysis, how it works and when to use it.

What is sentiment analysis?

Sentiment analysis definition: sentiment analysis is the process of determining the opinion, judgment or emotion behind natural language. Sentiment analysis provides an effective way to evaluate written or spoken language to determine if the expression is favorable, unfavorable, or neutral, and to what degree. Because of this it gives a useful indication about how the customer felt about their experience.

If you’ve ever left an online review, made a comment about a brand or product online, or answered a large-scale market research survey, there’s a chance your responses have been through sentiment analysis.

The opinion you expressed by typing into a text box would be turned into categorical data (like “positive”, “negative” or “neutral”), added to data from many other people’s comments, and summarised to give a business a bird’s eye view of how the general public responded to their brand or product.

Sentiment analysis is part of the greater umbrella of text mining, also known as text analysis. This type of analysis extracts meaning from many sources of text, like surveys, reviews, public social media, and even articles on the Web. A score is then applied based on the sentiment of the text. For example, -1 for negative sentiment and +1 for positive sentiment. This is done using natural language processing (NLP).

Today’s algorithm-based sentiment analysis tools can handle huge volumes of customer feedback consistently and accurately. A type of text analysis, sentiment analysis, reveals how positive or negative customers feel about topics ranging from your products and services to your location, your advertisements, or even your competitors.

Automatically uncover trends, problems and opportunities with TextiQ

What is sentiment analysis used for?

So if sentiment analysis is hard to do, what’s the benefit? Why do we use tools to categorise natural language feedback rather than our human brains?

Mostly, it’s a question of scale. Sentiment analysis is helpful when you have a large volume of text-based information that you need to generalise from.

For example, let’s say you’re in marketing for a major motion picture studio, and you just released a trailer for a movie and got a huge volume of comments about it on Twitter.

You can read some – or even a lot – of the comments, but you won’t be able to get an accurate picture of how many people liked or disliked it unless you look at every last one and make a note of whether it was positive, negative or neutral. That would be prohibitively expensive and time-consuming, and the results would be prone to a degree of human error.

On top of that, you’d have a risk of bias coming from the person or people going through the comments. They might have certain views or perceptions that color the way they interpret the data, and their judgment may change from time to time depending on their mood, energy levels and other normal human variations.

With sentiment analysis tools though, you can get a comprehensive, consistent overall verdict with a simple button-press.

Sentiment scores help businesses understand what sort of emotions their brand evokes in a group of people. These emotions can be happiness, sadness, anger, or simply impartialness. From there, it’s up to the business to determine how they’ll put that sentiment into action. One way is to let sentiment inform how customers are currently experiencing your brand.

Why is sentiment analysis important?

Sentiment analysis is critical because it helps provide insight into how customers perceive your brand.

Customer feedback – whether that’s via social media, the website, conversations with service agents, or any other source – contains a treasure trove of useful business information, but it isn’t enough to know what customers are talking about. Knowing how they feel will give you the most insight into how their experience was. Sentiment analysis is one way to understand those experiences.

Sometimes known as “opinion mining,” sentiment analysis can let you know if there has been a change in public opinion toward any aspect of your business. Peaks or valleys in sentiment scores give you a place to start if you want to make product improvements, train sales reps or customer care agents, or create new marketing campaigns.

Use cases for sentiment analysis

We live in a world where huge amounts of written information is produced and published every moment, thanks to the internet, news articles, social media and digital communications. Sentiment analysis can help companies keep track of how their brands and products are perceived, both at key moments and over a period of time.

It’s commonly used in market research, PR, marketing analysis, reputation management, stock analysis and financial trading, customer experience, product design and many more fields besides.

Here are a few scenarios where sentiment analysis can save time and add value:

• Social media listening – in day-to-day monitoring, or around a specific event such as a product launch
• Analysing survey responses for a large-scale research program
• Processing employee feedback in a large organisation
• Identifying very unhappy customers so you can offer closed-loop follow up
• See where sentiment trends are clustered in particular groups or regions
• Competitor research – checking your approval levels against comparable businesses

Types of sentiment analysis

Not all sentiment analysis is done the same way. There are different ways to approach it and a range of different algorithms and processes that can be used to do the job, depending on the context of use and the desired outcome.

Basic sub-types of sentiment analysis include:

• Detecting sentiment
  This means parsing through text and sorting opinionated data (such as “I love this!”) from objective data (like “the restaurant is located downtown”).
• Categorising sentiment
  This means detecting whether the sentiment is positive, negative or neutral. Your tools may also add weighting to these categories, e.g very positive, positive, neutral, somewhat negative, negative.
• Clause-level analysis
  Sometimes, text contains mixed or ambivalent opinions, for example “staff were very friendly but we waited too long to be served”. Being able to score feedback at the clause level indicates when there are both good and bad opinions expressed in one place, and can be useful in case the positives and negatives within a text cancel each other out and return a misleading “neutral” result.

In addition, you can choose whether to view the results of sentiment analysis:

• Document level (useful for professional reviews or press coverage)
• Sentence level (for short comments and evaluations)
• Sub-sentence level (for picking out the meaning in phrases or short clauses within a sentence

Pros and cons of sentiment analysis

Sentiment analysis is a powerful tool that offers a number of advantages, but like any research method has some limitations.

Advantages of sentiment analysis:

• Accurate, unbiased results
• Enhanced insights
• More time and energy available for staff do to higher-level tasks
• Consistent measures you can use to track sentiment over time

Disadvantages of sentiment analysis:

• Best for large and numerous data sets. To get real value out of sentiment analysis tools, you need to be analyzing large quantities of textual data on a regular basis.
• Sentiment analysis is still a developing field, and the results are not always perfect. You may still need to sense-check and manually correct results occasionally.

How does sentiment analysis work?

Sentiment analysis uses machine learning, statistics and natural language processing (NLP) to find out how people think and feel on a macro scale. Sentiment analysis tools take written content and processes it to unearth the positivity or negativity of the expression.

This is done in a couple of ways:

• Rule-based sentiment analysis
  This method uses a lexicon, or word-list, where each word is given a score for sentiment, for example
  “great” = 0.9, “lame” = -0.7, “okay” = 0.1
  Sentences are assessed for overall positivity or negativity using these weightings.
  Rule-based systems usually require additional finessing to account for sarcasm, idioms and other verbal anomalies.
• Machine learning based sentiment analysis
  A computer model is given a training set of natural language feedback, manually tagged with sentiment labels. It learns which words and phrases have a positive sentiment or a negative sentiment. Once trained, it can then be used on new data sets.
  In some cases, the best results come from combining the two methods.

Sentiment analysis challenges

Developing sentiment analysis tools is technically an impressive feat, since human language is grammatically intricate, heavily context-dependent, and varies a lot from person to person. If you say “I loved it,” another person might say “I’ve never seen better,” or “Leaves its rivals in the dust”. The challenge for an AI tool is to recognise that all these sentences mean the same thing.

Another challenge is to decide how language is interpreted since this is very subjective and varies between individuals. What sounds positive to one person might sound negative or even neutral to someone else. In designing algorithms for sentiment analysis, data scientists must think creatively in order to build useful and reliable tools.

Getting the correct sentiment classification

Sentiment classification requires your sentiment analysis tools to be sophisticated enough to understand not only when a data snippet is positive or negative, but how to extrapolate sentiment even when both positive and negative words are used. On top of that, it needs to be able to understand the context and complications such as sarcasm or irony.

Human beings are complicated, and how we express ourselves can be similarly complex. Many types of sentiment analysis tools use a simple view of polarity (positive/neutral/negative), which means much of the meaning behind the data is lost.

Let’s see an example:

“I hated the setup process, but the product was easy to use so in the end I think my purchase was worth it.”

A less sophisticated sentiment analysis tool might see the sentiment expressed here as “neutral” because the positive – “the product was easy to use so in the end I think my purchase was worth it” – and negative-tagged sentiments – “I hated the setup process” – cancel each other out.

However, polarity isn’t so cut-and-dry as being one or the other here. The final part – “in the end, I think my purchase was worth it” – means that as a human analysing the text, we can see that generally this customer felt mostly positive about the experience. That’s why a scale from positive to negative is needed, and why a sentiment analysis tool adds weighting along a scale of 1-11.

Scores are assigned with attention to grammar, context, industry, and source, and Qualtrics gives users the ability to adjust the sentiment scores to be even more business-specific.

Better understand your customers with real time sentiment analysis

Understanding context

Context is key for a sentiment analysis model to be correct. This means you need to make sure that your sentiment scoring tool not only knows that “happy” is positive—and that “not happy” is not, but understands that certain words that are context-dependent are viewed correctly.

Let’s see an example:

As human beings, we know customers are pleased when they mention how “thin” their new laptop is, but that they’re complaining when they talk about the “thin” walls in your hotel. We understand that context.

Obviously, a tool that flags “thin” as negative sentiment in all circumstances is going to lose accuracy in its sentiment scores. The context is important.

This is where training natural language processing (NLP) algorithms comes in. Natural language processing is a way of mimicking the human understanding of language, meaning context becomes more readily understood by your sentiment analysis tool.

Sentiment analysis algorithms are trained using this system over time, using deep learning to understand instances with context and apply that learning to future data. This is why a sophisticated sentiment analysis tool can help you to not only analyze vast volumes of data more quickly, but also discern what context is common or important to your customers.

3 places to analyse customer sentiment

In a world of endless opinions on the Web, how people “feel” about your brand can be important for measuring the customer experience.

Consumers desire likeable brands that understand them; brands that provide memorable on-and-offline experiences. The more in-tune a consumer feels with your brand, the more likely they’ll share their emotions in written text (through surveys, reviews, social media, and more).

But the opposite is true as well. As a matter of fact, 71 percent of Twitter users will take to the social media platform to voice their frustrations with a brand. Those users also expect brands to respond to queries within an hour of tweeting, but we’ll save that for another day.

These conversations, both positive and negative, should be captured and analysed to improve the customer experience. Sentiment analysis can help.

Let’s first look at how more direct interactions with brands can be analysed.

1. Text analysis for surveys

Surveys are a great way to connect with customers directly, but they’re also ripe with constructive feedback. The feedback within your survey responses can be quickly analysed for sentiment scores.

For the survey itself, consider questions that will generate qualitative customer experience metrics, some examples include:

• What was your most recent experience like?
• How much better (or worse) was your experience compared to your expectations?
• What is something you would have changed about your experience?

Remember, the goal here is to acquire honest textual responses from your customers so the sentiment within them can be analysed. Another tip is to avoid close-ended questions that only generate “yes” or “no” responses. These types of questions won’t serve your analysis well.

Next, use a text analysis tool to break down the nuances of the responses. TextiQ is an example of a tool that will not only provide sentiment scores but extract key themes from the responses.

After the sentiment is scored from survey responses, you’ll be able to address some of the more immediate concerns your customers have during their experiences.

Another great way to acquire sentiment is through customer reviews. This method is a bit more indirect compared to surveys.

2. Text analysis for customer reviews

Did you know that 72 percent of customers will not take action until they’ve read reviews on a product or service? An astonishing 95 percent of customers read reviews prior to making a purchase. In today’s feedback-driven world, the power of customer reviews and peer insight is undeniable.

Review sites like G2 are common first-stops for customers looking for honest feedback on products and services. This feedback, like that in surveys, can be analysed for emotional responses.

The benefit of customers providing reviews compared to responding to surveys is that it’s more indirect, which could lead to more honest and in-depth feedback. This isn’t a rule-of-thumb, but analysing customer reviews has its clear upsides.

To improve the customer experience, you can take the sentiment scores from customer reviews – positive, negative, and neutral – and identify gaps and pain-points that may have not been addressed in the surveys. Remember, negative feedback is just as (if not more) beneficial to your business than positive feedback.

3. Text analysis for social media

One of the most indirect ways to acquire textual data is through social media mining. This is made possible using social media management software with monitoring capabilities.

Monitoring tools essentially scrape public social media like Twitter and Facebook for brand mentions and assign sentiment scores accordingly. This has its upsides as well, considering users are highly likely to take their uninhibited feedback to social media.

One downside to text analysis for social media is character limitations. Whereas in surveys and review sites there is a set of contexts, social media is more free reign. This could create some noise within the data, but this isn’t a huge concern.

Regardless, a staggering 70 percent of brands don’t bother with feedback on social media. Because social media is an ocean of big data just waiting to be analysed, brands could be missing out on some important sentiment.

Sentiment analysis tools

When choosing sentiment analysis technologies, bear in mind how you will use them. There are a number of options out there, from open-source solutions to in-built features within social listening tools. Some of them are limited in scope, while others are more powerful but require a high level of user knowledge.

Text iQ is a natural language processing tool within the Experience Management Platform™ that allows you to carry out sentiment analysis online using just your browser. It’s fully integrated, meaning that you can view and analyse your sentiment analysis results in the context of other data and metrics, including those from third-party platforms.

And like all Qualtrics tools, it’s designed to be straightforward, clear and accessible to those without specialised skills or experience, so there’s no barrier between you and the results you want to achieve.

Analysing customer sentiment, creating better experiences

Whether you’re using a text analysis tool for survey responses or a social media management tool for mining purposes, the key here is to be on the lookout for customer feedback. Sentiment analysis is not a one and done effort and requires continuous monitoring. By reviewing your customers’ feedback on your business regularly, you can proactive get ahead of emerging trends and fix problems before it’s too late.

Acquiring feedback and analysing its sentiment can provide businesses with a deep pulse on how customers truly “feel” about their brand. When you’re able to understand your customers emotionally, you’re able to provide a more robust customer experience.