Improving Sentiment Analysis: Star Rating Vs. Sentiment Index

Published on — Written by Wonderflow

sentiment analysis

In this article, Wonderflow’s Business Intelligence analysts shed more light and clarity on Star Rating and the Sentiment Index score, two very important metrics in sentiment analysis. This may help our readers, particularly current users, understand better why analyzing star rating alone, for example, is not enough to understand what customers want and need and to achieve product success.

 

What is the first thing you do when looking for a product on Amazon? You most likely check its star rating. 

If it’s too low, you just keep searching, but if it’s good enough and you are interested in buying, then you probably go and read some online reviews. There’s a reason you do that: the star rating alone doesn’t provide you with any information about the product and its features. It is just a number. 

For this same reason, companies are also interested in tracking not only what star rating their products get but also what customers say about them, positively or negatively.

This is why at Wonderflow, we provide two different metrics (among others) for our customers to measure and track the health of their products and brands: Star Rating and Sentiment Index.

But let’s take a step back and start by defining what these two sentiment analysis metrics are.

What is a Star Rating?

Star Rating is simply the average of all customer scores on a certain product they bought online. It goes from 1 to 5, where 1 star means that the customer is not happy at all with the product, while 5 stars mean they are very satisfied with the purchase. 

This Apple Watch sold on Amazon.com, for example, gets an impressive 4.8 as the Average Star Rating:

What is a Sentiment Index?

The Sentiment Index is a slightly more complex metric. Consider that each review can contain comments about different positive and negative aspects. Let’s look, for example, at the following review: 

Above, the customer is generally happy with the coffee machine. The quality of the coffee and design is poor, but he is dissatisfied with the steamer and instructions. 

So what the Sentiment Index tells us is what is the ratio between the amount of positive and negative words or comments. The Sentiment Index score plots the difference between the total positive and negative comments normalized in the interval , where +1 means that we only have positive comments and no negatives. 

-1 means that we have only negatives and no positives. 0 means that we have as many positive as negative comments.

This is how the Sentiment Index is calculated:

(Positive comments – negative comments) / (Positive comments + negative comments)

Why Star Rating & Sentiment Index Together Improve Your Sentiment Analysis 

Analyzing these metrics together can provide a more comprehensive understanding of how a product or brand performs and the overall customer experience. As we initially said, the Average Star Rating is just a number, and customer evaluations are obviously subjective. 

So, for example, two customers can give a 4-star rating to a product but express a different opinion. The first one would report 1 positive aspect and 1 negative, while the other 3 positive and 1 negative aspect. So, we can have a more positive or negative sentiment for the same rating.

Let’s have a look again at real reviews like this one about a dishwasher:

Both customers gave a 4-star rating to the product, but we can easily appreciate from the reviews that their satisfaction with the product is quite different. While the second customer is enthusiastic about it, the first one has some areas to complain about and is just “all in all” happy about it. Not the same thing, right? 

So if we only look at the Star Rating, we would conclude that both customers are equally satisfied. Analyzing Sentiment Index, we would get a different picture and understand that one customer is less satisfied. Of course, collecting this kind of data for the hundreds and thousands of reviews that a product or brand might get can provide powerful information about its performance.

Here’s an example of two products, where one has a higher star rating but actually a lower Sentiment Index. If we look only at the star rating, we would conclude easily that product B is better than product A. Still, the Sentiment index tells another story: investigating the reasons behind this difference provides useful information on the overall performance of the product.

Massimo Paludet, Group Customer Care Director at the De’Longhi Group, is an expert user of the Wonderboard. He has confirmed how:

“Users tend to evaluate the relevance of the positive and negative aspects in relation to their individual expectations. In this sense, the Sentiment Index becomes a fundamental metric to turn aspects and considerations that are purely subjective into something objective, allowing companies to define action plans based on real priorities.”

Monitor the Trend Over Time of Star Rating & Sentiment Index 

Star Rating and Sentiment Index can also be analyzed over time, providing useful additional information. At a general level, we expect to see a similar development of the two metrics because if customers are increasingly complaining about certain aspects of a product, we also expect its Star Rating to decline.

Or conversely, if people are saying more and more positive things, the Star Rating would increase, and that’s often the case. However, there might be situations where the two KPIs develop differently. 

Suppose we see, for example, that the Sentiment Index is increasing, but the Star Rating is staying flat. In that case, this might suggest that the topics that customers are talking about in a positive way don’t have such an impact on the rating they are giving. 

Let’s assume, in another example, that a new feature has been added to a product. Customers might add a positive comment in their reviews about that (“feature X works well,” “this is a nice feature to have”). 

This would increase the Sentiment Index since there’s one more positive topic to count. However, that feature might be just a nice-to-have; if it were not there, they would still give the same Star Rating.

Conversely, if we see a flat Star Rating but a declining Sentiment Index, this might be a red flag that the Star Rating may also start decreasing over time. The reason is there’s likely some aspect that customers are increasingly complaining about, which might soon affect their overall satisfaction.

Therefore, seeing discrepancies in the trendline of these two metrics would bring us to investigate further what customers are saying and understand if there are any issues we need to solve or aspects we need to promote more.

Final Thoughts

Nowadays, Star Rating is a widespread metric, the most adopted way of evaluating something online. On almost every website that sells products or services, you will find yellow stars that indicate the level of customer satisfaction (sometimes, we can even find the star rating of the website itself). 

However, what Star Rating doesn’t tell you is why a user is giving 5 stars or 1 star. Only the review left by the user can provide such a level of detail. That is why the Sentiment Index is so important. Looking at single metrics in isolation increases the risk of having only a partial view of the picture.

Again, Massimo Paludet underlines how:

“The Sentiment Index provides a more trustworthy indication of the real degree of satisfaction of a product, being correlated to objective criteria (positive or negative), beyond star rating alone. It’s clear that we have to consider both KPIs, as companies tend to start working with the star rating and set targets on it. However, the Sentiment Index and the details related to it represent actionable feedback of huge value and relevance.”

If you’re interested in understanding more about how the natural language processing technique of sentiment analysis works, including how AI can help predict star ratings, talk with one of our VoC experts today to see a free demo.

About Wonderflow

Wonderflow empowers businesses with quick and impactful decision-making because it helps automate and deliver in-depth consumer and competitor insights. All within one place, results are simplified for professionals across any high-UGC organization, and department to access, understand, and share easily. Compared to hiring more analysts, Wonderflow’s AI eliminates the need for human-led setup and analysis, resulting in thousands of structured and unstructured reviews analyzed within a matter of weeks and with up to 50% or more accurate data. The system sources relevant private and public consumer feedback from over 200 channels, including emails, forums, call center logs, chat rooms, social media, and e-commerce. What’s most unique is that its AI is the first ever to help recommend personalized business actions and predict the impact of those actions on key outcomes. Wonderflow is leveraged by high-grade customers like Philips, DHL, Beko, Lavazza, Colgate-Palmolive, GSK, Delonghi, and more.

Start making winning decisions based on customer feedback todayGet a free demo

pet care

Marketing · Nov 11, 2022

Pet Care Industry: Trends & Growth to Help Improve VoC Programs

When it comes to our furry (feathery, scaly – you name it) friends, brands and consumers alike will do their utmost to satisfy them. In fact, the global pet care market has seen astounding spikes in sales in recent years, particularly during the COVID-19 pandemic. In this article, we look at the ever-growing and competitive world of pet care. In particular, we examine how the 2020 pandemic affected the market and what global trends have…

tire

Market Research · Oct 12, 2022

VoC Analysis: What Are Consumers Really Saying About Tires?

Wonderflow’s Business Intelligence team conducted an original market analysis of the tire industry, including a cross-country comparison. Using our very own advanced natural language processing technology, we’ve uncovered several key trends and insights that tire brands, manufacturers, and marketers can leverage to create effective strategies and improve the consumer experience.   In the world’s most advanced countries, on average, 65% of the population owns a car. Vehicle owners living in the U.S. reach 87%. Most…

Andrea Evangelista

Interview · Sep 15, 2022

Employee Spotlight: Andrea Evangelista, NLP Developer & Security Manager

  In this month’s Employee Spotlight feature, we welcome Andrea Evangelista. Currently, our Software Developer in the natural language processing  (NLP) & Data Science and Security Manager,  Andrea’s career journey started sometime in 2017. He was studying for his MSc in Computer Science Engineering (CyberSecurity) in Eindhoven, the Netherlands. He worked as a freelancer for some clients. Back then, when Andrea applied for an analyst position, many things were still manual and needed constant human…