Guest Blog: Advances In AI Will Transform Sentiment Analysis

Published on — Written by Wonderflow

wonderflow image

HIPSTO has developed a premier AI open knowledge discovery platform. As one of our technology partners, their advanced natural language understanding technology works within Wonderflow’s advanced voice of the customer analytics platform to provide users with the most reliable and accurate results.

In their guest blog post, HIPSTO helps CX professionals to gain a clearer understanding of what sentiment analysis is, including key challenges in this exciting and emerging research field and how certain solutions can combat them. This article is contributed by HIPSTO’s CTO, Andrii Pylypenko.

 

Customer experience (CX) and brand teams know the importance of understanding what their customers think and how they feel.

Having detailed insight into customer sentiment unlocks the door to building global brands and creating amazing customer experiences. However, until now, revealing the full depth and range of customer opinion has remained an elusive goal – a destination just beyond the horizon. 

This is partly because of the underwhelming progress that has been made on sentiment analysis to date. It’s an area where AI can make a huge difference, yet the market has been slow to evolve.

This is about to change, as a breakthrough has been achieved that will deliver a far more comprehensive and accurate approach than previously possible, enabling CX professionals to gain the upper hand with a deeper understanding of customer opinion. 

It is about the technology: 94.5% accuracy, with consistency over 100 languages

The differentiator here is an underlying technological approach to sentiment analysis.

As the CTO of HIPSTO, a deep tech AI development company, I’m delighted to already partner with Wonderflow on projects that automate the collection of data from around the web (and world).

Extracting value from customer feedback is already something the Wonderflow platform excels at.

However, what’s really exciting is how advanced sentiment analysis technology can take customer understanding to the next level.

We’ve focused on accuracy and consistency across languages (100+) as key factors, producing unparalleled results with a 94.5% accuracy rate (substantially above the 70 to 80% industry average), and outperforming major providers including Amazon Comprehend and Google Cloud.

The problem with sentiment analysis

Why has sentiment analysis been so underwhelming up to now?

Language is highly complex and has deep nuances. The sentiment expressed in a piece of text is not always clear: slang, sarcasm, evolving vocabulary, multiple opinions expressed within one sentence, context about the person expressing the opinion, and even the use of emojis make understanding sentiment very difficult.

Automated sentiment analysis is extremely challenging for a computer, and approaches that replicate human understanding are needed. 

Existing solutions have found it hard to replicate this human understanding.

Many mainstream sentiment analysis solutions do not even leverage AI and machine learning to understand the sentiment, instead, they depend on statistical techniques that often fall short when it comes to ‘genuine’ accuracy.

Underwhelming progress

More advanced sentiment analysis that does actually use AI has seen some improvements but has still not achieved an acceptable threshold.

Earlier AI solutions utilized neural networks, but these methods have drawbacks too.

Recently, more solutions are leveraging ‘Transformers’ – a more advanced neural network architecture that better understands the context of content – in a move from Natural Language Processing to Natural Language Understanding

This is a move in the right direction, but some Transformer frameworks like Google’s BERT are no longer state-of-the-art, again underpinning the problem with accuracy across the whole industry. 

A different approach

We wanted to rectify this situation by producing the industry-leading sentiment analysis that marketers need.

It has taken a lengthy, prolonged effort from our experienced data scientists, but has achieved superior results-producing ground-breaking sentiment analysis. 

This has been attained by leveraging state-of-the-art neural networks, and best-in-class Transformers (including mT5) across a proprietary neural network architecture, in order to understand the full context of the long and short-form text.

We then run a round of proprietary weightings and algorithms that aggregate sentiment at different levels and present findings based on positive, negative, and neutral sentiment. 

This has five key advantages over other approaches:

1. From Natural Language Processing to Natural Language Understanding

Because we only use truly cutting-edge techniques and Transformers, our approach results in Natural Language Understanding, mimicking the human comprehension and judgment required to deliver advanced sentiment analysis.

The consequence for marketers is a significantly improved understanding of customer sentiment that takes into account the context and finer nuances of the text. 

2. Industry-leading accuracy

A standard success metric relating to sentiment analysis is the F1 score.

This considers precision (the proportion of results that are accurate) and recall (the proportion of relevant results that are identified) and aggregates them into one number.

The closer to 1 the better, and HIPSTO currently scores 0.9446, meaning 94.5% accuracy

3. Multi-language coverage

Because the HIPSTO platform leverages the best-in-class Transformer (mT5) over other frameworks like Google BERT (which is now showing its age), we have consistently high performance in over 100 major languages.

This is absolutely critical for global brands wishing to track opinion across the global media, including unstructured web pages and social media which obviously incorporates conversations in many native languages.

With this level of language understanding, other use cases become possible, for example, the ability for global organizations to track the sentiment of employees. 

4. More sophisticated aggregation

It’s about rethinking the approach currently taken to produce better results. Google Cloud’s AI framework has (in my opinion) too much neutral content, caused by positive and negative sentiment canceling each other out.

Amazon Comprehend simply adds too much content containing mixed sentiment into a catch-all ‘mixed’ bucket.

Both of these companies’ approaches result in an influx of data that isn’t very helpful for analysis because much of the positive and negative sentiment gets underplayed.

The HIPTSO platform has been designed to avoid this issue by providing a better picture of what people are thinking – both positive and negative. This is particularly important in customer experience because it’s those opinions that other solutions miss that provide the most valuable and actionable insights for marketers.

5. Coverage across all content types

It’s important that technology advancements work across all types of content – structured and unstructured, high- and low-frequency, long and short form. This allows teams to track media sentiment, customer opinion, and more across news reports, Reddit groups, Twitter feeds, and product reviews; there are no constraints on what our sentiment analysis platform can cover.

Transforming the market

High accuracy sentiment analysis for digital marketing and customer experience teams is an essential ingredient for every successful VoC program and customer experience improvement initiative.

It helps brands to defend and promote their reputation, improve products, and more.

Here, the technology does make a difference, and we aim to lead a sentiment analysis transformation that will change the market, raising the collective game to provide the sentiment analysis that marketers need and deserve.

About Wonderflow

Wonderflow empowers businesses with quick and impactful decision-making because it helps automate and deliver in-depth consumer and competitor insights. All within one place, results are simplified for professionals across any high-UGC organization, and department to access, understand, and share easily. Compared to hiring more analysts, Wonderflow’s AI eliminates the need for human-led setup and analysis, resulting in thousands of structured and unstructured reviews analyzed within a matter of weeks and with up to 50% or more accurate data. The system sources relevant private and public consumer feedback from over 200 channels, including emails, forums, call center logs, chat rooms, social media, and e-commerce. What’s most unique is that its AI is the first ever to help recommend personalized business actions and predict the impact of those actions on key outcomes. Wonderflow is leveraged by high-grade customers like Philips, DHL, Beko, Lavazza, Colgate-Palmolive, GSK, Delonghi, and more.

Start making winning decisions based on customer feedback todayGet a free demo

Other articles you might like:

wonderflow image

Product Development · Sep 12, 2022

Proper Review Analysis: How to Analyze Customer Reviews (+ Examples)

It’s pretty difficult to imagine a world without reviews. Not many of us walk into a shop and make a purchase without browsing at least a handful of online reviews beforehand.  In fact, a study from TrustPilot found that almost 9 out of 10 customers consult online reviews before buying.  Though these reviews are incredibly helpful to consumers, for business owners, trying to use reviews to guide the product development process or improve the customer…

positive customer reviews

Marketing · Aug 25, 2022

More Smiles, More Stars: 10 Examples of Leveraging Positive Customer Reviews

Brands know that reviews are everything. Whether they are positive, negative, or neutral, customer feedback is at the core of any business. It’s a matter of how well your company leverages or responds to each type. For instance, positive customer reviews are just as powerful of an influence on business success, if not more, than negative ones. And do you just simply smile when you see a customer’s compliment? Of course not! You may want…

utility cx

Utility & Energy · Aug 09, 2022

More Than Another Bill: Why VoC Programs Improve Utility CX

When did your utility provider last ask you to rate your customer experience (CX) with them? With rising competition, energy costs, customer demands, and regulatory and stakeholder pressure to improve customer services, these key factors drive service providers of water, electricity, and gas to engage deeply with their customers. Old traditional business models must be changed to adapt to the times, and utility brands must prioritize CX more than ever before.  The Utility Customer Experience …