Guest Blog: Advances In AI Will Transform Sentiment Analysis

Published on — Written by Wonderflow

wonderflow image

HIPSTO has developed a premier AI open knowledge discovery platform. As one of our technology partners, their advanced natural language understanding technology works within Wonderflow’s advanced voice of the customer analytics platform to provide users with the most reliable and accurate results.

In their guest blog post, HIPSTO helps CX professionals to gain a clearer understanding of what sentiment analysis is, including key challenges in this exciting and emerging research field and how certain solutions can combat them. This article is contributed by HIPSTO’s CTO, Andrii Pylypenko.

 

Customer experience (CX) and brand teams know the importance of understanding what their customers think and how they feel.

Having detailed insight into customer sentiment unlocks the door to building global brands and creating amazing customer experiences. However, until now, revealing the full depth and range of customer opinion has remained an elusive goal – a destination just beyond the horizon. 

This is partly because of the underwhelming progress that has been made on sentiment analysis to date. It’s an area where AI can make a huge difference, yet the market has been slow to evolve.

This is about to change, as a breakthrough has been achieved that will deliver a far more comprehensive and accurate approach than previously possible, enabling CX professionals to gain the upper hand with a deeper understanding of customer opinion. 

It is about the technology: 94.5% accuracy, with consistency over 100 languages

The differentiator here is an underlying technological approach to sentiment analysis.

As the CTO of HIPSTO, a deep tech AI development company, I’m delighted to already partner with Wonderflow on projects that automate the collection of data from around the web (and world).

Extracting value from customer feedback is already something the Wonderflow platform excels at.

However, what’s really exciting is how advanced sentiment analysis technology can take customer understanding to the next level.

We’ve focused on accuracy and consistency across languages (100+) as key factors, producing unparalleled results with a 94.5% accuracy rate (substantially above the 70 to 80% industry average), and outperforming major providers including Amazon Comprehend and Google Cloud.

The problem with sentiment analysis

Why has sentiment analysis been so underwhelming up to now?

Language is highly complex and has deep nuances. The sentiment expressed in a piece of text is not always clear: slang, sarcasm, evolving vocabulary, multiple opinions expressed within one sentence, context about the person expressing the opinion, and even the use of emojis make understanding sentiment very difficult.

Automated sentiment analysis is extremely challenging for a computer, and approaches that replicate human understanding are needed. 

Existing solutions have found it hard to replicate this human understanding.

Many mainstream sentiment analysis solutions do not even leverage AI and machine learning to understand the sentiment, instead, they depend on statistical techniques that often fall short when it comes to ‘genuine’ accuracy.

Underwhelming progress

More advanced sentiment analysis that does actually use AI has seen some improvements but has still not achieved an acceptable threshold.

Earlier AI solutions utilized neural networks, but these methods have drawbacks too.

Recently, more solutions are leveraging ‘Transformers’ – a more advanced neural network architecture that better understands the context of content – in a move from Natural Language Processing to Natural Language Understanding

This is a move in the right direction, but some Transformer frameworks like Google’s BERT are no longer state-of-the-art, again underpinning the problem with accuracy across the whole industry. 

A different approach

We wanted to rectify this situation by producing the industry-leading sentiment analysis that marketers need.

It has taken a lengthy, prolonged effort from our experienced data scientists, but has achieved superior results-producing ground-breaking sentiment analysis. 

This has been attained by leveraging state-of-the-art neural networks, and best-in-class Transformers (including mT5) across a proprietary neural network architecture, in order to understand the full context of the long and short-form text.

We then run a round of proprietary weightings and algorithms that aggregate sentiment at different levels and present findings based on positive, negative, and neutral sentiment. 

This has five key advantages over other approaches:

1. From Natural Language Processing to Natural Language Understanding

Because we only use truly cutting-edge techniques and Transformers, our approach results in Natural Language Understanding, mimicking the human comprehension and judgment required to deliver advanced sentiment analysis.

The consequence for marketers is a significantly improved understanding of customer sentiment that takes into account the context and finer nuances of the text. 

2. Industry-leading accuracy

A standard success metric relating to sentiment analysis is the F1 score.

This considers precision (the proportion of results that are accurate) and recall (the proportion of relevant results that are identified) and aggregates them into one number.

The closer to 1 the better, and HIPSTO currently scores 0.9446, meaning 94.5% accuracy

3. Multi-language coverage

Because the HIPSTO platform leverages the best-in-class Transformer (mT5) over other frameworks like Google BERT (which is now showing its age), we have consistently high performance in over 100 major languages.

This is absolutely critical for global brands wishing to track opinion across the global media, including unstructured web pages and social media which obviously incorporates conversations in many native languages.

With this level of language understanding, other use cases become possible, for example, the ability for global organizations to track the sentiment of employees. 

4. More sophisticated aggregation

It’s about rethinking the approach currently taken to produce better results. Google Cloud’s AI framework has (in my opinion) too much neutral content, caused by positive and negative sentiment canceling each other out.

Amazon Comprehend simply adds too much content containing mixed sentiment into a catch-all ‘mixed’ bucket.

Both of these companies’ approaches result in an influx of data that isn’t very helpful for analysis because much of the positive and negative sentiment gets underplayed.

The HIPTSO platform has been designed to avoid this issue by providing a better picture of what people are thinking – both positive and negative. This is particularly important in customer experience because it’s those opinions that other solutions miss that provide the most valuable and actionable insights for marketers.

5. Coverage across all content types

It’s important that technology advancements work across all types of content – structured and unstructured, high- and low-frequency, long and short form. This allows teams to track media sentiment, customer opinion, and more across news reports, Reddit groups, Twitter feeds, and product reviews; there are no constraints on what our sentiment analysis platform can cover.

Transforming the market

High accuracy sentiment analysis for digital marketing and customer experience teams is an essential ingredient for every successful VoC program and customer experience improvement initiative.

It helps brands to defend and promote their reputation, improve products, and more.

Here, the technology does make a difference, and we aim to lead a sentiment analysis transformation that will change the market, raising the collective game to provide the sentiment analysis that marketers need and deserve.

About Wonderflow

Wonderflow empowers businesses with quick and impactful decision-making because it helps automate and deliver in-depth consumer and competitor insights. All within one place, results are simplified for professionals across any high-UGC organization, and department to access, understand, and share easily. Compared to hiring more analysts, Wonderflow’s AI eliminates the need for human-led setup and analysis, resulting in thousands of structured and unstructured reviews analyzed within a matter of weeks and with up to 50% or more accurate data. The system sources relevant private and public consumer feedback from over 200 channels, including emails, forums, call center logs, chat rooms, social media, and e-commerce. What’s most unique is that its AI is the first ever to help recommend personalized business actions and predict the impact of those actions on key outcomes. Wonderflow is leveraged by high-grade customers like Philips, DHL, Beko, Lavazza, Colgate-Palmolive, GSK, Delonghi, and more.

Start making winning decisions based on customer feedback todayGet a free demo

Other articles you might like:

cx

Customer experience · Apr 22, 2022

Is the Customer Experience Career for You? The Rise of CX Jobs (Incl. Salary Chart and FAQs)

“Think of customer experience as customer service on steroids” – Forbes  That’s exactly what it’s like to work in Customer Experience (CX). When compared to customer service (CS), customer experience is more about proactively creating a seamless customer journey, from awareness of the brand to post-purchase. In contrast, CS is about reactively supporting customers with service/product questions and problems.  This article aims to shed light on the trending and unique career path of CX and how…

Massimo Paludet

Interview · Apr 15, 2022

Consumer-Centricity in Home Appliance: A VoC Interview with Massimo Paludet, Customer Care Director at De’Longhi

Our series of Voice of the Customer (VoC) interviews proudly showcases the perspectives of different industry experts on the customer experience (CX) world. It has always been an immense pleasure to support the customers with our VoC insights and gain their user feedback and leadership insights to empower our readers. Wonderflow’s Sales Account Executive, Adam Hosker, is joined today by De’Longhi Group. Specifically, we interviewed Massimo Paludet, Director of Customer Care. De’Longhi is one of…

Krisiana Dumi

Interview · Apr 07, 2022

Wonderstudent: Krisiana Dumi, Grant Recipient & Epicode Grad

It’s not enough, we believe, to deliver only the best experience possible to the end-users. Those with strong technical skills and talent are also deserving of the best experiences possible to succeed, particularly in academics. As such, Wonderflow’s charity, the Wonderflow Foundation has partnered with Epicode, an Italy-based coding school that’s helping to prepare those interested in an IT career. Together, we’ve been on a mission to scout students who are especially from disadvantaged families…