Text Analysis 101: An In-Depth Guide for Beginners

If your business is seeking to grow, there’s one key factor in making that happen: keeping up with the rapid advancements in technology. These days, this can seem like an overwhelming task. There’s so much new technology available on a corporate level that it can be difficult to sort through it all and decide what’s worth the money and time. One of the technological advances most beneficial to businesses is text analysis software, or software that works with natural language processes.

This type of software can take both marketing and customer service to the next level – keeping customers happy, aiding in the search for new customers, observing these customers’ opinions and reactions, and even monitoring the competition.

In this day and age, text analysis is a must-have for anyone truly interested in succeeding as a business. Below, we discuss the basics of text analysis – what it is, what it can do, what you can do with it, and what information is available to you.

• What is NLP & text mining?
• What can text analysis do for modern brands?
• Text analysis techniques
• Text analysis examples
• Software options for text analysis

Text analysis is a multi-faceted idea that involves a variety of terms that are relatively new to marketing and customer service. Research into text analysis can delve into artificial intelligence (or AI) and other equally difficult areas for those who aren’t as versed in technology. Here, we break down some of the major related terms: natural language processing (or NLP) and text mining.

Natural language processing is, at its core, a means to an end – and that end is text analysis or text mining.

Natural language processing is simply how computers interact with human language. This can take different forms, including but not limited to creating text, transcribing text, and categorizing or labeling text or speech.

NLP is most closely linked to AI in this way; NLP understands the world based entirely on how it is programmed, though the goal of this programming is to make the software understand and express language as clearly and naturally – as much like a human – as possible.

For example, humans speaking English know when to use contractions and when not to. It sounds much more natural to say “It’s a bird, I know it is!” than to say “It is a bird, I know it’s!” This type of language construction is tricky and is a big part of making speech sound human and not robotic.

Natural language processing software can work in several different ways, but the gist is the same: patterns and phrases indicate moods or ideas, which can then be modeled within set parameters. This is machine learning – the software knows one rule and solidifies how that rule is used as it takes in more and more examples.

Text mining, or text analysis, software includes natural language processing, but also other aspects. Generally, to be considered text mining software, it must include these facets:

• Gathering text and other language data
• Use NLP to take meaning from the collected data
• Provide visualizations to make sense of the collected language data and resulting insights

Text mining cannot occur without NLP, and NLP isn’t nearly as useful without the other parts of text mining.

Without natural language processing, the software cannot know what text to mine or what to do with it. This becomes a more traditional form of data gathering: humans sorting through any number of responses or comments, hoping to find something useful.

Natural language processing allows your business to take the humans out of this part of the equation. A computer can sort through all incoming data in a fraction of the time, and now it can determine which data is most important, too.

Without the other aspects of text mining, though, NLP simply provides data, with no meaning to it. The gathering of the data is necessary to have language to process in the first place, and the visualizations of the results make sense of this data.

Text analysis and natural language processing may be relatively new, but that doesn’t mean there aren’t already many areas in which they’re being used successfully and already have future implications. Any modern business can make use of text analysis in marketing, sales, and customer service. The benefits of text analysis have already been seen in big ways:

One of the biggest ways that text analysis can affect marketing is by influencing ad placement.

With text analysis, the software can sift through an individual’s search history and other data to determine what they’re most likely to buy, and then they can place the most effective ads right in front of them.

Additionally, NLP and text mining can help determine keywords for SEO content. The data gathered in text mining takes all the guesswork out of knowing which words are worth including. Past that, text mining can hone your primary keyword lists to adhere better to semantic searching and even point you in the direction of the best topics to focus on.

Utilizing text mining to increase sales has also seen success. In this case, much of the process is based on sentiment analysis. Sentiment analysis is when NLP is used to determine how people feel about your product based on the sentiment behind their comments on it.

Obviously, this is closely tied with customer service, but the implications can be farther reaching than that. If your business knows what features or products are popular among customers and which they simply tolerate, you can better target advertising, make adjustments to products, and address problems with your customers.

Text mining also gives you the opportunity to see how you measure up to your competitors, which can provide more insight on product changes or customer opinion.

One very specific and niche example of using text mining to increase sales is the addition of chatbots to sales pages.

If a customer is browsing a specific product online and has instant, 24/7 access to a conversation where their questions can be answered – even if it’s by a specially-trained bot – they’re more likely to buy or consider the product than they would be if they had to do additional web searches to get their questions answered.

Obviously, customer service is closely tied to both sales and marketing in the realm of text mining and NLP. Chatbots are only one way that customers can be reached, and sentiment analysis is another way that businesses can receive feedback.

Really, text mining allows customers’ concerns, ideas, and queries to be responded to as quickly and efficiently as possible.

If a computer can do the work of identifying what a customer needs and why, then that customer can be assigned more quickly to the right person to speak with. This saves time both for employees and customers, hopefully cutting short any opportunities that customers have to become frustrated.

Text mining also allows for additional personalization in the customer service process or customer experience. If all of a customer’s interactions with your business have been mined for meaningful phrases or patterns, you not only know what to expect but what to avoid or increase.

Responding to a customer’s wishes or indications helps that customer feel like you’re listening, which makes them happy – and a happy customer is the entire goal of customer service.

• Efficiency – analysis via computer is almost always faster than analysis done by a human.
• More information – computers simply have a greater capacity to find, store, and remember data than humans do on our own. Text mining software can analyze vast amounts of information, looking for patterns on a much grander scale,
• Precision – simply stated, using text analysis software practically erases the possibility of human error in analysis. There will be no important reviews overlooked because someone was sleepy or conclusions drawn on hunches that may or may not be true. Instead, all analysis stems from a computer.
• Comparison – because most text mining software provides such easily digestible visual representations of findings, these findings can be easily compared and reviewed. This can be especially beneficial in comparing competitor data to your own organization’s data.

Of course, not all text mining software does the same thing, and not every business will use such software in the same ways. Here, we go over uses for some of the most popular techniques.

 

As we’ve mentioned before, sentiment analysis is exactly what it sounds like – an assessment of customers’ feelings towards you, your competitors, or other related entities. Even a very general understanding of this can help you determine how you’re doing from your customers’ perspectives.

There are three ways of completing sentiment analysis, and the three techniques capture slightly different information.

The first is a polarity analysis. This measures if any given piece of data is negative or positive in its tone. The second is categorization. This places the piece of data in a more specific ranking – instead of “good” and “bad,” you have categories like “angry,” “confused,” “frustrated,” or “pleased. The third is a scale, ranging emotions and ranking them on a scale from 0-10.

One of the biggest tricks with sentiment analysis is detecting irony or sarcasm. While many tools have a way of detecting this sometimes, few are consistent and correct. To better understand irony and sarcasm, text mining software will have to contextualize data more and more.

For a lighthearted example of polarity analysis, see this chart using the Lord of the Rings trilogy as an example data set:

 

Topic modeling is somewhat less individualized. This technique helps to sort through huge amounts of material across different documents and summarize recurring themes. As it does so, it can give a count of words devoted to each topic, and therefore classify the documents or groups of documents based on what percentage of each document is about a given topic.

One of the problems identified with topic modeling is that it doesn’t easily scale between different sized data sets, which makes subsets or sub-themes incredibly important in analyzing larger groups of data.

Term frequency – inverse document frequency (TF-IDF) is another useful text mining technique. TF-IDF looks at how often certain words are used across a data set and assigns them relative importance. It creates an understanding of what terms are most useful within certain subject areas.

For example, if you have a data set of customers who have left your services, and another from customers who have repeatedly purchased your products, TF-IDF could gauge what topics were most often brought up by each group.

One of the greatest weaknesses here is that there is no checking for related but different terms. If more than one word is used for the same concept or idea, they may be counted as two separate topics and assigned a much lower importance than they should.

Named entity recognition focuses on identifying nouns within data sets. These could be identified by the pattern of capitalized letters or a particular word that is usually followed by certain nouns. For example, a number followed by “USD” is likely a monetary amount. “Mr.” followed by one or more capitalized words is probably a man’s name.

One of the challenges of such recognition is that few nouns that denote important categories – such as geographic location, name, monetary amount, or something similar – lack abbreviations. The presence of such abbreviations can confuse the software and lead to a lot of ambiguity.

For this reason, some suggest avoiding this technique unless you have staff with plenty of time and opportunity to train the software to look for exactly what you need.

Event extraction takes named entity recognition one step further by searching for the relationship between the nouns. The event where the two nouns interact then becomes the data that is extracted.

Sometimes these events are activities happening on a specific date, but other times they’re simply how a change in one noun has caused a change in another – two companies merging, for example.

Here are some of the practical ways that text analysis can be integrated into modern businesses:

Spam and junk mail filters are made infinitely more reliable when integrated with text mining. While this may seem trivial to some, the consistent detection of spam and junk mail can save the more susceptible fraction of the population from falling prey to such messages.

Social media feeds such as Facebook and Twitter are one of the largest sources of unstructured data for most companies. Having a concrete tool to sift through and organize all of that data is imperative if you’re a company that has a high level of online engagement.

Text mining software can help sort through a lot of data in very little time. In time-sensitive professions such as healthcare and law, this can be an incredibly useful approach to searching for patterns or themes across many dense sources.

Gone are the days of in-person focus groups. Another use of text mining is to take in the data from customer reviews and analyze it to determine what customers are truly satisfied with and what needs changes. This saves all of the time it would take for employees to go through such data manually, as well as any guesswork that would go into decoding reviews.

Any customer or client that interacts with a business leaves some sort of trail of information. In many cases, all of this information can be analyzed to bring the business to a better understanding of that customer’s interests and needs. If they’ve recently searched for laundry hampers, asked questions on a forum about the best laptops for college students, and chatted with a bot about the difference between twin and twin XL sheets, for example, they likely have a child starting college or are going to college themselves.

Information like this can help you market items closely related to what you know they’re interested in. Instead of generic ads for diapers or hair dye, for example, you could send this customer ads or coupons for desk lamps and laptop cases.

As beneficial as all of these techniques and options may seem, not every text analysis software is the right fit for every company. Different software offers different services that your own company may or may not need. Here are some of the text mining and analysis software options to consider:

Wonderflow is a user-friendly option that doesn’t require a lot of staff training or orientation to get started. In fact, we tell clients that they can become proficient with our natural language processing tool in less than half an hour – saving you valuable time and allowing you to get started right away.

Additionally, Wonderflow is highly accessible and provides insight in ways that are easy to understand. To learn more, you can check out an analysis of our services from one of our clients, view our other free resources, or contact us.

SurveyMonkey’s text analysis tool is included with their other services. Specifically, SurveyMonkey allows for incoming unstructured text data to be categorized by theme. These themes are paired with individual customers or respondents, so the resulting data can be sorted in several different ways to determine what subset of customers or influencers is most concern with each topic or category.

 

IBM is one of the world leaders in AI development, and they use this same technology to allow

their text analysis tool to process natural language with a high degree of clarity and context. IBM’s services are available in thirteen languages, which can be important for companies working on a global scale.

This is coupled with a high level of customization. IBM’s platform allows your employees to train their product to search for the data that is most useful to you, personalizing the tool for your own industry or organizational data.

The goal of this article is to equip you with the necessary knowledge to make a decision that will be efficient, easy to use, and cost-effective for your business. We hope you consider Wonderflow as you make this big decision and jump into the world of text analysis.

With so many options for text analysis software and applications, it can be difficult to decide just what is right for your company. What isn’t difficult to determine is that text mining can be helpful. With uses in sales, marketing, and customer experience, text mining is a multi-purpose tool that has a place in any business seeking to be successful.