You’ve realized the value of conversational marketing, and are excited to launch your company’s first chatbot. You’ve received buy-in from leadership, but how are you actually going to measure your chatbot’s success?

You’ll need to start with some objective data that can help you make sense of chatbot performance. Since there are so many different ways to measure performance, you need to narrow down what chatbot key performance indicators (KPIs) you should be following on a regular basis.

We’ve compiled six key KPIs to track through your chatbot analytics that will help you understand where your bot is excelling and where it might need a little more work.

Determining How You Will Track KPIs

While there are some businesses with the bandwidth and resources to create custom chatbot analytics dashboards, it’s not something every company has the luxury of executing.

Does the framework you will use to build your chatbot offer robust reporting? If not, you may want to consider established chatbot analytics solutions like Botanalytics and Dashbot are popular options. They’re relatively user-friendly, so marketers with minimal chatbot analytics experience can get up to speed pretty quickly.

Analytics options vary in terms of price and capabilities, so it’s worth doing your research when you’re ready to find a reliable solution.

Chatbot KPIs You Should Be Tracking

If you want a more in-depth look into how your chatbot is performing, you need to track specific KPIs through your chatbot analytics. We’ve compiled a list of six essential KPIs you should be monitoring regularly for your chatbot.

KPI #1: Missed/Failed Utterances

Natural language understanding (NLU) for chatbots has come a long way in recent years, but there are still some things that get lost in translation.

Unfortunately, the bot isn’t always able to understand what the user means in the interaction, which then becomes a missed/failed utterance. In many cases, the bot might simply not respond or attempt to start the conversation over.

Why it matters

Reviewing missed/failed utterances in the conversation logs of your bot can help you identify bot functionality that users are looking for that you haven’t yet created. For example, if you own a restaurant business that helps users book reservations online, you may learn by looking through failed utterances that a large percentage of potential customers who interact with your bot are asking about outdoor seating—a question that your bot doesn’t currently answer. This may be a signal that your bot should add functionality to address this user utterance. 

While it might be impossible to account for every possible misspelling or user utterance, your bot should have a fallback intent in place. This phrase lets users know that the bot can’t fully process what is being said and that further help (generally from a human) may be required. The fallback intent can be as simple as a message that says, “I’m sorry, I didn’t understand that. Would you like to speak to a human?”

KPI #2: Interaction Rate

In the world of conversational marketing, interaction rate is one of the most important engagement metrics you can track. This KPI shows you how many messages are exchanged between a user and chatbot on average during a single session.

Why it matters

A high interaction rate means that users are actively engaging with your chatbot and using it as the resource it was designed to be. If you have a low interaction rate, it could be the result of different problems.

One issue could be that the interactions themselves are glitchy or don’t flow naturally enough for users. In that case, you would need to reassess your chatbot script to ensure it’s up to snuff. If it’s a technical problem like a glitch, however, you may need to speak to IT support at your company or your chatbot provider.

Another issue could be that your bot isn’t visible enough. People may go onto your website and not even realize there’s a bot they can interact with.

Source: Michelin

In the example above, tire company Michelin has deployed their chatbot to automatically introduce itself to website visitors and ask if they have any questions. Make sure that your bot’s greeting indicates what it can do to help users navigate information about your business.

KPI #3: Retention Rate

Much like you’d measure how many users are returning customers, you also want to know your chatbot’s retention rate. This KPI refers to the number of people who are coming back to your chatbot and interacting with it on more than one occasion.

Why it matters

This KPI helps you gauge people’s satisfaction with your chatbot. If people interact with your bot frequently, it’s clear that it provides value to users.

Invest in creating hyper-personalized chatbot experiences to drive your retention rate even higher. The more data your chatbot can retain about users from past conversations, the more value it will be able to provide in future conversations. This personalization is a great way to build a longstanding customer relationship where your bot functions as a trusted brand advisor.

KPI #4: Goal Completion Rate (GCR)

Your chatbot’s goal completion rate (GCR) refers to the number of preset conversions your chatbot facilitates.

Start by determining the goal you want your chatbot to carry out. If you’re a SaaS company, your bot’s primary objective might be to get a user to sign up for a free trial of your product. In this case, a free trial signup counts as a completed goal.

Why it matters

The entire success of your chatbot is predicated on the goals you set for it. That makes GCR arguably the most important KPI to track. If you’re seeing a very low GCR, there are a few different ways you can handle the issue.

First off, adjust the chatbot script itself to guide users toward the goal more smoothly. Sometimes bots can throw too much information at users, and the action you want them to take gets muddled. Map out logical script scenarios that get users to your primary goal, whether that’s a sale or email signup.

You can also adjust goals as needed. Sometimes a low GCR might mean that you’re simply tracking the wrong goal. Maybe you should be tracking how many leads you’re bringing in versus the number of sales through your chatbot. Make sure to check in on GCR performance on a regular basis to see if any adjustments should be made.

KPI #5: Human Takeover Rate

Human takeover rate tracks how often users are asking to speak to a human during a chatbot interaction. This request may happen because the user is frustrated, or they may just need more detailed information than your bot can provide on the spot.

Why it matters

If live rep requests are happening more often than not, consider implementing a blend of live reps and bots. You might let your bot answer basic user questions and hand the conversation off to a live rep when they need more specialized support.

Source: PostFunnel

A high human takeover rate might also signal a larger issue about your chatbot’s overall performance. If a large number of people are looking for human support, there might be a problem with your chatbot’s scope of answers. This could be the result of the chatbot script itself or any software bugs that stifle conversations.

KPI #6: Total Number of Users

Tracking the total number of users for your chatbot is really just that—finding out how many people are using your chatbot at any given time.

Why it matters

If you’ve just launched a new chatbot for users to engage with, this KPI is useful for making sure you’re getting a return on your chatbot investment. Track this KPI at the very beginning of your chatbot’s lifecycle to see how it evolves over time.

If you’re seeing a significant increase or decrease in the total number of users, adapt your chatbot strategy, or the channels on which your chatbot lives, accordingly. With an increase in users, you can invest in more advanced capabilities for your chatbot to handle. If you’re facing a decrease, you should analyze your chatbot performance carefully to see why people aren’t using it.

You should also keep a close eye on how many of your total users are new or returning. A high number of new users with a low number of returning users could mean that regular users don’t view the bot as a value add. On the flip side, a high number of returning users with lower new users could mean there’s only a certain segment of your audience that enjoys interacting with a chatbot.

Review Your Chatbot Conversations, Not Just KPIs

KPIs are a key piece of the chatbot analytics puzzle, but they don’t give you insight into everything. If you want additional context into chatbot performance, you should also be analyzing chatbot conversations.

It’s important to regularly review your chatbot conversations to get a better understanding of how people are interacting with your bot. If you are in charge of maintaining and improving a bot, block off some weekly time on your calendar to go through a handful of conversations to uncover any underlying user trends. The goal is to find any gaps in chatbot interactions and also examples of how it’s working to its potential.

The post 6 Essential KPIs to Track Through Your Chatbot Analytics appeared first on Alexa Blog.

Read more: blog.alexa.com