4 steps to chatbot retention

To survive, businesses need to find and engage customers. To go beyond survival and achieve success, they need to both grow an audience and retain customers over time. A post-mortem analysis of over a hundred failed companies found that the inability to identify the right market and genuinely connect with customers caused them to fold up and implode with a whimper.

What happens when you bring AI-driven bots into the picture? Things could get messy.

According to Kissmetrics, around 70 percent of customers of a business leave because they received poor treatment, while only 14 percent do so because of dissatisfaction with the product or service. If customers care more about how you treat them than about your product, should you leave them at the mercy of algorithms?

Well, there’s promise. Amit Sharma is CEO of Narvar, a company that specializes in post-purchase customer service for retailers like BCBG, DSW, The Limited, and GameStop. Narvar’s chatbots allow users to opt in for follow-up notifications for their transactions, which typically include inquiries around package status (20 percent of customers) or delivery address and time changes (10 percent). Previously Narvar sent millions of SMS messages to deliver these notifications, but found that less than 1 percent of customers engaged because they didn’t believe anyone was reading their messages on the other side.

Above: Narvar’s chatbots, such as AskGameStop, see nearly 100 percent engagement rate from customers, compared to under 1 percent engagement for identical notifications delivered over SMS.

With chatbots, the story changed. Narvar’s chatbots see almost 100 percent response rates from customers, 65 percent of whom treat the bot the same way they would treat a human, saying things like “Thanks” or sending emojis and thumbs-ups. By default, people see chatbots as a bi-directional channel, as opposed to SMS, which was perceived as a uni-directional push. These chatbots replace and streamline the service provided by call centers and email response teams, leading to cost savings for retailers and more efficient experiences for customers.

But there’s also peril. Chatbots are programmed by humans, and humans aren’t perfect. According to Ilker Koksal of Botanalytics, nearly 40 percent of users stop talking to a bot after the first message and another 25 percent are gone after the second one. Sadly, daily and monthly retention numbers aren’t stellar either, with daily rates hovering between 1 and 2 percent and monthly rates barely exceeding 7 percent.

Bots have the potential to significantly enhance customer engagement, but poor chatbot design can lead to disappointing user experiences. How do you ensure that your chatbots positively engage new users and retain existing ones?

1. Identify the basic ‘unit’ of conversation

Mixpanel powers over 20,000 products across web, mobile, and bots — including every single bot on the popular Kik messaging platform. According to Shawn Hansen, Mixpanel’s CMO, one key question bot developers should ask themselves is “What is the wrapper around a conversation?” In other words, what is the basic “unit” of a successful bot interaction?

The answer differs dramatically depending on the type of bot. Simple human exchanges often begin with “Hello” and end with “Goodbye,” yet they can vary dramatically in between. Hansen says the top five performing categories of bots are weather, fashion, humor, knowledge-based quizzes, and notification-driven news. A weather bot like Poncho often starts with an inquiry and ends with a forecast, while a fashion bot might end with a recommendation. An ecommerce bot may start with a search and end with a purchase, while a news bot ends with links to content.

Different communities also drive different levels of engagement with bots. Since gamers are often highly engaged and excited for new product releases, GameStop’s chatbot sees over 2 times more engagement than other retail chatbots powered by Narvar. Sharma also points out that merchandise category makes a difference.

For categories like high-profile games and coveted health and beauty products, customers often ask about new releases and start different conversations than the ones they use to check on pre-existing transactions. Price also matters. More expensive items see between 2 and 3 times higher engagement rates than cheaper ones.

2. Drive true performance with strategic analysis

Understanding a bot’s unique conversation structure — based on category, function, and user base — is the beginning of deeper analytics that drive chatbot retention and engagement. Only after correctly defining what a core conversation consists of can you answer important questions to improve bot performance. One common mistake bot developers make is to overfocus on an isolated metric, such as response speed or volume of responses, rather than higher level success.

While Mixpanel measures many chatbot-specific metrics, such as response time (both of human and of bot), conversation verbosity, and richness of interaction (use of emojis and other media content), Hansen emphasizes that a key conversion metric is whether a user subscribes to your bot or otherwise indicates they’re interested in a long-term relationship. “Success for most companies consists of daily engagement,” Hansen points out.

“Bots that offer a daily recommendation, or are otherwise connected with a daily or weekly ritual, have a better shot at getting that subscription and frequent engagement,” adds Aaron Krivitzky, a Mixpanel solutions architect with expertise in bots. However, not all bots have a natural daily ritual to design around, so what can be done instead to improve retention and engagement? Analyzing failed conversations is a start.

Companies often think of the first set of interactions (how to engage), but should instead focus on what makes people come back over and over again. Krivitzky advises bot makers to nail the answer to the question “How does the next version of the bot improve upon the first one?” Not thinking strategically about long-term relationships often results in the common failure of users never getting past the first two interactions.

For humor bots, a core conversation often ends with a punchline, and a measure of success is whether a user liked or responded to the joke. By analyzing and identifying patterns of failed jokes, where users never reacted to the punchline, a bot’s development team can adjust factors such as joke choice, timing, emoji use, and alternative media such as images or video to salvage a conversation. Similar strategic analyses and adjustments can be made for bots of other categories.

3. Benchmark against the tech the chatbot will replace

Chatbots are a new and growing technology, so the industry is still figuring out what baseline retention and engagement rates should look like. In the meantime, bot developers can benchmark against the technologies that they are replacing, such as SMS, emails, mobile apps, or advertising campaigns.

In the case of Narvar, their SMS notifications inspire less than 1 percent engagement from customers, but their retail chatbots get nearly 100 percent engagement performing identical functions — because users perceive Facebook Messenger to be bi-directional and conversational. The chatbots also alleviate pressure on human customer support staff responding to phone calls and emails.

Kuan Huang, founder of the popular weather bot Poncho, initially struggled with poor retention on Facebook Messenger despite having a popular mobile app and email newsletter. After segmenting out and analyzing their most engaged users, the Poncho team discovered that their killer use case was proactively alerting people for unexpected weather events, like sudden rain. Optimizing their bot for such use cases lead to a 300 percent improvement in engagement and superior performance over the mobile app.

Chatbots can also be used to complement traditional advertising campaigns like video marketing or banner ads. Disney recently partnered with Imperson to produce the Zootopia chatbot, which engaged users to solve crime mysteries with the movie’s protagonist. If they solved the mystery, users got customized name badges from the Zootopia Police Department, which successfully drove high volumes of social shares.

When used in combination with existing social media followings, chatbots can be leveraged by brands to offer millions of customers personalized one-on-one conversations at scale. Across Imperson’s brief campaigns for Disney, Universal, and other clients, their chatbots exchanged over 100 million messages over conversation sessions that lasted over 9 minutes on average, much longer than a user typically spends watching a video trailer or glancing at a display ad. And 87 percent of users reported positive experiences with the chatbot, much more favorable than most traditional ads.

Due to the lower barrier to entry to using bots on a popular messaging platform versus having to separately download an app, bots can also offer engagement beyond previous technologies, when strategically used. YouTube recently partnered with Kik on a bot for the YouTube Fan Fest in Toronto. The bot managed the festival logistics, raised awareness of the event, facilitated ticket purchases, tracked attendance, and even encouraged festival goers to meet each other. Over 10,400 unique users interacted with the bot, which achieved over a 15 percent conversion rate to ticket purchases and generated nearly 1,000 matches between users over the course of the event.

4. Prepare for scale

Managing conversations with users becomes unwieldy when you have millions of users. Scale can be both an opportunity and a hindrance: You get more overall conversations to use as training data, but common metrics like response time and quality of responses can suffer due to the variability and volume of input your bot receives. Bots usually see retention and engagement numbers decline with increasing numbers of users, with the natural exception of bots that depend on “network effect,” like social communities or dating matchmakers, whose value improves with volume.

Mitsuku is a popular chit-chat bot whose only purpose is to provide entertainment and companionship. Steve Worswick, the chatbot’s developer, has seen usage skyrocket to over 5 million users across web and mobile. Yet the bot continues to intrigue users, 80 percent of whom come back for more interactions after their first encounter. The longest session any user has had with the bot was 9 hours, longer than most humans can last in continuous conversation.

Worswick implemented a number of features to keep Mitsuku entertaining even as her user base swells. First, he implemented a massive common object database that tracks attributes and relationships of items like “banana,” “movie,” and “cat,” allowing Mitsuku to comment logically on the world around us. Second, he carefully monitors user logs to identify trending topics and adds timely and relevant commentary. For example, Mitsuku has her own witty answer if you tell her who the new President of the United States will be:

The challenge of retention

While many bots purport to provide utility to users beyond traditional websites and apps, very few actually convince users to stick around long enough to reap the benefits. Overcoming habit and incumbent inertia requires bot developers to hone in on what a “successful” conversation truly consists of, analyze both failures and successes, and experiment with both strategic and tactical measures to drive such conversations. Otherwise both users and companies will stick with websites, mobile apps, and traditional advertising methods that provably outperform risky and experimental technologies like bots.

Additionally, developers should keep in mind that retention and engagement naturally vary with bot category. Just as with mobile apps, bots that naturally tie into a daily or weekly ritual see the best retention and engagement numbers, as do bots that capitalize on vibrant communities like gamers or beauty enthusiasts or those that handle high price or high stakes transactions.

Finally, retention and engagement numbers can both suffer and improve with scale, but the latter only occurs if developers utilize the increased volume of data to stay on top of changing trends and user needs.

This post appeared originally at TOPBOTS.