How to get people to use your chatbot every day

A little bit over a year ago, the number of monthly active users on the top 4 messaging apps surpassed those of the top 4 social networks. Recently, Facebook Messenger, WhatsApp, and WeChat have either crossed or are about to cross the 1 billion monthly active user mark. With the current scale of messenger platforms and the speed at which they are growing, every media company will eventually have to answer one question: How do I create and deliver native content on the messaging platforms?

At Poncho, we have the same goal  —  except that we want to be the best at it. To achieve that, the first step is obvious: Get people to use your bot every day. One of the most important metrics in determining the stickiness of a product is user retention rate. Putting aside buzzwords like “conversational,” we believe that building a good conversational experience on messaging platforms is the key to a high retention rate. And that’s one area where our team has been relentlessly making improvement for the last five months.

Above: Progress we have made on user daily retention

Besides some common tactics around optimizing product experience, it’s worth highlighting a few internal tools that we’ve been building to generate this improvement.

Live chat feed: We built a live feed that lets the team watch how people interact with Poncho. It’s by no means a scalable solution, but it has been super effective. We were able to identify some of the most common questions people ask Poncho and update his responses on the fly.

The bot analyzer (Google Analytics for conversations): Over the summer, one of Betaworks’ data science interns, Devin, took our “human learning” approach (having a bunch of people stare at the live feed) to a machine-learning level. He built an algorithm that treats the Poncho-user interactions as a network: Each conversation is a node, and the transition between conversations is an edge.

The algorithm goes through the entire network regularly to discover popular/unpopular conversations, how users move from conversation to conversation, and where they tend to drop off in the flow. The most interesting feature is that the tool can start making recommendations to our editorial team about which conversation flows have the most impact on overall traffic in the network. For example, if we can reduce the bounce rate of a flow (the rate of people who stop having conversations with Poncho) by X percent, we should expect Y percent increase in the overall number of user sessions and message exchanges. This starts eliminating more and more of the guesswork when the team is writing conversations. In a way, the algorithm is like Google Analytics, but for conversations.

The next tool is the CMS, where our editorial team updates scripts and visual elements (emojis, GIFs, etc) in real time without the involvement of the engineering team. The system was built on RiveScript. On average, our team deploys 50 to 100 edits per week.

Complementing the CMS, we also rolled out our notification dashboard, which enables editors to craft notifications and target users by location, demographics, weather conditions, etc. Faced with chatbots, most humans become “introverted,” because we humans are just started learning how to talk to bots, and most of the time, we don’t even know how to start the conversation. Indeed, our internal metrics show that more than 70 percent of conversations between Poncho and users are started by push notifications.

These tools are still in the infancy stage, and there is lots of work to be done to make them more efficient (WE ARE HIRING!!). But we’ve started to see the potential of building fun conversations for the bot through a data-driven approach.

The past five months have passed in the blink of an eye. The entire team has been focusing on three “ones”: one platform (Facebook Messenger), one content vertical (Weather) and one metric (Retention). The team is pretty happy with the current progress we have made on the retention metric and has decided to branch out to new platforms and content verticals.

1. We quietly pushed out Poncho Horoscope 0.1 via the current Facebook bot. You can try it out by messaging “horoscope” to Poncho.

2. We launched our bot on Kik yesterday. Scan the code to try it out on Kik.

Above: Poncho on Kik

We really enjoyed building the Poncho bot. But we also think that in the future everyone will have more than one bot they interact with on a daily basis. The way my partner in crime, Sam Mandel, puts it: “It’s like the Marvel Universe. Every bot is a superhero who has a distinctive personality and a superpower. And they can achieve even greater things when they work together.”

So look out for new bots from the Poncho team. We’re excited to see what the future will look like when everyone has a few “superheroes” in their pocket, car, living room, and so on.

A special thank you to James Cooper and Stephanie Chan for reading and editing my post. And as always, go team Poncho! 

Previous updates:

1. Poncho on Facebook Messenger: three months in
2. Two weeks after we launched Poncho at F8

This post appeared originally on Medium.