What 300 million users taught Kik about chatbots

Traditional chat apps gather metrics around message volume or time spent in app.

But as bots begin to enter the chat space in a much bigger way, allowing users to carry out commerce, play games, and meet personalities, Kik thinks it’s time to rethink the metrics used to prove the success of a bot or chat experience.

“I think it’s time for some new thinking, and chat is something completely different. The engagement in chat is completely different than traditional feed-based content on the web and most apps, so just realizing that people’s lives require them to multitask and that’s part of chat is actually good thinking,” Kik head of data Joel Cumming told VentureBeat in an interview.

Since January, chat app Kik has been monitoring aggregate data about its 300 million users to create a new method to follow usage habits that may be helpful to bot makers, based on active and passive usage.

Active users respond to messages within 20 seconds of receiving a message. Passive users take minutes to respond to a message.

“This isn’t eyes spent on glass the entire time but engaged in conversations,” Cumming said. “Right now this is an internal-only metric, but it’s going to serve as the foundation for bot metrics we push out in the future.”

Some initial findings shared Tuesday:

• Roughly 20 percent of Kik usage is active and 80 percent is passive.
• The 40 percent of U.S. teens who use the app on average take part in six chat sessions a day, each 12 minutes long.

Kik found that measuring success depends greatly on the purpose of a bot. If you have a restaurant bot, then you want usage to be quick and short. If you have a game bot or a narrative bot or a bot intended to engage, you want people to spend as much time with the bot as possible.

Usage tends to be passive when speaking to friends.

The active-passive breakdown wasn’t made to be the end-all, silver bullet metric that’s able to track bots most effectively, Cumming said. It’s meant to be a start.

“We’re looking to change people’s thinking for the bot world vs. the app world,” he said. “We’re still thinking about the most effective metrics to make available [to developers and user].”

Currently the active-passive metric is only being used in-house at Kik. There is no scheduled plan to share the metric with developers or users who create their own bots.

Since Kik began to allow users to make their own bots in April, Kik users have created more than 6,000 bots, and 60 bots are available in the Bot Shop. Bots not included in the bot shop may be sent to up to 50 people.