Bots don’t need to pass the Turing test — just the beer test

The Turing test is a test, developed by Alan Turing in 1950, of a machine’s ability to exhibit intelligent behavior equivalent to, or indistinguishable from, that of a human. When I joined Slack, my vision was to help developers build bots that pass the Turing test 2–3 times a day. As time passes, I understand the “beer test” might be much more important.

So what is this beer test?

Now the Beer Test is much less complex than the Turing Test: When I worked in New Zealand, I was asked to interview several engineers and product managers. In addition to assessing their tech and product skills, I had to answer this question: “Would you go out for a beer with this person?” (If you’re not into beer, substitute sushi or hummus or hanging out.) It’s a simple question but one of great depth:  Do you think this person is someone you’d want to work with every day? Is this a person who will not only be effective but is also a good person? At Google we called it Googliness, although I am not sure that it has an accurate synonym. I would say that a person who passes the beer test is a delightful, interesting, and fun person.

How to make your bot pass the beer test

Here are some best practices collected by our team from an internal Slack bot hackathon and working with bot developers:

• A bot should always support the “help” command and reply with a short manual on how to work with it.
• A bot should respond to a DM or a mention, even when it does not understand it. Saying “I did not understand” is better than ignoring.
• A bot should not DM all the users in a team, unless there is a fire!
• A bot should always introduce itself to a channel politely before sending too many messages (and ideally it shouldn’t use @channel or @here).
• A bot should always ask people to opt-in to interacting with it rather than emailing or DMing everyone on a team. Ideally, work with the person who installed your bot to facilitate the onboarding of new people to interact with the bot.
• A bot should support a “mute” or “pause” request when applicable.
• Before a bot sends a bunch of unsolicited messages, have the bot send something like, “I have something for you. Is now a good time?”
• Carefully consider whether your bot is one that should be used in public channels, or in DMs, or both. Sometimes picking just one and sticking to it makes it much easier to set expectations with users.
• Whenever your bot doesn’t understand what someone is saying, give simple usage hints and possibly a link to more detailed usage instructions.
• Personalize your bot to each user; let it remember the preferences and defaults of individuals on the team so that future interactions go more smoothly.
• Randomize your answers, even if you are trying to say the same thing:

Why strive for delightful bots, not humanized bots?

There are several reasons service providers should build delightful bots rather than humanized bots:

• Building humanized bots that pass the Turing test is super hard; it involves NLP, complex conversational skills, and much more.
• The technologies for building a great conversational user experience are available today.
• You can provide a great service without requiring all the complex skills humans have.

As we move to the conversational office era, it is time to take the step forward and provide your service as a bot — not an impersonator bot, but a friendly and approachable bot. It’s easy peasy.

Amir Shevat is head of developer relations at Slack.