Dear Facebook Messenger: We’re in a relationship, but it’s complicated

Have you talked to a chatbot yet? Everyday brands are releasing bots to provide everything from trivia games to customer support to flower delivery, but chatbots aren’t working perfectly yet.

Let’s explore what you need to know before you start building one for your brand.

Be realistic

At YourMechanic, we were exploring the idea of an artificially intelligent chatbot a few months before the Facebook Messenger platform was announced. The idea was to have an automated system that could serve as a first responder to any customer inquiries and then fall back to a human agent when needed.

The purpose was to automate repetitive and mundane inquiries while leaving the more complex conversations to our agents. The expected result was happier agents whose time was more efficiently used for tasks requiring specialized skills, which in turn would lead to a better experience for customers.

Our initial feature set for the chatbot was ambitious, and we soon adjusted our course to start with the most important functions for our customers: the ability to secure a car repair estimate, ask a mechanic a specific question related to their car, and browse our library of articles and answers on more than 25,000 automotive topics.

Learn the platform

Before building a chatbot, we learned a handful of important details and limitations about the Facebook Messenger platform.

The Messenger API is largely a pass-through for messages from the customer communicated via HTTP POST requests to a designated webhook URL. There are a few AI frameworks that provide frameworks for management of the context of discussion and the flows around them, like Api.ai and Wit.ai.

While these are useful for general conversational logic, complicated logic — such as a variable set of questions that need to be asked when a user requests an estimate for car repair, which is tightly entwined in the system — is sometimes simpler to implement natively. Moreover, these frameworks don’t yet have some important features, like fuzzy match on a client-supplied dictionary of words (car names or repair services, for instance).

Also, the Messenger platform isn’t purely conversational. It provides the ability to show actions in the form of buttons and cards that users can click on to complete an intended action. This reduces the need to implement complex natural language processing (NLP) on user responses to determine their context and intent. Consequently, it is quicker to get a minimum viable product (MVP) done with moderately complex flows.

The Messenger platform is still in its infancy, and it has limitations that impede development:

• There is no good way to collaborate on the bot development process, and throwaway pages need to be created for each developer to be able to do the development. Moreover, a different page needs to be used for development vs. production.
• Access token changes occur after app approval without any indication or notification. However, to Facebook’s credit, the app approval process itself is extremely quick. The site claims it could take up to five business days, but our app was approved in 15 minutes.
• Webhooks are separated from the Messenger app for some reason, and it is not intuitive that a second app (called webhooks) must be added to manage them.
• Account creation (user signup), even via Facebook login, is not supported. The bot needs to request user information to create an account, and it is not safe to request users to provide their passwords in an unprotected interface.
• The features of the platform are largely oriented to ecommerce/retail/product (“buy” mentality), and there is room for growth to cater to marketplaces and services (“book/schedule” mentality), like those offered by YourMechanic.
• When the webhook request returns an error, Messenger throttles the message transmission, which slows development significantly.

Things are changing, however. Facebook is now making rapid improvements on the Messenger platform. They have released integrated webviews support and payment processing support via PayPal, Stripe, and others. These updates significantly mitigate the limitations listed above, as webviews could be implemented to get around most of them.

Building our bot

For the YourMechanic chatbot, we chose to not use any of the existing chat management and AI frameworks, like Wit.ai and Api.ai, as we needed tight integration with flows that already exist on our website.

For effective parallelization of development, we first set up a protocol for each of the contexts to manage their individual flows. Thereafter, three engineers picked up the top-level contexts — get an estimate, get advice, and ask a question — as the functions are largely independent. These features more or less followed the previously defined flows that users are familiar with on our web offering, while adapting the presentation of these flows to the user in a conversational manner.

It is important to note that when developing a conversational product, the difficulty isn’t in the flows where the user behaves as expected, but rather in how the bot manages unexpected responses. Ideally, a bot would respond in a humorous and self-aware manner, but as we were developing an initial MVP, the basic requirement was to guide the user towards their goal.

A few simple guidelines were applied to fulfill this requirement:

• Provide examples to the user to clarify the expectation when an unexpected response is received.
• Give the user an option to exit out of the flow to higher levels from any stage. For example, if a user is providing the model for a car and the response isn’t understood multiple times, the user is likely trying to exit the current flow.
• Ask questions with a finite set of possible answers when possible so that it’s easier to guide the user to a defined goal.
• Try to avoid building too many levels into any user flow as it makes it hard for the user to switch contexts when needed.
• Recognize keywords that indicate another flow or action that the user wants to perform, and provide a good transition out of and into a context.
• Build a mechanism to notify a human agent to take over after several unsuccessful attempts to understand the user’s intent.

Since we released our bot about a month ago, we have had several surprises and gained important insights into what users expect when they’re conversing with our bot. For example, we found that when a message reaches a certain character length, it’s typically because someone is trying to relate a negative experience they have encountered.

To handle this, we programmed our chatbot to react to extra-long posts by deferring to one of our team members, who reaches out to the customer.

As the popularity and ubiquity of conversational platforms grow, more and more businesses will need to decide whether to devote resources towards the development of a bot.