MessinaBot: What worked and what didn’t

Last week MessinaBot launched on Messenger. MessinaBot, one of the first public personal bots of its kind, was built by Olabot. It provides basic information about me and my interests, and can act like an automated concierge for simple requests like posting a product to Product Hunt or scheduling live chat office hours without requiring any intervention on my part.

I’ve carved out a few one-hour slots in a special Google Calendar that my bot fills up in 15-minute chunks (with up to 10 participants per segment). I open office hours by sending a command to MessinaBot that does two things: notifies each attendee that I’m ready to start chatting via my bot, and disables MessinaBot’s autoresponder for those attendees.

[aditude-amp id="flyingcarpet" targeting='{"env":"staging","page_type":"article","post_id":1995957,"post_type":"guest","post_chan":"none","tags":null,"ai":false,"category":"none","all_categories":"bots,","session":"B"}']

In theory, this all sounds great, and technically went off without a hitch. In practice, several things went wrong with this model. I’m definitely still stoked about bot-mediated interactions, and to that end, here’s what went right, wrong, and how the experience could be improved.

What went right

Thirty-two people used MessinaBot to sign up for the first office hours, and another 120 are signed up for round two! I’d call that pretty great engagement! MessinaBot made it easy for me to see the roster of attendees, and then to open the office hours through a secret superuser menu:

Tapping the “Go Live Now!” button opened up 10 individual conversations through Facebook’s Pages Messaging tool. Good so far.

I had several interesting topical conversations with different participants  —  exactly what I was hoping for. However, as each conversation developed, I fell behind on others, which is exactly when things went south.

What went wrong

If you were paying attention above, it might have seemed insane to slot 10 people into 15-minute increments. Well, my friend, you are correct.

I mean, I’m as good at multitasking as any other human, but even under the most ideal circumstances, it’s hard to carry on three simultaneous conversations, let alone ten.

So if humans are naturally cognitively ill-equipped for this kind of task, try wrapping the metaphorical equivalent of two shellfish bands around your four non-thumb fingers and keeping up with nearly a dozen chat-based conversations. That’s basically what using Facebook’s Pages Messaging is like. Contrary to the svelte experience of Messenger.com, it’s slow, cumbersome, and constantly got in my way:

[aditude-amp id="medium1" targeting='{"env":"staging","page_type":"article","post_id":1995957,"post_type":"guest","post_chan":"none","tags":null,"ai":false,"category":"none","all_categories":"bots,","session":"B"}']

It is clearly not designed for a high volume of simultaneous chats as I soon discovered.

Load balancing conversation is an art unto itself

You might note in the above screenshot that my bot has been receiving messages every few minutes since it launched on June 28 (we crossed 20K messages in the first two days!). While this activity is incredibly promising, it also meant that during my office hours, non-attendees would pop to the top of the conversation list, making it impossible for me to track the live chat participants! This little snafu lead to additional filtering overhead for me, coupled with a daunting reality: I do not scale.

[aditude-amp id="medium2" targeting='{"env":"staging","page_type":"article","post_id":1995957,"post_type":"guest","post_chan":"none","tags":null,"ai":false,"category":"none","all_categories":"bots,","session":"B"}']

So what else went wrong?

Conversation switching placed a significant cognitive load on me, and I had to read up in each thread to revisit what I’d already said, slowing down conversation.

Switching between threads in the Pages Messaging UI could take 4-7 seconds, and my agitation skyrocketed every time Chrome froze as the thread loaded. It was shocking how poorly this page performed!

I became increasingly flummoxed as more office hour participants arrived and wanted to connect with me, and I couldn’t get to them fast enough. My intention was to provide an interesting, novel experience — not a frustrating one! Once it became clear that the Pages Messaging tool was going to keep getting in my way, I wanted to plead for patience from the attendees, but couldn’t, because MessinaBot doesn’t support broadcasting to subsets of users yet. Consequently, I messaged each participant individually to apologize

MessinaBot required me to explicitly end one session before transitioning to the next:

[aditude-amp id="medium3" targeting='{"env":"staging","page_type":"article","post_id":1995957,"post_type":"guest","post_chan":"none","tags":null,"ai":false,"category":"none","all_categories":"bots,","session":"B"}']

This would turn on the bot’s autoresponder for the previous cohort of users, and turn it off for the next batch so I could talk to them directly. Of course, after 15 minutes without so much as a hello to the first 10, I felt compelled to start the next session on time, leaving the first group in an ambiguous state. I felt trapped inside a real world game of Tower Defense with the waves of participants outflanking my conversational gunners. As things flew off the rails, I feverishly switched between threads, typing like a madcat, channeling my inner Bill O’Reilly: “FUCK IT, DO IT LIVE.” Things had gone horribly astray; there was no way to catch up. It was time to end the experiment and regroup. I painfully notified each participant that I was ending office hours early and would try again once these issues were ironed out.

What could be improved next time

You know, we think bots are all cool and sophisticated, but it’s still the simple things that trip them up.

Better participant management

[aditude-amp id="medium4" targeting='{"env":"staging","page_type":"article","post_id":1995957,"post_type":"guest","post_chan":"none","tags":null,"ai":false,"category":"none","all_categories":"bots,","session":"B"}']

Live chats and AMAs aren’t new. In fact, I did one not long ago for Postman in Slack, and it went really well, thanks to Priyanka Sharma’s moderation. In the future, MessinaBot should field and filter questions from the participants, stack rank them, and then offer them up to me to answer one by one, Q&A style, i.e.: “Drew Austin would like to know What have you been learning about bot UX?”

Focus on the intimacy of the group

One of the assumptions built into the current approach is that each participant should get an individual conversation with me. But to scale more effectively, I think it’d be better to create a shared context for conversation. While not all conversations are relevant to all comers, at least there’d be a shared context in which people could get a sense for what I’m doing when I’m not talking to them. With the single-threaded approach, attendees are left waiting without their expectations set for when (or if) I’ll ever get to come say hi. Perhaps the bot could manage their place in line (“You’re #6 with three others in front of you”), but that too feels less satisfying than jumping into a shared-yet-intimate conversation space.

Tools for admins are key

[aditude-amp id="medium5" targeting='{"env":"staging","page_type":"article","post_id":1995957,"post_type":"guest","post_chan":"none","tags":null,"ai":false,"category":"none","all_categories":"bots,","session":"B"}']

In order to to disable MessinaBot’s programmed replies, the user can send the “pause” command. As the botmaster, I want similar control to start and stop automation, so I can have an uninterrupted dialogue. I suggest personal bots adopt standard slash commands (i.e. /pause, /resume, /restart, etc) to make these commands consistent and unambiguous. The lack of clear and easy to use tools like these made it hard for me to feel in control and able to guide the experience. I also couldn’t get MessinaBot to shut up when I wanted him to!

Conversation assistant tools would be handy

Several times during conversations, I wanted to share a link to something I’d written or seen. This meant I had to leave the context of the conversation, go search, return, and then catch up on anything I may have missed during my absence. It would have been way more useful to have an inline reference tool that would have allowed me to pull from my existing store of links and generated content. This would have allowed me to answer questions more quickly by pointing to existing content, rather than have to rewrite incomplete or summary responses.

Final thoughts

I’m stoked about personal bots that can mediate human-to-human interaction and take care of the logistics of making those connections happen. I like Amy Ingram from x.ai and Clara from Clara Labs, but those services are currently restricted to email. The promise of conversational bots here is their ability to evolve beyond the lowest common denominator of text. They can alter and enhance their interface based on the relative complexity of a task or the needs of a user — providing form-like UI widgets, looking up information, or creating automatic follow-on actions and updates.

I’ll definitely be hosting more office hours in the future, but will either take a more modest approach with fewer participants or migrate to a group context. For now, though, if you’d like to check out MessinaBot and see if he’s got any available slots, you can talk to him on Messenger.