Why botsters need to get a CLU

My recent bot manifesto warned that bots better smarten up fast, admonishing our current bot generation for being not only dull but also dumb. I said they are lacking both personality and basic natural language processing skills. However, this is clearly a more nuanced problem: Chat varies wildly from spoken language. In fact, chat is a unique IM-based dialect consisting of slang, abbreviations, shortcuts, and emojis that are purpose-built for mobile devices to save time typing.

To help, I’ve provided four tips to help chatbot developers get a CLU (which stands for chat language understanding) and make their bots better and smarter.

1. Chat ain’t small talk

Colloquial chat doesn’t fly as dinner table conversation. This fact was lampooned in a hysterical Curb Your Enthusiasm episode called “Social Assassin,” in which Larry David’s friend contracts Larry to convince the guy’s wife to curtail her annoying habit of annoyingly saying “LOL” out loud. This ushers in the Vegas principle: What happens in chat stays in chat. Chat messaging lingo is highly contextual in nature. It was shorthand developed primarily from the difficulty of typing long sentences in SMS on telephone keypads and grew from there. There’s a big difference between understanding complete grammatically accurate sentences and understanding dialogue, which is often less grammatically accurate than chat.

2. CLU is not the same as or better than NLU

For natural language processing (NLP) and natural language understanding (NLU) to work for chatbots requires CLP/CLU. Unfortunately for developers, chatbots need to be good at bilingual parsing and understanding both normal, grammatically correct language and chat lingo.

Problematically, chat lingo is evolving very rapidly. Android’s Nougat release boasts at least 72 brand new emojis. Evolutionary chat linguistics is now an integral part of the chatbot game for brand marketers and developers alike. Wrangling emojis is particularly tricky since so many of the emoji libraries vary. The platform companies themselves have a double advantage over individual bot developers: They have the top-down advantage of being able to exert full control over new emoji introduction and the bottom-up advantage of monitoring aggregate IM traffic to archive and learn emergent grassroots chat slang. So not only do bots need CLP/CLU, they also need ELP/ELU — emoji language processing/emoji language understanding.

When people send emojis, they are doing so with the understanding that the person on the other end of the conversation in the chat window will understand the emoji and interpret it properly. Bot developers and brand marketers should have the expectation that their bot’s friends will do the same exact thing — send emojis — and these bot providers must be prepared to respond properly to the emoji’s intent. Expanding emoji lexicons increase the complexity for bot developers.

3. A picture is worth a thousand words

It’s possible bot developers familiar with logogrammatic writing systems (Chinese, Japonic) may be slightly culturally advantaged in decoding emojis as it appears that chat is leaning in a logographic direction. Who knows, maybe reading chat in the future is going to require a James Spader-type linguist called upon to do a Star Gate Reveal.

4. This is the return of the dingbats

Back in the ’70s when I first learned about typography, genius font designers like Hermann Zapf built entire picture-based fonts like ITC Zapf Dingbats for editorial designers and typesetters to embellish print layouts with, with icons telling readers to flip the page or marking the end of the article. Dingbats were the print equivalent of today’s chat emojis — they imparted commands or emotions. This raises that classic theme: Everything old is new again. Chat emojis are the new dingbats, and we better embrace them because they’re here to stay.