IBM researcher can decipher your personality from looking at 200 of your tweets

An IBM researcher says she can make a good educated guess about your personality just from looking at 200 of your Twitter messages.

That may seem obvious, as people typically express their personalities through the 140-character messaging service. But Michelle Zhou, a researcher at the IBM Almaden Research Center in San Jose, Calif., can do this personality analysis in a disciplined and automated fashion using the science of data analytics. Companies that pay attention to this research could save hundreds of millions of dollars — and stop annoying people.

She used it on me, and my personality graph, as expressed in my Twitter messages, is in the image above. She evaluated my personality and broke it down into 41 different traits, out of a total of 52 that she measures. The evaluation was based on “psycholinguistics,” or analysis of my word choice.

“Computers can derive people’s traits from linguistic footprints,” Zhou said in an interview with VentureBeat. “That hasn’t been widely applicable before, because where do you get those linguistic footprints? Now, you can do that with social media and digital communications. Those are readily available, so we saw an opportunity there.”

Above: Michelle Zhou of IBM Research.

Image Credit: Dean Takahashi

Zhou, a senior manager for user systems and experience research, has honed her analysis using both Twitter’s and IBM’s own internal social network, IBM Connection. She has compared the deeper information available on the IBM network and found that it matches closely to the conclusions drawn about personality from analyzing tweets by the same people.

There are plenty of caveats and questions that arise from doing such research. Privacy concerns will be at the top of that list, but Zhou says she is making use of publicly available information about people and that there are plenty of useful applications of the technology that don’t involve invading anyone’s privacy. For instance, you can use the technology to evaluate the personality of a company representative in negotiations.

Today’s marketers gather terabytes of data on potential customers and spend a lot of money trying to segment people into different target markets so they can tailor their commercial messages to specific groups. But they can get that targeting wrong because they are trying to understand customers by studying demographics (age, sex, marital status, home, income) and current buying habits. Indeed, companies spent $170 billion last year on delivering unsolicited electronic junk mail. The effectiveness rate is about 0.1 percent.

“Three or four years ago, we started a series in many people-centric operations, from marketing to customer service to product development to HR,” Zhou said in an interview with VentureBeat. ” Understanding an individual has to go beyond behavior analysis. Amazon.com is a quintessential example of behavior analysis. You read something or buy something.”

Research shows that a person’s behavior and decision-making process is heavily influenced by their intrinsic traits, Zhou said.

“Those intrinsic traits include what motivates you, what you believe, your fundamental needs,” Zhou said. “Thinking about it, it’s very hard to imagine – in a traditional way – how you could learn someone’s intrinsic traits, aside from standard psychometric tests. You can’t ask a customer something like, ‘Welcome to my store. Would you like to take a personality test?’ let alone scale out to literally hundreds of millions of customers in the real world.”

IBM can go deeper than other social analytics tools, which often provide sentiment analysis or brand analysis. Using its own analysis, IBM can generate a deep psychological profile from a lot of noisy social media. And that should enable marketers to tailor their marketing approaches to be more accurate.

Zhou’s analysis aims to extract intrinsic traits, such as personality, values, and needs. But it gathers that information by making correlations between such trait’s and a person’s word choice and activity patterns.

IBM Research will bring together the expertise in text analytics, human-computer interaction, psychology, and large-scale data processing to enable new insights. The research could be useful in retail, government, media, and banking. In a hospital, it would be useful for the staff to know your personality and how you are likely to respond to a certain level of service or treatment.

The data can tell a marketer a lot. For instance, Zhou knows that idealistic people often opt to buy organic foods or use organic skin care products. People who are happy or depressed use different words.

Zhou said she is confident because the evaluation is backed up with IBM’s computing expertise. One of the things she can do is correlate the choice of words in tweets for people who have taken personality tests. She can then apply the word choice correlation to people who have not taken the personality tests. And so she can decipher someone’s personality by their word choices.

Above: Maslow’s hierarchy.

Image Credit: Wikipedia

“We also take advantage of big data, as you implied earlier. If we were sitting here five or six years ago, I wouldn’t be very confident, because I could only analyze a few people,” she said. “I’d have no way of validating any of that. Now you have literally hundreds of millions of people out there. We keep improving our system. The dictionary of personality for Big Five was a very small one. We extended that greatly, because we’ve seen so many samples come in.”

She added, “It’s a little like Google Suggest. It’s the power of the cloud. Google has done a great job of creating a spellchecker. You can spell Britney Spears’ name in hundreds of ways and still get hits for Britney Spears. [Laughs] This is the same thing. You analyze so many people that you can beef up your technology to make it more and more accurate and cover more and more people. Now that we’ve hopefully covered western culture, we need to move on to eastern culture as well. That’s another adventure.”

The project is still in the research stage, and IBM is working with some customers to validate its interdisciplinary research in the field. Zhou said that her experiments show she needs only about 200 tweets, or 2,500 to 3,000 words, to create a personality evaluation that is accurate to within 10 percent. She did the analysis on me in about 20 seconds. So far, IBM has gone through 500,000 different people and found that no two people are exactly alike.

In my own personality evaluation, Zhou found I am agreeable, have a high level of self-expression, and favor conservation (or tradition). I am very dutiful and conscientious, and I’m an introvert. I’m not exactly sure that that means, but at least I don’t need to visit a therapist to find that out anymore. But I noted that my tweets are an expression of a public persona, as a writer at VentureBeat/GamesBeat. I don’t necessarily share my private thoughts on Twitter. She acknowledged it would be an interesting experiment to figure out the difference between my public self and my private self.

As for privacy concerns, Zhou said, “We have to be very open about that. Let’s use health care as an example. If I’m a customer of this company, when I log in to my account, I can see what doctor’s visits I’ve gone to, the vitals from my last checkup, and so on and so forth. What we can tell people using that kind of service is, ‘You can use this tool to learn more about yourself and about how other people like you are staying healthy.’ Someone like me, I’m very busy, but I also like going out and being social. What kind of wellness program works for me? A health care provider can let me know that if I share some of this data – say, a Yelp profile and a Twitter feed.”

She added, “My take on this is that for regular people like us, we can’t afford our own exclusive personal physician. We can’t afford a personal financial analyst or personal trainer. But if other people like us have had a good experience taking a certain path, we can learn more and benefit from that.”

For fun, Zhou also analyzed the personality of the VentureBeat Twitter account.

Above: VB Twitter personality

Image Credit: IBM