How to find — and hire — data scientists

Data might just be the new driver of business success.

Airbnb uses data to match lodging and travelers. Netflix uses data to personalize your entertainment. Amazon uses data to sell you more stuff. And Google uses data to drive better search, advertising, email, and social.

But data doesn’t reveal its secrets easily — or cheaply. Hence the exploding role of data scientist, which has grown from practically nothing to a major hiring focus, with stunning growth over the last three years at job site Indeed.com of thousands of percent. And the cost of hiring a data scientist, which is up to $150,000/year.

Above: Massive growth in demand for data scientists, according to Indeed.com

Image Credit: Max Song

So how do you find and hire these newly essential employees?

Not easily, according to data scientists Max Song and Carl Shan, who are working on a Data Science Handbook along with two other data scientists, William Chen and Henry Wang.

Together, they’ve published 120 Data Science Interview Questions to help organizations hire.

“The first reason why it’s hard to hire data scientists is simply that the supply of potential data scientists is lagging behind the demand,” Song told VentureBeat. “A McKinsey report identified this problem and estimated that there would be a shortage of 150k – 190k people with data science talent by the year 2018.”

Roughly 140,000 people are employed as data scientists right now, but by 2018, we’ll need 440,000 to 490,000, according to that report. The shortfall is due to the fact that this job is so new there’s not a direct educational equivalent, or an obvious career path. There’s no data science major in college, with the closest equivalent being perhaps a rare statistics major. But data scientists also need a firm grounding in computer science.

That makes hiring difficult, to say the least.

However, Song told me, Pete Skomoroch, the former principal data scientist at LinkedIn, has estimated that there are up to 250,000 people in the workforce today with the talent and ability to be data scientists. Which means that employers and recruiters need to broaden their search to people who may not have the title of data scientist, and may have followed diverse educational and career paths.

Max Song and Carl Shan have published 120 Data Science Interview Questions on VB Insight

“The skills of a data scientist are very broad, so often being a quick and eager learner is one of the best skills one can have as a data scientist,” Shan told me. “Candidates who learn quickly and are really creative also ask the best questions about the data, and can extract useful and valid insights that no one else had thought of before.”

When you’ve found candidates, the challenge then becomes interviewing someone with a skillset that likely doesn’t exist on your team, and testing their abilities even though you may not really understand data science.

“Common skills that a data scientist is usually evaluated on include data analysis, programming, statistics, machine learning, probability, and effective communication,” Song says.

That can often be accomplished by asking the candidate to walk through a large dataset, explaining what they’re doing, or ask the data scientist how he or she would approach a problem you’re currently facing, or giving him or her some structured data and seeing what lessons the data scientist candidate can derive from it.

Ultimately, however, you’re looking for someone with much more than simply technical skill.

“Data scientist candidates also need to be able to establish trust and rapport with whoever they will be interfacing with at the company,” Shan says. “The data scientists must be able to convince the product / business teams as to the validity and impact of the insights, and have an open enough relationship with them to enable as much communication and feedback as possible.”