Problems with ‘big data’ and tips on how to start gaining value from it

Big data, the buzzword du jour, has captured our attention with some truly impressive examples of what the future may hold. From medicine and potholes, to restaurant menus and the Higgs Boson, big data is everywhere.

And now for a reality check: most businesses have a lot of work to do before they can embark on a big data initiative.

It is easy to get hung up on the definition of big data, which typically involves the “3 Vs” of volume, velocity and variety, or even the more expansive “7 Vs.” For the vast majority of the 23 million small businesses in the U.S., however, big data can be thought of as a proxy for any data challenge exceeding their storage, compute or management capacity for accurate, timely and cost effective decision-making.

While the threshold for what constitutes big data continues to evolve, businesses of all sizes want to unlock additional value from the data that is most relevant to them, be it on a large or small scale.

A recent survey of 1,000 business and technology executives conducted by the Computing Technology Industry Association (CompTIA) confirms this point. Three in four respondents indicate data is one of their company’s greatest assets. At the same time, an even greater percentage (78 percent) agrees with the statement “if we could harness all of our data, we would be a much stronger business.”

Editor’s note: Our upcoming DataBeat/Data Science Summit, Dec. 4-Dec. 5 in Redwood City, will focus on the most compelling opportunities for businesses in the area of big data analytics and data science. Register today!

Executives participating in the CompTIA study said shortcomings in managing and using data impact their business in several ways, ranked as follows:

1. Wasted time that could be spent in other areas of the business.
2. Internal confusion over priorities.
3. Inefficient or slow decision-making and a lack of agility.
4. Inability to effectively assess staff performance.
5. Lost sales and reduced margins due to operational inefficiencies.

The mission critical nature of these issues helps explain the growing interest in big data. For those looking to take the next step, it may be tempting to start shopping for technology. Granted, this may be the most fun approach, but inevitably, it will be necessary to take a step back and think about a number of fundamental questions.

First and foremost: do you truly understand your company’s data profile? Hint: many companies do not. Consider:

• According to CompTIA research, eight in 10 businesses report high or moderate degrees of data silos — collections of data that have grown across the organization or within specific departments which are not connected in a cohesive plan.
• Few business (31 percent) report being able to provide a comprehensive single customer view (SCV), meaning potentially valuable customer data points are not fully integrated.
• Two-thirds of businesses report definitely or probably experiencing some degree of shadow (or rogue) data repositories, whereby staff may be maintaining their own contact or prospect lists, for example. The wide range of cloud-based options makes this easier than ever.

To get a handle on these and related issues, a good first step entails a data audit. As the name suggests, this is the process of mapping, describing, and documenting the data within the company. This doesn’t have to be an overly complex exercise. A simple template that captures key characteristics associated with a data stream is typically a sufficient starting point. This may include things such as the type of data, its frequency, how it is connected or not connected to other data streams, how it is stored and maintained, its value, and so on.

Those going through this process are often surprised at just how much data activity occurs within their company and how much of it was not fully accounted for.

Big data technologies and techniques show much promise in helping businesses become smarter in how they manage and use data. While the benefits of these strategies always take longer to fully materialize than expected, those that lay the proper ground work will be best positioned for a positive outcome.

Tim Herbert is the vice president for research and market intelligence for CompTIA (www.comptia.org), a non-profit association for the information technology industry.