Why BI isn’t ready for the cloud

You’ve heard it again and again: The cloud is the new black. Only ever eclipsed by “big data” or “IoT” buzz, it remains a massive focus for all data-driven companies.

But while the cloud has worked very well for business applications with low data volumes and simple security requirements (CRM, for example), it hasn’t proven as successful for analytics use.

Enterprises are still chasing adoption

Over the last two decades, companies have collectively spent billions of dollars on business intelligence tools. I recently learned about a large financial services company that owns 92 analytics tools across different departments and geographies. Notice I said owns instead of uses. The challenge for them isn’t accessing BI technology — it’s that their employees won’t adopt it. Despite the hefty software spend every year and three times that amount being spent on BI consulting, the adoption rate of BI tools has remained flat in the low 20s over the last decade.

Deployment efficiency leans heavily toward on-premise

For enterprise IT, the challenge of adoption far outweighs the operational efficiency of how the technology is deployed. Most established IT organizations already have heterogeneous environments with both cloud and on-premise technologies. They have the skills to do both. They care most about deploying the software with the highest likelihood of being used. Once that’s been decided, only then can they evaluate some of the finer points of the deployment delivery options. And when it comes to cloud BI, there are still a number of challenges that make it an unattractive option for Enterprise IT.

Here are the top 4 reasons why BI isn’t ready for the cloud:

1. Data gravity is still inside the firewall

Every enterprise today has some applications that run in the cloud, be it a CRM application, an HR suite, or some other application. In terms of sheer volume, however, these applications are a tiny fraction of the enterprise’s data — somewhere around 1 percent. For analysis across enterprise data sources, it’s easier to pull down this small fraction of data from the cloud and bring it on-premise than to pull all the on-premise data into the cloud. I recently met with a large mobile provider whose brand managers are trying to analyze tens of millions of marketing communications per month and correlate them with phone usage and point-of-sale data. They are capturing terabytes of data every month in their data warehouse. Moving that data to the cloud was a non-starter for them.

2. WAN bandwidth is limited and growing slowly

WAN bandwidth continues to be bought and sold in megabits, while data volumes are growing from terabytes to petabytes. Moving even a fraction of this data to the cloud is impractical for most enterprises. For example, Amazon offers a “sneaker net” service for moving data into and out of the cloud. At today’s WAN transfer rates, it is incredibly hard to provide any kind of interactive analysis experience to end users if the bulk of the data is sitting on premise.

An analytics leader at one of the world’s top five retailers told me how he has been tasked with helping hundreds of merchants analyze sales trends and dig into which SKUs are selling fastest each day. With billions of rows of data to analyze and so many transactions coming in each minute, he places huge importance on the speed of access for end users. For his company, only an on-premise solution can keep end-user latency low enough to meet the demand for timely insights.

3. The security bogey still looms

Though enterprises have become more and more comfortable using cloud infrastructure, the majority of such use cases are still fringe workloads (testing and development, web analytics, etc.). A lot of organizations are still not comfortable moving the bulk of their enterprise data to the cloud. These concerns can be even more pronounced for industries such as financial services, retail, telecom — industries that put a lot of emphasis on data security for financial and personal information. I recently met with the head of lending services at a large financial firm whose team is tasked with analyzing the outstanding float of the company’s top retail banking partners. The firm’s bank account transaction data is incredibly sensitive, so security is critical. All of the data lives on-premise because the risk of moving it to the cloud is above the company’s security threshold.

4. The “cloud’ is not one place

CRM, HR, and ERP cloud apps do not typically run from the same physical location. Assuming that aggregating these sources in the cloud will magically make everything work is a major oversimplification. Someone still needs to connect all these data sources together, transfer data, and go through the traditional BI deployment process that continues to be the biggest challenge for enterprise analytics teams. The average enterprise has over 500 applications. Even if the majority of these applications at a company are in the cloud, pulling the data from each app into a BI solution requires the same amount of work whether they are in the cloud or not.

SMBs are feeling the pain, too

Small and medium-sized businesses with much smaller data volumes don’t face all of these same hurdles, and we’ve seen several vendors introduce affordable hosted cloud BI services. With Cloud BI, a regional coffee chain can swipe a credit card on a vendor’s website and then enable its store managers to monitor sales per day/week/month.

In theory, this is a great thing. In practice, though, it only works if the following are true: The software is connected to the point of sales system for its data feed; all the data sources are integrated; the data is loaded; a data expert builds a model; a business analyst gathers requirements for the reports and dashboards; and finally, the analyst builds these requirements.

So while the cloud eliminates some of the infrastructure complexity, a significant amount of technical implementation work remains. And that’s before all the ongoing services to make ad-hoc changes to those reports as new requirements arise. Labor intensity aside, the cost of such services can quickly outgrow the original license fees paid for the software. Many “successful” SMBs are now trying to determine if it was really worth it.

Ready, set … wait

Yes, I know it’s hard to get beyond the cloud buildup. It’s changed the way enterprises interact with technology and find scalable solutions. It’s changed the expectations we have around interacting with our personal data. But it’s not the best solution for all company size segments and technology use cases. Some segments (SMB) and use cases (business applications) are better suited for the Cloud than other segments (mid-to-large enterprises) and use cases (analytics infrastructure). As network bandwidth issues and security concerns ease out over the next decade, we will see a bigger hybrid cloud world emerge in which businesses will shift more aggressively to the Cloud for analytics. But for now, on-premise BI remains the dominant trend in mid-to-large enterprises.

Ajeet Singh is CEO of ThoughtSpot.