Timo Elliot [Blog | @timoelliott] wrote yesterday that BI is Dead and Julie Koesmarno [Blog | @mssqlgirl] was kind enough to tweet about it. The compelling point in Mr. Elliott’s post is:
“The most charitable view is that Gartner feels it has to exaggerate the demise of BI in order to get customers to pay attention to the changes before it’s too late.”
Mr. Elliott goes on to point out Gartner is redefining the term “BI” to apply only to what most BI folks now refer to as “self-service BI.” But Gartner also admits:
“’IT-modeled data structures… promote governance and reusability across the organization’ and ‘In many organizations, extending IT-modeled structures in an agile manner and combining them with any of the sources listed above is a core requirement’”
If you’re not confused you’re reading it wrong. I can hear you asking, “Andy, what do you think?” I’m glad you asked!
What I Think
I won’t pretend to understand the subtleties of Gartner’s logic at the time of this writing. I promise to dig more and try to get my head around what they’re saying and writing. I doubt, however, that Gartner is being wishy-washy. I think they’re trying to tell us something about a trend in Business Analytics.
What’s the Trend?
<trend>Self-service BI is gaining traction.</trend>
I’ve been in the Business Intelligence field since before I knew what it was called. When I did learn what it was called, we called it Decision Support Systems (DSS) and (in manufacturing) Manufacturing Execution Systems (MES). Later, I learned that the part of I really enjoyed was called Data Acquisition, a vital part of Supervisory Control and Data Acquisition (SCADA). Which led me to a career in Data Integration.
I’ve heard variations on this theme of “data integration is dead” for two decades. It’s not true. I believe it may one day become true, but today’s not the day.
“Why Isn’t the Report of BI’s Demise Accurate, Andy?”
Consider Grant Fritchey’s [Blog | @grfritchey] recent foray into R. Grant is, in my humble opinion, one of the smartest and most capable data minds on the planet. He’s an engineer who gets the value of analysis. Don’t take my word for it, ask anyone who’s read his books or heard Grant present. I promise I am not picking on Grant, I am using his post to make a point. And that point is this:
“Data analytics is often hard.” – Andy, circa 2016
Self-service BI is in better shape today than at any time in the past. The plethora of available tools are awesome and empower users like never before.
Raw data is rarely in a format that lends itself to consumption by self-service BI tools.
I believe this is the lesson of Grant’s post.
It’s Not Just Big Data, Either
There’s a lot of buzz surrounding Big Data, and the buzz is right and proper most of the time. But most of my customers do not have data volumes that reach the threshold of Big Data. As I wrote a few years back, Little Data Remains Important and most of my customers have Complex (or Difficult, or Hard) Data – not Big Data.
Just Add Garbage…
Data Quality is important because garbage-in = garbage-out. But what you may not realize is data collected into any store for analysis is going to be aggregated. Small variations in data quality, when summed or multiplied, bend predictive analytics lines in less-than-accurate directions. So much so that – sometimes – the ratio of signal-to-noise (good quality data to bad quality data) can be north of 99% when analytics solutions become so inaccurate that they’re useless for predictive analytics. Note: this isn’t the DQ threshold for all solutions but it is for some. All data-based solutions become useless once data quality dips below some threshold, and I’d hazard a guestimate that most solutions cross that line with a good:bad ratio in the 90-something percentile.
Combined, complexity and data quality limit the effectiveness of self-service BI. It’s not the fault of the individual or the individual self-service BI tool. It’s the data. As I stated, data analytics is often hard.
In my opinion, Gartner is right to raise the flag about the shift towards self-service BI because this shift will impact the workloads for many IT shops. For now, there’s still plenty of data collection, master data management, integration, and cleansing work to be had; much of it in support of self-service BI tools.
IESSIS1: Immersion Event on Learning SQL Server Integration Services
IESSIS2: Immersion Event on Advanced SQL Server Integration Services
Stairway to Integration Services
Stairway to Biml
Enterprise Data & Analytics – We are here to help.