Business Intelligence – also known as BI – is one of those trendy technologies that modern businesses have to have, even where they aren’t quite sure exactly what it is and why they need it. Business Intelligence is about bringing order out of chaos, or in this case, out of overwhelming amounts of data that are too voluminous and often too unstructured to be easily digested and processed by more conventional means.
Marketers of BI solutions freely throw around terms such as insights and visibility – but what does that really mean? Business Intelligence is often touted as an almost magical tool with which you can predict the future and that will make tough business decisions suddenly easy. But how much of that is true and how much is hype? As with all universal panaceas, it comes off to a skeptic like me as, at best, a mighty exaggeration. So I decided to take a look at the reality of BI, how it can actually benefit your organization and the pitfalls of which you need to be aware.
What is business intelligence, anyway?
Intelligence, in this context, refers to the gathering of information, especially information that isn’t readily available and can be used for the benefit of the individual or agency that collects it. The term is used in this fashion by the military and government (often shortened to intel) and has been enthusiastically adopted by the corporate world.
The goal of BI is to aid companies in making decisions that make their operations more efficient, enhancing revenues, and better competing within their markets by detecting patterns and trends and identifying potential trouble spots so problems can be addressed early.
A brief history of BI
The concept of “business intelligence” has been around for more than a century and a half, with first usage of the term generally credited to a book by Richard Millar Devens published in the 1860s. The process was, of course, a manual one at that time.
The idea of automated business intelligence systems was introduced in the late 1950s in an article for the IBM Journal of Research and Development by inventor named Peter Luhn, who is now considered the “father of BI.” He also coined the term “information on demand,” and was prescient enough to consider, even in those early times, the importance of security and data ownership. Howard Dresner, who later became an analyst for Gartner, was a driving force in the development of Business Intelligence as we know it today.
Decisions, decisions
BI uses such technologies as data mining, complex event processing, and predictive analytics (among others) with the goal of helping business decision-makers arrive at the most accurate conclusions and make the best choices.
There are different types of decisions that must be confronted in the business environment. At the top level, strategic and policy decisions set the foundation for success or failure (or something in between). At another level, operational decisions affect the implementation of those policies and impact the direction in which the business ultimately goes.
Making effective decisions is one of the most essential components in a business leader’s skill set. Some decisions are straightforward and the right path is obvious. Most are not; they involve a myriad of factors – many of them not apparent at surface level – and often conflicting priorities. The best decision makers are able to objectively and logical evaluate situations and take appropriate risks. Those evaluations and that risk assessment are based on facts, which are gleaned from information, which comes from data.
It’s all about the data
There’s a popular quote from Clifford Stoll, who helped catch KGB hacker Markus Hess in the 1980s: Data is not information, information is not knowledge, knowledge is not understanding, understanding is not wisdom. However, each of those levels is based on the one before, and to get to wisdom (or at least understanding), you first need data. Turning that data into usable information is the job of BI. Turning the information into true knowledge, understanding and wisdom is still arguably the province of the human brain (artificial intelligence notwithstanding).
That first step is no small feat. Businesses today generate a copious amount of data on a daily basis. Some is nicely structured in databases. Some is semi-structured or completely unstructured. Emails, memos, notes, reports, presentations, web pages, images, videos, chat sessions, phone calls, meeting minutes, news stories, PR and marketing materials, user group communications, and social media discussions. And then there’s the metadata – the data about the data.
It’s no wonder that we all feel as if we’re constantly suffering from TMI – too much information. In many cases, it’s not even information that we’re getting; it’s just piles and piles of raw data. According to some statistics, as much as 85 percent of data is unstructured. Different pieces of data – including some that’s crucial making the right decisions – are stored in a wide variety of formats that make them more difficult to access. Searching it can be difficult, and getting an overall view of the big picture that it paints can be impossible, without assistance from some sort of analytics solution. That’s where BI can be useful.
In order to be useful, though, that information has to be presentable. It doesn’t exist in a vacuum, so the last step is to organize and display it in a way that’s easy to digest. BI solutions include tools for data visualization: creation of charts, performance scorecards, infographics and such to present the metrics and other actionable information gleaned from the analytic process.
How BI works – and how it doesn’t
Let’s get down to the nitty-gritty of what’s involved in the Business Intelligence process. The first step is to define measurable strategic goals. Before BI can be used to further those goals, you have to get all that data into databases. These can be relational databases such as SQL for traditional structured data, or non-relational databases such as NoSQL that are appropriate for unstructured and polymorphic data (data that can have more than one form). Then all the data is brought together, and the databases linked.
The data is analyzed using predictive analysis tools, analytic dashboards, by performing multilayer queries, and other methods. The analyses can pinpoint historical trends that can be helpful in predicting future performance. However – and it’s a big “however” – it’s important to remember the disclaimer that’s emblazoned prominently on the forehead (or at least the sign on the wall) of every investment adviser: Past performance is not necessarily indicative of future results.
The problem with BI isn’t that it doesn’t do what it’s supposed to; it’s that too many people expect it to do much, much more. If predictive analysis software could provide the sort of crystal ball into the future that some demand of it, there would no longer be any risk involved in playing the stock market or betting on the horses, and we wouldn’t so often be shaking our fists at the weather channel. In fact, there are way too many variables for analytics, even using the biggest of big data, to infallibly forecast what will happen a year from now, a month from now, or even tomorrow.
It’s also important to keep in mind that oldest of computer caveats: GIGO – Garbage In, Garbage Out. Sometimes organizations get enamored by the sheer quantity of big data, but the quality of that data matters. Business Intelligence forecasts are only as good as the data on which they’re based. Finally, even with good, relevant data and realistic expectations, in order for BI to add value, organizations must implement it properly. The details of how to utilize BI solutions to best advantage are dependent to some measure on your organization’s specific needs and goals, and beyond the scope of this article, but there is an excellent white paper called How Business Intelligence Should Work that can get you started in the right direction.
Photo credit: Alpha T Knowledge
