Data Centers

Don't confuse data with intelligence

Facts, figures, and complex calculations based on data are important, but they are not intelligence.

The amazing recent events in the Middle East have brought home an important distinction between data and intelligence that is particularly relevant for IT and organizations as a whole. For the United States in particular, the Middle East has been an area of intense focus over the past decade. Since the end of the cold war, the United States has ramped up its data gathering, diplomatic efforts, and outreach toward the Middle East, investing millions of hours and billions of dollars in the region. Yet the response to the recent popular uprisings sweeping the region seems one of mild shock at best and borderline ineptitude at worst. With all this investment in data gathering, how could we have not seen this coming?

We often confuse data with intelligence. Facts, figures, and complex calculations based on them are important, but they are not intelligence. In effect, data provide information about the past, while intelligence provides information about the future. Intelligence often relies on these data, but having all the data in the world does not give you any intelligence advantage if they are not used correctly.

While I don't know what led to what seems like a dearth of intelligence about what was happening in many of these Middle Eastern countries, I have seen several cases of businesses that have confused data with intelligence. Many executives and leaders are encouraged and rewarded for being "data driven." This can be a very effective strategy when internal and external factors remain in a steady state; effectively you become better at playing a game with fixed rules. Where this falls apart is when the rules change.

We talk endlessly about data in IT, and many of us have spent portions of our career cleaning data or corralling reams of data into a data warehouse application, yet often the return on these investments is not what was planned. Many technology vendors recognized the symptoms of the problem of data versus intelligence, but rather than encouraging the difficult questions in fixing the problems, they played a semantic game and started calling their data warehouse packages "business warehouses" or even "business intelligence" packages.

Many of the tasks we associate with data lend themselves to a technology-oriented solution. Our systems and processes are geared toward moving massive amounts of data, and once we've consolidated and correlated them into a warehouse of some sort, we tend to pat ourselves on the back and consider our company more intelligent. However, as painful as deploying all this technology may be, it is only the first step toward a truly intelligent business, much as painstakingly gathering books about a particular topic and placing them on your bookshelf does not actually impart any skills or knowledge to the collector until he actually reads them.

"But what about the thousands of reports and ad hoc queries and other wizardry our data warehouse offers?" one might ask. If these reports and queries are designed primarily to analyze past performance, then you're looking at yesterday's news. Once the data are marshaled, true intelligence results from having the right people looking at the data and having tools that allow them to do two things:

  1. Monitor a handful of key indicators that notify them that the "rules of the game" might be changing, and
  2. Provide tools that allow them to simulate different new rules and develop strategic and tactical plans to be successful under the new rules of the game.

While there has been an increasing emphasis on all manner of dashboard applications attempting to accomplish the first point, there is a tendency to focus on the wrong metrics, on too many of them, or on drilling down to such detail that one misses a fundamental change to the external environment.

Not painstakingly and methodically considering the two points above usually is due to the bane of many IT projects: the company effectively "outsourcing" what should be a business project to IT, merely because there are boxes and wires involved. While data gathering and consolidation may be well within the realm of IT, the true intelligence work must be driven by leadership. Until then, you're only left building an ever larger pile of yesterday's news, without any real intelligence.

About

Patrick Gray works for a global Fortune 500 consulting and IT services company and is the author of Breakthrough IT: Supercharging Organizational Value through Technology as well as the companion e-book The Breakthrough CIO's Companion. He has spent ...

21 comments
Duhble_Dee
Duhble_Dee

Service Management Group does this perfectly. SMG.com

kalmand6
kalmand6

Throught the years of the great wars from biblical times to present data has always been a part of intelligence and the gathering of data are seen as intelligence; whether present or for future. If we should look at data not been intelligence then there would be not intelligence in the world to help in continued civilization. Man has hunten on data gathered form trees, other animals and ground droppings. Present day has given us the privilige of advance technology, but the procedures are still similar in that we gather information (DATA) and use if for intelligence. While I might not be privy to the situations leading up to the middle east uprising, one cannot just label specific events as been responsible or playing a major role in unprecedented events. Yes I detest the wiki leak fiasco for endangering our fellow men in volatile areas but there are other factors that may not be as plain as wiki leaks to us but has been building a bottle neck for years to explode. We cannot just take simple events that has historical meaning and say it is a single of a few actions; because for every action there is a reaction and this may have been the final straw in a long historical suppression of the masses but the few. Critical thinking is needed when approaching situations such as this and any documentation for the public should show unbias analytic and constructive criticism.

techrepublic
techrepublic

The only positive comment that can be made about this waste-of-time article is that it did not kill as many trees as if it had been printed. Mr. Obvious (that's you if you are the author): If your point was to point out the obvious dysfunctional characteristics of BI you did a good job. Except you forgot to mention the most obvious and most devistating part of the BI professional's job which is the strong human instinct to kill the messenger when you do not like the message. But I am not going to elaborate on this awful part of BI because it will do little more than give some of you poor beleagured BIstards the feeling that you are not alone. What I am going to do is give some REALL WORLD examples of SUCCESSFULLY overcoming the obstacles of BI: 1. At the outset of the BI project/BI initiative obtain the names of all the domain experts who management would assign the entire project/initiative to but for their lack of technological expertise. 2, Obtain permission to assess their technical aptitude by having them take some online assessment exams and by submitting to being interviewed by you or a small panel of IT hiring managers who understand how to identify people who have the right aptitude for taking on various parts of the project. 3. Arrange for formal training of the person or persons that were identified as having applicable aptitude and whose participation management will agree to support. I have had very good results in my REAL WORLD experience of doing the above. Most domain experts who have technical aptitude are excited to become more technically skilled. It is a resume enhancement for them. And the investment for the company in both time and money is much, much less than when IT has to keep iterating over the same deliverables because of the lack of domain expertise on their part. There are other things that can be done to significantly mitigate the problems mentioned in thi Mr. Obvious article. But unlike the Mr. Obvious who wrote this article, I am not getting paid to impart my "intelligence" to you. So I'm outta here.

George Tirebiter
George Tirebiter

Data is simply information: numbers, isolated and interrelated facts, etc.; intelligence is derived and distilled from that data by careful analysis based on reliable and validated interpretation, triangulation with other variables analyzed by others, and the formative (in-progress) conclusions that should be evaluated and assessed at the time. One cannot derive conclusions based solely on hypothetical and speculative assessments of what certain data/information actually "means". Too often state and military intelligence analysis is politicized or skewed towards a certain postulated, and even desirable, outcome. Intelligence analysis should be utterly unbiased and scientifically pure so as to disallow the taint of preconceived notions, or worse, theoretical or ideological points-of-view. I know - "Good luck with that!"

TGGIII
TGGIII

Your explanation of not adapting to changing environment is spot on and is the subject of "Fooled by Randomness" and "The Black Swan" by Taleb. Excellent reads. I would like to insert an intermediate step in your data to intelligence line: Information. Data are the raw facts, information is the answer to a question. Example: What is your phone number? - the data is found in the phone directory - the information is your exact phone number. (IT enables bad business behavior all the time by the way by delivering phone books instead of phone numbers,) Intelligence allows me to postulate what to do with information. I develop a hypothesis, act on it, observe, learn and take the next step. Failure to follow these steps looking forward and at all factors leads to random acts of management. Attribution error (attributing cause to things that are independent but close in time and space) is rampant and forms the basis for much of the western management system of thinking. It is a rare company that can escape the trap.

MyopicOne
MyopicOne

A corollary on this is that when or if the warehouse/collection and analysis of it is published, it is not believed - not because it's wrong, but because it is right - and at direct variance with the statements from leadership.

Rowfus
Rowfus

We also shouldn't confuse intelligence with wisdom.

tshediso.letseleha
tshediso.letseleha

I agree , Many time intelligence is seen as the graphic representation of data. I see intelligence as foresight , not current or yesterday. In short Business and Intelligence = Past + Today = Value. To be able to derive unknown from known to make more of business intelligence.

mmsszz
mmsszz

I'd just add that the missing link between the data (and even when well formed into information) and intelligence is a human property called "vision". This is, again, Human: you can not just pushit right down into an application. Leaders who enjoy this particular talents are distinct. You may have some progress with that through some training and of course experiences, but talent remains on the lead.

gneal
gneal

I think it would be interesting to point out another contrast, data versus information. Let me illustrate an example: Data can be simply raw numbers. For instance, the temperature at 3 pm was 12??C, at 4 pm it was 10??C, 8 PM it was 9??C, and at midnight it was 15??C. Each of these pieces of the data alone doesn't tell you much. However when analyzed in aggregate, they tell you something quite interesting. If you were to graph this or simply examine the numbers you would notice that the temperature went up at night, something you would not normally expect. Analysis of the data produces the information showing a temperature rise at night. This is the information. In fact, it's quite interesting piece of information because it is counterintuitive. If you were going to do something outside you would have to conclude that something very unusual is happening in the atmosphere. The fact that the temperature was 15??C at midnight is not very interesting. It's a piece of data. The fact that it was 5??C warmer than it was at four clock in the afternoon is information. I actually had an experience like that once on a climb in Montana. Severe wet weather followed the next morning . This resulted in us cancelling our climb.

Mike_Borner
Mike_Borner

Great starting point to a discussion, as I think the topic of intelligence was left up to the reader's interpretation. I agree that there are still two "sides" which need to come together. The side which understands the data movement and integration; and the side which understands the ramifications of what the analysis provides and its effects on the external world.

richord
richord

Not sure that this is an argument on data versus intelligence or historical versus predictive practices or inductive versus deductive reasoning. The article appears to relate to political intelligence to business intelligence as if recent changes in the Middle East can be summarized by examining a set of data using an algorithm in the same manner that businesses process data. These contexts are so diverse that this association itself suffers from the same confusion suggested in the article; confusing news data with intelligence. Even the suggestion that an organization can be managed using ???intelligence??? derived from a handful of key indicators is overly simplistic. An organization is comprised of people. People don???t conform to algorithms in spite of their buying habits on Amazon, their viewing preferences on Netflix or their searches on Google. Economics assume ???rational??? behavior as a precept and look where that has taken our economy. Society and business are chaotic, random, biased, individualized and disruptive. Analyzing history, making predictions or applying inductive or deductive reasoning are techniques for interpreting data but they don???t necessarily result in ???intelligence. Data analysis is experimentation resulting in opinions and theories. Let???s take the word ???intelligence??? out of our vernacular. Business intelligence is an oxymoron!

Raksha.Pahlad
Raksha.Pahlad

Good article but don't agree with the use of the word "intelligence" in this context. Intelligence is a characteristic of biological life, though not all biological life, that provides us a with the capability to infer, reason, deduce, synthesise, interpret, contextualise, create (i.e."think"). ((Data --> Information --> Knowledge) + Experience)) + (the capability to think) = Wisdom

robert.cartledge
robert.cartledge

Excellent blog, and to the point, although I do not entirely agree with your Middle East analogy (another discussion for another time), I do agree that most business confuse data with intelligence. Formally a CIO within the legal environment, much was focused on matter centricity, buzzwords (among others) that absolutely churned my thought pattern. The concept behind mater centricity was to centralize all matters and have all applications integrated into a centralized database, however, as you so accurately pointed out it was only the first step toward a truly intelligent business. The problem was there was never a clearly defined second step, the books were gathered by particular topics, placed on a bookshelf, but never actually read. During my tenure as a CIO my main focus was technology driven applications and delivering those applications across the business units. The analysis of the data was left up to the financial experts, who never quite understood the value of warehousing and wrote it off as expenditure. The primary reason for their lack of understanding was terminology. What exactly does matter centricity mean? It means exactly that, phase 1 and nothing more, it was a ploy to invest monies into technology to delve into phase 2, which was never clearly defined. Perhaps the industry would have been better off naming it matter management so a phase 2 could have been incorporated into the project planning. So how do we resolve those misleading buzz words which tend to blur the lines between data and intelligence? It pains me to say that the CIO position must be reinvented and defined more as an entity existing between the CFO and CKO (whatever CKO means today). This new position would need to span business units and be able to mandate and govern. The position must align management to meet business objectives; it must incorporate knowledge of Finance and the business in order to project future value; but most of all the position must be agile, adaptive and innovative in its objectives. We need to stop using expressions like ???Think out of the Box??? (which usually means, I don???t like the way your thinking in your box so step into my box and think the way I???m thinking) and look at the business from a platform, taking all business units into account while striving to achieve a true business intelligent system.

pgerard
pgerard

Although the Middle East was just your segue into this subject, I think that what caught our "intelligence" organizations flat-footed was the part that documents released by WikiLeaks played in prompting what we are seeing. To generalize, it is always difficult to predict what disruptive elements will cause, no matter how well we process the data into intelligence.

putchavn
putchavn

They have some intrinsic value depending on the validity, precision, age, relevance and availability. Data also has derived value depending on how and for what purpose they are analysed and used. Any over rating would get corrected as data are processed and used.

putchavn
putchavn

These are the tems which are thought of as exclusive human capabilities applied to data / information. When they are defined and categorized it has become possible to apply mechanical / electronic manipulation on data / information and produce results matching what humans produced with their capabilities. Understanding of human capabilities and emulating them is an ongoing process. Progressively more and more of them may come within the capabilities of machines / computers---if it is profitable to do so. There may still be some special human talents which we may nurture and utilize in their natural form.

putchavn
putchavn

The definitions of Data and Information given here are popular but NOT VALID. What you call information is actually an interpretation or inference but NOT INFORMATION. See Donald E Knuth's Definitions which are well researched rigorous and valid under various conditions. or email to me for my research note on the subject. Best wishes, Putcha V. Narasimham putchavn@yahoo.com.

seanferd
seanferd

As this is a long-accepted use of the word "intelligence".

richord
richord

A few years ago IBM published an article that defined CIO as ???Career Is Over???. I could never understand why organizations required a CTO and CIO since they appeared to fulfill the same role. If the role was supposed to be focused on ???information??? most CIO???s missed the point. Do CIO???s concern themselves with all forms and mediums of information? Verbal, written, electronic etc. Are CIO???s skilled in ontologies, taxonomies and metadata management? Do CIO???s concern themselves with the quality of information? Do they develop policies regarding the quality control of reports, analysis and the output???s from the numerous systems and databases? Is the CIO aware of data standards such as ISO 11179 or practices such as Verification, Validation and Certification of reports and analysis? No wonder there is a lack of ???intelligence???. The CIO, like the CTO is playing with technology. The number crunchers don???t care about the technology so they are accused of not understating the technology. SOA, cloud computing and appliances, what do they care? A data warehouse is just a database that typically doesn???t contain the data that is needed, is organized in a manner that only a schizophrenic would appreciate and is lacking in well-constructed and managed metadata and contains numerous errors and omissions. Why don???t these number crunchers appreciate this? Maybe because the CIO is spending too much time with his/her IPad!

Editor's Picks