Big Data

With Big Data, what's better--qualities or quantities?

When selecting the data scientists that will fuel for your Big Data innovation and/or strategy, you must make sure you understand how they feel about research.

Research is research, right? Wrong! In fact, you can get yourself into real trouble if you hire data scientists who are philosophically incongruous with how you plan to employ Big Data in your business strategy. This may sound like a statement laced with ether and hyperbole; however, not many executives pay attention to the research orientation of their prospective data scientists because they don't think there's much to consider.

Au contraire, when selecting the data scientists that will fuel for your Big Data innovation and/or strategy, you must make sure you understand how they feel about research.

The numbers have it all

The most classical method of research is called quantitative research. Data scientists who believe in quantitative research--referred to as positivists in management science circles--believe in numbers and statistics to tell the story. They start with a hypothesis and use deductive reasoning to prove their hypothesis.

In other words, they start with a belief then break it down with numerical analysis for validation. For instance, you might have a strong hunch that your product would do well with young men who enjoy sailing. If you frame your problem this way for a positivist, they will gladly take this hypothesis and attempt to prove what you're saying with analytics.

Quantitative researchers are terrific when you have a well-defined problem that needs to be solved and you have some theories as to what might be happening. Once the problem is defined and there's a hypothesis to explore, your data scientists will follow the well-known scientific method that I've talked about before, to put numbers behind your idea.

If things go right, your hunch becomes more than a hunch because you have good statistics to warrant your idea. This engenders a good level of confidence to commit down a strategic path.

Tell it like it is

On the opposite side of the philosophical spectrum are qualitative researchers--sometimes referred to as interpretivists. These researchers start with research questions--not preformed hypotheses--and use inductive reasoning to describe the whole from observed parts. Instead of conducting controlled experiments like their more analytical counterparts, they observe and interpret phenomena as it exists it its own environment.

You'll see this a lot in biological research, where the researcher sets up camp close to the species' habitat in an attempt to extract deeper insights; trying desperately not to disturb the incumbent ecosystem.

Qualitative data scientists are useful when you don't have any idea what's inside your data. For instance, you might have been collecting transaction logs from your operations for the last five years but you have no idea if there's anything valuable in there that you could convert to an information product. If you frame your problem this way for an interpretivist, they will gladly explore your data so you can understand it better.

They are not looking for anything in particular and they're certainly not trying to prove anything. They're simply trying to get a better understanding of what's inside your data.

I am very critical of qualitative analysis because it's often abused in the strategic setting. However, there may be a very important place for qualitative analysis in your Big Data strategy--you just need to understand how to use it properly without throwing money away.

Can we all get along?

There is a hybrid called mixed methods research that combines both qualitative and quantitative research into one effort. There may be a place for this in your Big Data strategy; however, it's not the panacea most people think it is when they first discover it.

Mixed methods research is both nascent and complex--two things you might not want to combine on a strategy already rife with risk. Furthermore, it's not so much its own method, as a collection of ways to combine the two types of research. For instance, you could start with qualitative research then triangulate with quantitative research.

Or, you could start with a quantitative research question and elaborate with qualitative findings. You could additionally run both methods in parallel and triangulate when both are complete. The combinations are mind-numbing: ergo the complexity.

The biggest challenge for executives however, has to do with organizational development. This usually comes as a surprise. The difference between qualitative and quantitative researchers is philosophical. Whether they realize it or not, there are deep-rooted cultural differences between the two camps.

Positivists believe that data has objective meaning, and the cause/effect relationships will universally apply to a domain of influence if you can accurately determine what they are. Interpretivists believe that data has subjective meaning that is not as generalizable. They're happy describing anything for you, but they'll have a problem with most innovation-related application of their discovery.

Putting the two camps together is like mixing Mentos and Diet Coke; you'll waste a lot of time in explosive debates if you don't control this organizational dynamic.


There are three types of research methods, two types of data scientists, and one best way to form the research foundation of your Big Data strategy. Quantitative researchers are great when you have a strong hunch and qualitative researchers are great when you don't have a clue; however, the two together can be explosive. Mixed methods research seems to be a logical compromise, but it can actually do more harm than good if you're trying to use it to solve the wrong research question.

That's why I suggest you get your business strategy together before you start thinking about data scientists and fancy software. Take some time today to reconsider the resource plan for your Big Data strategy. It's hard to undo a bad choice when philosophy matters.


John Weathington is President and CEO of Excellent Management Systems, Inc., a management consultancy that helps executives turn chaotic information into profitable wisdom.