If your company is currently experimenting with Big Data in an attempt to prove some sort of concept--interrupt this right away. If you're going to prove something--it better be value.
Many of the executives I talk to about a Big Data strategy haven't put much thought into executing the strategy because they're waiting for IT to finish their exploration. In fact, I know based on direct experience and through trusted colleagues, that most companies are developing a Big Data proof-of-concept before taking the plunge on a full-out strategy.
As reasonable as this approach sounds, the research on change leadership would suggest a different approach. Although a proof-of-concept seems logical, a proof-of-value is more pragmatic.
As I scan the most prominent literature on leading change from legendary authorities like Kurt Lewin and John P. Kotter, nowhere do I find the idea of a "proof of concept." In fact, the only place I come across this construct is in IT circles. That's because, IT people come from the engineering subculture, and in their culture you never release anything that's not perfect. The proof-of-concept is a way for them to reduce risk with experimentation without releasing a sub-par product.
However, the problem with concepts--even if they are proven--is their intrinsic dearth of business value. Conversely, what I call a proof-of-value--more commonly known as a small win--has an amazing amount of business value. Let's compare the two approaches as they relate to one of the biggest challenges in a Big Data effort: executive sponsorship and leadership.
What exactly are we proving?
Executive support is one of the biggest challenges today with any Big Data effort. Executives know there's something there because of all the inescapable hype; however, you cannot rally strong executive support until there's a strong business case.
When I was helping Visa with their enterprise data strategy, we would spend weeks assembling strong business cases because it was the only way to get the executives' attention--and more importantly their dollars. Big Data efforts struggle to get executive support because, from an executive's point of view, it looks like IT is playing around with no real business purpose.
Real executive leadership usually comes too late in the proof-of-concept model. The CIO typically finds some money to buy fancy technology and embarks on a data-mining effort with his team of highly-paid data scientists and consultants.
This means combing through the company's data looking for something--exactly what, nobody knows. The general managers of the business units patiently wait for IT to discover something of use. If IT actually manages to prove a concept, general managers perk up and the Big Data effort might take flight. This is when executive leadership and sponsorship starts--if at all.
This approach carries a lot of risk. Real money is being spent without any indication of whether these analyses will have any business value. If the CIO takes too much time (a timetable that's usually not shared with the CIO), top management's patience will precipitously vanish, and the whole effort will be shelved until the next wave of analytics hype rolls through.
A better model of leadership
To ensure real value from your Big Data strategy, you should lead it properly. The proof-of-value model is either driven by the general manager or the CEO, depending on the scope of the effort. A proper vision is established upfront that ties to a business value, and then Big Data folds in to support the effort.
Following the vision is a small win--and bigger victories build on the early momentum. Finally the Big Data strategy is solidified into the organization into what Kurt Lewin would call a refreezing. This is change leadership 101 that every executive should be familiar with.
Having the CEO or a general manager lead a Big Data strategy in this way completely obliterates the risk of insufficient executive sponsorship; thereby, neutralizing one of the biggest risks I see today on Big Data efforts. It also puts success in the hands of the leaders who are more likely to produce a return for your Big Data investment.
As I stated earlier--it's cultural. IT managers--even as high as the CIO--play it too safe with their proof-of-concept mentality. Proving value makes more business sense and it accelerates your collective understanding of whether there's merit in your hypotheses. You'll learn a lot more from falling down on the run for a small win than you ever will from a successful proof-of-concept.
Most importantly, this approach keeps executive sponsorship engaged for as long as it makes sense. Executives throw away a lot of money and opportunity on Big Data experiments because they lose faith. They lose faith because of inadequate engagement, which causes miscommunication around the opportunity.
This is the key failing with the proof-of-concept approach. In contrast, when business executives lead the effort, they monitor the opportunity closely. Win or lose, the results of a small win guides them down the path that makes the most sense for the company.
To succeed with your Big Data strategy, you'll need an executive mindset--not an engineering mindset. Engineers spend a lot of time getting things right, and there's nothing wrong with that. We need engineers to be precise; however, that mentality doesn't fit well with a business strategy where value matters.
If your company is currently experimenting with Big Data in an attempt to prove some sort of concept--interrupt this right away. You're wasting money. Instead, tell your general manager to build a tight business case and redirect the leadership of your Big Data team. If you're going to prove something--it better be value.