How your executives will screw up your next analytics project

Analytics projects routinely fail, but often it's not the technology at fault, a new Harvard Business School study revealed.

Image: iStockphoto/demaerre

Roughly two-thirds of analytics projects fail. Sometimes, that's a function of poor technology choices, but usually it comes down to the people involved. Specifically, the members of the C-suite, according to a new Harvard Business School study.

As the study's authors conclude, "leadership issues were often at the heart of the problems." The type of "leadership issues," however, are somewhat surprising.

People are the problem

Despite enterprises making ever bolder claims to being "data-driven," the reality is a bit different. As a variety of surveys have uncovered, people love long as it corroborates the actions they already prefer to take. Indeed, as a Fortune Knowledge Group 2014 survey found, 62% of business executives said they tend to trust their gut, and 61% said real-world insight tops hard analytics when making decisions.

Harvard Business School's more recent study, however, details a very different people problem.

SEE 6 myths about big data

The problem, this study notes, is one of leadership:

Since there was no natural owner of analytics within the traditional organizational structure, multiple executives competed hard to own the new capability. While not every C-suite member wanted to manage such a high-stakes opportunity, the most powerful members were eager to oversee an influential new pool of talent and command more time on the board's agenda.

Unfortunately, not everybody gets to be king or queen. In most organizations, this led to executives resisting the big-data projects they weren't allowed to lead, with negative consequences:

With the exception of the "winner," a feeling of vulnerability settled over the other executive team members when the analysis conducted by the analytics group revealed inefficiencies and missed opportunities in their respective functions. For these individuals, the triple whammy of ceding budget to the new initiative, having your turf "scanned" for innovation opportunities, and then being assessed on whether you have the skills to manage in a new data-driven environment was disruptive, to say the least.

The report authors thus conclude that the CEO must "manag[e] the power shifts in the C-suite that analytics can spark," helping non-participating executives deal with feelings of vulnerability and "think through the new skills they will need to use."

It's not just an executive problem

That's great, but it's also insufficient. One of the hard truths about our new fetish with data is that it never truly speaks for itself. Just ask Facebook, which tried to use algorithms to improve upon human editors in its news feed...with disastrous consequences.

SEE Why data science is just grade school math and writing

It turns out that machines can't think for themselves, and throwing more data at them doesn't make them any "smarter" at finding correlations or causalities. Ultimately, people program machines to ask certain questions or look for types of patterns. Machines, then, are never truly independent of people, and the data always reflects that.

So whether you have executives trying to sabotage data analytics projects, or programmers imbuing data with their own biases, big data is messy because the people behind it are messy. The sooner we acknowledge this, the better.

Also see