Success is most likely when the data science team reports to IT

Regardless of how many big data initiatives companies have, most still find that they can only afford one data science team.

Big data has suddenly made data fashionable throughout business, and yesterday's mundane data reporting projects have jumped to the head of the pack overnight with new promises of competitive advantage in the market, and innovative ways to save money and streamline business processes.

Understandably, every functional area in the company wants its big data project, too - whether it is marketing trying to determine where customer sentiment is, finance trying to discover new ways to assess risk, customer service and fulfillment trying to streamline product delivery, or even manufacturing engineering trying to develop new methods to manufacture products.

Meet the team

This company-wide enthusiasm is forcing data administrators to build out many different big data "data marts" for functional areas throughout the company - but regardless of how many data marts and big data initiatives companies have, most still find that they can only afford "one" data science team.

A data science team requires three key areas of expertise:

  • A business analyst who can work well with end users and quickly grasp the salient business issues that big data must deliver to the business, as well as how to ask for this data;
  • A data scientist who is skilled in big data programming languages, and who also has statistical analysis skills that can be used in the development of big data queries that can bring meaningful insights; and
  • An IT person who is able to interact on technical matters with the DBA or data architect in IT, and who can also manage big data compute resources that are likely centralized in the data center, ensuring that these resources are fully optimized, and also that jobs are properly scheduled, completed, and delivered.

Where should this data science team report?

Faced with a hybrid function that blends business and IT skills, this is an organizational question that many companies are facing now.

Some companies opt to position the data science team where it reports directly into end business functions, while others choose to establish a direct reporting line to IT. Still other organizations use a primary reporting line to IT, with dotted line reporting relationships going out to the business units.

Is there a best method?

Methods never universally fit organizations, but there is empirical knowledge from IT and business relationships from the past that can be applied.

What we know

Dotted line reporting relationships, while they might look good on org charts, seldom work. In the end, the employee is going to work at pleasing the individual whom he directly reports to, because that individual controls salary increases and promotions.

Joint reporting relationships don't work well, either. Often, mixed messages and priorities confound employees, and this quickly becomes counter-productive.

IT has repeatedly demonstrated that it doesn't have great sense for the business, or what the business needs to know in order to compete. One reason is that IT personnel tend to be evaluated (and rewarded) to perform in highly technical skills areas. For most, there's very little time left to learn the business.

A majority of end business units have repeatedly demonstrated poor project management skills and a poor grasp of technology. Both are exactly what the data science team requires if it is to produce at a high level.

So what to do?

Big data is fashionable!
One school of thought is to have the data science team report into IT, which in most cases already has command of big data technology in the data center - and a staff that at least two-thirds of the data science team (the data scientist and the IT person) are heavily engaged with on a daily basis.

The data science team's business analyst would also report into IT, but this analyst would be tasked with coordinating with business function "point persons" throughout the company.

The IT department would be responsible for putting together a set of SLAs (service level agreements) with its various business unit "clients" that establish metrics for performance, and business clients would have ultimate "say" as to whether IT was getting the job done.

There are bound to be some organizations where this idea won't work - but in many cases, it will.

One inherent advantage of the concept is that it keeps everyone focused on results. The concept also puts into play client-oriented service metrics (like SLAs) that offer a more objective means of assessing the ultimate goals of the data science team - to deliver value to the end business through the effective use of IT.