Big data has suddenly made data fashionable throughout
business, and yesterday’s mundane data reporting projects have jumped to the
head of the pack overnight with new promises of competitive advantage in the
market, and innovative ways to save money and streamline business processes.
Understandably, every functional area in the company wants its
big data project, too – whether it is marketing trying to determine where customer
sentiment is, finance trying to discover new ways to assess risk, customer
service and fulfillment trying to streamline product delivery, or even manufacturing
engineering trying to develop new methods to manufacture products.
Meet the team
This company-wide enthusiasm is forcing data administrators to
build out many different big data “data marts” for functional areas throughout
the company – but regardless of how many data marts and big data initiatives
companies have, most still find that they can only afford “one” data
A data science team requires three key areas of expertise:
- A business analyst who can
work well with end users and quickly grasp the salient business issues that
big data must deliver to the business, as well as how to ask for this
- A data scientist who is
skilled in big data programming languages, and who also has statistical analysis
skills that can be used in the development of big data queries that can
bring meaningful insights; and
- An IT person who is able
to interact on technical matters with the DBA or data architect in IT, and
who can also manage big data compute resources that are likely centralized
in the data center, ensuring that these resources are fully optimized, and
also that jobs are properly scheduled, completed, and delivered.
Where should this data science team report?
Faced with a hybrid function that blends business and IT
skills, this is an organizational question that many companies are facing now.
Some companies opt to position the data science team where
it reports directly into end business functions, while others choose to establish
a direct reporting line to IT. Still other organizations use a primary
reporting line to IT, with dotted line reporting relationships going out to the
Is there a best method?
Methods never universally fit organizations, but there is
empirical knowledge from IT and business relationships from the past that can
What we know
Dotted line reporting relationships, while they might look good
on org charts, seldom work. In the end, the employee is going to work at pleasing
the individual whom he directly reports to, because that individual controls
salary increases and promotions.
Joint reporting relationships don’t work well, either.
Often, mixed messages and priorities confound employees, and this quickly
IT has repeatedly demonstrated that it doesn’t have great sense
for the business, or what the business needs to know in order to compete. One reason
is that IT personnel tend to be evaluated (and rewarded) to perform in highly
technical skills areas. For most, there’s very little time left to learn the
A majority of end business units have repeatedly
demonstrated poor project management skills and a poor grasp of technology. Both
are exactly what the data science team requires if it is to produce at a high level.
So what to do?
One school of thought is to have the data science team
report into IT, which in most cases already has command of big data technology
in the data center – and a staff that at least two-thirds of the data science
team (the data scientist and the IT person) are heavily engaged with on a daily
The data science team’s business analyst would also report
into IT, but this analyst would be tasked with coordinating with business
function “point persons” throughout the company.
The IT department would be responsible for putting together
a set of SLAs (service level agreements) with its various business unit “clients”
that establish metrics for performance, and business clients would have ultimate
“say” as to whether IT was getting the job done.
There are bound to be some organizations where this idea won’t
work – but in many cases, it will.
One inherent advantage of the concept is that it keeps
everyone focused on results. The concept also puts into play client-oriented service
metrics (like SLAs) that offer a more objective means of assessing the ultimate
goals of the data science team – to deliver value to the end business through
the effective use of IT.