Leading a data science team isn't as easy as it seems. The hard truth is that most experiments into data science fail, and the number one reason why by far is inappropriate leadership.
Notice I didn't say poor leadership (not that poor leadership doesn't exist, because it absolutely does); many of the leaders I've encountered are not poor leaders — they're just leading inappropriately for the circumstances.
Contingency theory postulates that leadership style should adjust based on the situation, and I agree. Victor Vroom and Phillip Yetton developed one of my favorite outgrowths from the contingency theory movement in the 1970s, and then Vroom extended his ideas with Arthur Jago in the 1980s. Their model suggests five decision-making styles with varying levels of group involvement. Let's explore the Vroom-Yetton-Jago model as it applies to leading a modern-day data science team.
1: Autocratic leadership
Autocratic leadership is when you make a decision by yourself in a vacuum with no involvement from anyone. Notwithstanding the overwhelming amount of advice out there to eschew this style of decision making, there's a time and a place for all leadership styles, including this one.
The key benefit of autocratic leadership is speed. If you are very decisive — which most leaders with autocratic muscles are — then a decision comes quickly, allowing your organization to be extremely agile. The downside is that your data scientists will be insulted much more than a typical group. To smooth things over, take time to explain to them why a quick decision was necessary, and why it would have been dangerous to take any more time to make it.
2: Modified autocratic leadership
Modified autocratic leadership is when you gather information from your data scientists to help you make your decision. You don't share the problem or decision with them, but you ask very pointed questions to fill in blanks where you're blinded or unsure. You must have sufficient information before making any decision, so if you need a quick decision but you don't have all the information, you'll need to reach out to your team.
3: Consultative leadership
Consultative leadership involves the data scientists in the problem or decision on an individual basis. In this case, you're seeking alternatives instead of information.
Visit each data scientist and explain the decision or problem at hand, and then give the person time to process the information, and come back with an alternative. The advantage here is that your data scientists are probably better at processing information than you are — that's what they're experts at. However, you have a different, but still estimable, change management problem if you don't use their alternative. Plus, this style will take an order of magnitude longer to make a decision than any sort of autocratic style.
4: Modified consultative leadership
Consultative leadership can be modified to be a group activity. Instead of visiting each data scientist individually, bring the whole team together as a group and involve them in the problem or decision.
One thing to note: You still make the decision, regardless of what decision the group arrives at. It's important to set these expectations upfront and clearly; otherwise, your team will assume your decision will favor the group's consensus, and you'll have a huge change management issue group buy-in. It may take time, but you control the clock, because the decision ultimately lies with you.
5: Group-based leadership
At the opposite extreme of autocratic leadership lies group-based leadership, wherein the team makes the final decision. The facilitation is very similar to modified consultative leadership, the only difference being how the final decision is made.
Although this style engenders the greatest amount of buy-in, it can be a bit dangerous. There's a reason why you're the leader, so if you delegate all of your decision-making power to the group, they might question your contribution to the process. I've seen this style of leadership overused, especially here in Silicon Valley where no decision is made until everybody is happy. Use this style only when you have a lot of time and need a strong commitment from the team.
Leading a data science team can be a challenging endeavor, but you can increase your odds of success if you keep a contingency model in mind.
There's nothing heretical about an autocratic decision if speed and agility are key value drivers. If you have a delicate decision that requires group ownership and you have a good amount of time before the decision needs to be made, employ a group-based leadership style. And there's a few shades between depending how much time you have, how much buy-in you need, and how good your relationship is with your team.
Think about the next decision you need to make, and which leadership style makes the most sense. There's no sense in using an inappropriate leadership style when you have so many options.
- Leadership trumps management when allocating data science resources
- Focus on success, not perfection: Look at this data science algorithm for inspiration
- 10 things you shouldn't expect big data to do
- 7 guerrilla tactics for retaining data scientists
John Weathington is President and CEO of Excellent Management Systems, Inc., a management consultancy that helps executives turn chaotic information into profitable wisdom.