As big data and the science that unleashes its potential become more prominent, companies are finding creative ways to make use of these great methods and technologies, including leadership and management science. Leaders can use data science in three ways: as a core strategy (Cloudera), as a supporting strategy (Progressive's Snapshot Program), and/or to build the organization's key capabilities (Netflix).
When it comes to key capabilities, every company needs good leadership and management, and data science can help significantly in both areas. But given the opportunity, should you give leadership and management the equal amount of data science attention? Probably not. When employing data scientists to bolster your organizational effectiveness, you should favor leadership capability over management capability.
Data science applied to organizational leadership and management
There are compelling reasons to use data science for both management and leadership. To be clear, management deals with complexity, whereas leadership deals with change. As Dr. Stephen Covey put it, "Management is efficiency in climbing the ladder of success; leadership determines whether the ladder is leaning against the right wall." So when data science is applied to management, the organization's ability to execute on its vision is enhanced.
In the 1980s, companies like Motorola and GE spearheaded the infusion of statistical process control (SPC) into management capabilities, instigating the Total Quality Movement and the widespread adoption of Six Sigma. In a similar way, modern data science can significantly increase the efficiency of an organization's critical processes by successfully employing advanced technologies (Hadoop) and scientific techniques (machine learning). Leadership, on the other hand, deals more with making quality decisions about where the company should be going.
When data science is applied to leadership, the organization's ability to make effective decisions is enhanced. Quality decisions involve high-quality information, creative alternatives, and logical processing of these alternatives to make the best decision — data science can help with all three. Some decisions are easy (e.g., pulling out a map and figuring out whether to go east or west), but leaders face very difficult decisions, such as should you expand into China, India, Brazil, Mexico, or Turkey or just leverage your dominance in the US? Data science can help you uncover creative alternatives with Exploratory Data Analysis (EDA) and hidden relationships between variables with regression and machine learning. These are powerful tools to help you make decisions at the caliber you wouldn't otherwise obtain.
Although both leadership and management are vital for an effective organization, leadership tips the scales. A key problem with Six Sigma in the '80s and '90s (and today) was that companies were vehemently deploying against the wrong challenges; companies thought Six Sigma was a silver bullet that could solve all their problems, including leadership problems. In truth, Six Sigma is best suited for management problems, where the vision and strategy are clear, and the value comes from making an already functional business process more efficient. Fortunately, with modern data science, you aren't so confined to one or the other, so focus the majority of your efforts on improving leadership capabilities.
Research shows that quality decisions add more value to a company than quality execution. Independent Project Analysis (IPA), an industry leader in the quantitative analysis of project management systems, discovered that the initial phases of framing a project, exploring alternatives, and determining the best alternative (what they call Front End Loading) adds the most value in the lifecycle of a capital project. This is my experience as well. Most projects that fail were doomed to fail from the start. That's not to say you can get by with poor management; without upfront leadership, even the best management is powerless.
Leadership and management are key capabilities that all organizations need, and your data science team can greatly improve both; however, it would be unwise to distribute data science resources equally among the two.
Leadership trumps management in most organizations, so focus your data science resources on helping you make better decisions. When faced with strategic decisions, use qualitative research methods to uncover creative alternatives and then use quantitative research methods to select the best option or hybrid.
John Weathington is President and CEO of Excellent Management Systems, Inc., a management consultancy that helps executives turn chaotic information into profitable wisdom.