Agile methodologies have set their place in the software development world and by extension the data science world, where advanced mathematics is combined with software development to obtain analytic solutions. Data scientists are often confronted with the grim reality of stakeholder management on agile projects.
There's a lot for stakeholders to like about agile projects, but they're usually not ready for the downsides. And, most stakeholder anxiety comes from mismanagement. Here are eight tips for running an agile project.
1: Secure sponsorship
Don't start your agile project without securing sponsorship from stakeholders' leadership and management.
There are many aspects of an agile approach that summon violent reactions from stakeholders — including lack of planning, lightweight documentation, and increased time commitments — and their managers fuel much of their behavior. IT managers are more tolerant of agile nuances because they're more familiar with them; you can't say the same for non-IT managers. So when your stakeholders report out to their boss, it's not going to make sense unless they're on board as well.
2: Spend adequate time on setup
On an agile project, you must go slowly to go fast. It's somewhat of a paradox, but it's absolutely true. If you don't prepare people for what to expect, you'll have a lot of problems with stakeholders during execution. When planning the execution of an agile project, reserve enough time for awareness and education.
When I worked with Sun Microsystems, before I started the team on execution, I spent a few weeks with the stakeholders on how agile looks and what to expect. No only did this give the developers time to set up a test harness and software change configuration management (SCCM), but it gave me enough time to prime the stakeholders for a successful agile execution.
3: Protect the developers' bill of rights
The rights of all data scientists and other developers must be ferociously guarded. This idea extends from the Extreme Programming days, when the developers' bill of rights was documented and sacrosanct. Among these rights are: the right to have open and immediate communication with the stakeholder, the right to update and own their duration estimates, and the right to own their day-to-day schedule.
If you're not careful, these rights will be violated on a regular basis. Stakeholders either micromanage developers or go absent for extended periods due to priorities.
4: Protect the customers' bill of rights
Your stakeholders have rights too, and they should be protected just as vigilantly. Among these rights are: the right to choose which user stories take priority, the right to get the most value out of every development day, and the right to cancel the project at anytime and be left with a working system that's better than when they started.
I see these rights violated all the time. Developers sandbag duration estimates and talk the stakeholders into an architectural or foundational build that has no functional value. It's easy to get away with this, because that's what everyone's used to. Agile changes the narrative, so you must protect it.
5: Give them a job
The best way for stakeholders to know what's happening is to put them in the middle of development with a real assignment. Most of the time stakeholders build requirements, and data scientists build solutions; however, once the requirements are done, stakeholders tend to disappear because they don't have a job anymore. That's a problem.
Once stakeholders unplug, they lose contact with the day-to-day issues that crop up. As their expectations stray from reality, their frustration increases when false expectations aren't met. The best way to avoid this is to give them a real job that contributes to the solution — that keeps them consistently plugged into what's going on.
6: Make sure they attend every stand-up meeting
You should hold daily stand-up meetings, and stakeholder attendance is required. They may not see the value in these meetings, as they're not allowed to participate like the data scientists. And certainly, other priorities will pull them away. It doesn't matter. They have a responsibility to attend, and you cannot let them off the hook. It's another way to keep them plugged in so they don't build those false expectations we talked about earlier.
7: Prepare to defend unusual practices
You must always be prepared to defend unusual agile behaviors like pair programming, lightweight documentation, and no firm commitment on scope.
Unfortunately, regardless of how well you prepare stakeholders for agile, there are some behaviors that will not sit well. Pair programming is a perfect example. I could spend three months explaining to stakeholders what it is and why we do it; they'll nod their head and go along — right up to the point when the developers start practicing it. That's when everything goes haywire. I've been through this pattern too many times to count.
8: Foster a culture of honesty and trust
Above all, there must be a culture of trust. In the absence of trust, agile breaks down quickly. Lack of trust causes developers to sandbag hours, and stakeholders to negotiate with developers on feature completion date.
To install trust, everyone must be honest. Honesty is hard sometimes, but it's necessary in an agile environment. The data scientists may be done a lot earlier than projected — disclose that to the stakeholders so you can add more value in the iteration. The stakeholders may know that the project is going to be cancelled — don't string the development team along hoping to get the most work possible until the last minute.
Once trust is compromised, you have no hope for success on an agile project.
These tips should prevent agile execution from coming back to haunt you in the forms of micromanagement, negotiating, and blindsiding. Be sure you apply all eight, or you might flex your way out of a job.
John Weathington is President and CEO of Excellent Management Systems, Inc., a management consultancy that helps executives turn chaotic information into profitable wisdom.