Leadership optimize

Agile grows up and new challenges emerge

The conversation shift from "why should we adopt agile?" to "how can we adapt to agile across the enterprise?" presents a new set of cultural and tactical challenges.

At the RallyON conference in Boulder, CO this month (which I attended at the invitation of Rally Software), I noticed a distinct shift in emphasis from similar conferences I've attended over the years. The previous conversations about pure methodology, such as the measurement of team velocity or the best ways to apply agile techniques to distributed teams, have been supplanted by an emphasis on two key topics:

  • Mastering the cultural evolution to agile, and
  • Applying agile concepts at the strategic or portfolio level.

From my perspective, this swing from the tactical to the strategic is a sign of maturity. The basic agile battle, which focused on comparisons of agile to traditional CMM and PMI-inspired methods, is over and won -- agile works. The case studies of organizations that have benefitted from agile's ideas are many and persuasive. The value it has delivered is undeniable. No longer do agile proponents need to prove that iterative, value-focused, collaborative, and self-directed methods promote innovation and creativity in product development. The conversation has shifted from "why should we adopt agile?" to "how can we adapt to agile across the enterprise?".

A new set of challenges

This change in emphasis presents the agile community, and the enterprises we serve, with a new set of challenges that cut to the core of the values, beliefs, and cultures of our stakeholders. In order to contribute to the evolution of our clients or stakeholders, we must elevate our sights from the tactical to the strategic, and from the methodological to the cultural. In fact, one of the key insights from my conversations with agile enterprises and their advisors is that organizations that focus solely on the methodological, and neglect the cultural and strategic, are the most likely to fail at agile transition.

Tactical tunnel vision is a common stumbling block. By focusing on the methods rather than the culture of agile, we set the expectations of everyone, from the executive suite to the PMO to the business stakeholders, that all we have to do to become agile is migrate from Work Breakdown Structures (WBSs) to backlogs and from Gantt charts to burndown charts, and we're on our way to The Agile Enterprise. I've observed organizations that implement a handful of these techniques without changing the expectations of executives for predictive estimates, or the requirement for business sponsors to commit to collaborative practices. I've even seen organizations try to introduce these techniques by "stealth," counseling their clients to keep calling their methods by the old names (calling a backlog a task list or project plan, for example) in order to "avoid confusing the organization."

This tactical paradigm breaks down quickly, for obvious reasons. No matter how agile the development team is, if the executive and business sponsors continue applying a top-down, long-term, predictive strategic planning cycle, the agile team will either get caught in unrealistic, externally imposed deadlines and budgets, or get stuck waiting for funding and approval to move forward as the executive committee engages in its three-year strategy cycle. If business sponsors have not internalized the need to take ownership of the projects that are critical for their business, and continue to engage in "thrown-over-the-wall" requirements development, agile development teams get stuck making decisions that can only be made by those with subject matter expertise and accountability for the results. Agile teams can whip through multiple iterations, adding value with high velocity, only to find themselves waiting for the sponsor to grant them an audience for a grudging "project walkthrough." In short, unless the entire method of planning, funding, and scheduling the enterprise's portfolio of projects evolves, the twain between traditional strategic planning and agile practice will never meet. This inevitably leads to the death knell known as "we tried agile and it failed."

In regard to culture shift, one idea that was explored in numerous sessions at RallyON was the "4 C's" concept of organizational focus, originally formulated by William Schneider in 1994 in his book The Reengineering Alternative. Underlying this concept is the central truth that changing culture is a long-term, painstaking, and often frustrating exercise, which is not likely to occur on the accelerated schedule necessary for agile transitions. According to 4 C exponents, it's better to understand the current culture and work within it to migrate to agile without trying to push the cultural boulder uphill. I won't try to condense this complex idea here - refer to the hyperlinks above for more depth on the topic. Suffice to say that recognizing which of the 4 C's - control, competence, collaboration, or cultivation - is dominant in the enterprise that's evolving to agility, and using that recognition to help the enterprise make the right decisions, is more likely to succeed than trying to force change through the organization against its cultural predisposition.

The popularity of roadmaps

The word I heard most during my attendance at the RallyON conference was "roadmap." Agile thinking requires us to recognize that we cannot plan further than we can see, and that predictions and estimates become hazier the further into the future we try to project. This recognition makes the traditional concepts of carved-in-stone estimates, schedules, and even corporate strategies untenable. If we can't know what the marketplace will look like in three years, much less how our teams will choose to respond to the new realities, what's the point in planting stakes in the ground?

The idea of a roadmap, a plan that acknowledges that strategies will shift while enabling the development of consensus on a foundational vision and path, has been zealously adopted by the agile community. Roadmaps can define the next few releases of a product in significant detail, and sketch out the expected feature set of future releases without the perceived certainty or the constraints to change and creativity of a signed-in-blood, task-by-task WBS. The leading vendors of agile development support tools (VersionOne, CollabNet, and Rally) have embraced the concepts of agile portfolio management and roadmap development and offered them in their product suites.

More about agile adoption

I spent the week in Boulder interviewing agile consultants, vendors, developers, and executives, and I'll be presenting and analyzing their comments over my next several TechRepublic columns. I learned a lot about the real-world challenges and successes of agile adoption, and hope to bring those lessons to TechRepublic readers.

While the basic questions of agile feasibility have been answered, the hard work of implementation and evolution remain. Through the heavy lifting required to guide enterprises to these new ideas, we're overcoming challenges and learning what really works, not just tactically, but culturally as well.

About

Rick Freedman is the author of three books on IT consulting, including "The IT Consultant." Rick is an independent consultant and trainer, working, through his company Consulting Strategies Inc., to help agile teams and organizations understand agile...

4 comments
StevenDDeacon
StevenDDeacon

Many Techno-Babblers are unaware that the phenomenon of "Agile Software Development" was a paradigm began as "Chief Programmer Teams" in the 1960's and was a complete methodology by the late 1970's. Harlan D. Mills, was the author of "Chief programmer teams, principles, and procedures", IBM Federal Systems Division Report FSC71-5108 (Gaithersburg, Md.) which I believe was published around 1971. As an IBM research fellow, Professor Mills adapted existing ideas from engineering and computer science to software development. These included the structured programming theory of Edsger Dijkstra and Robert W. Floyd (both awarded the Turing Award), as well as others. His Cleanroom software development process emphasized top-down design and formal specification. Frederick Phillips Brooks, Jr. was a software engineer and computer scientist, best known for managing the development of IBM's System/360 family of computers and the OS/360 software support package, then later writing candidly about the process in his landmark book "The Mythical Man-Month". He wrote the paper "No Silver Bullet: Essence and Accidents of Software Engineering" in 1987. F.P. Brooks has received many awards, including the National Medal of Technology in 1985 and the Turing Award in 1999. Larry LeRoy Constantine, spent several years studying the works of IBM Fellow Harlan Mills: Edsger Dijkstra; and Robert W (Bob) Floyd. Professor Constantine and began publishing many noteworthy papers gaining him an appointment as an associate professor at Carnegie Mellon University by the time he was 27. He became a full professor at Stanford University six years later. He obtained this position without a Ph.D. L. Constantine joined IBM's System Research Institute (SRI) in 1968. During those years Larry Constantine conducted a study of the most prolific software engineers and their methodologies for engineering software. He left SRI in 1972 having began development of a manuscript for "Fundamentals of Program Design: A Structured Approach". In 1974, after he resumed work on his manuscript, American Edward Nash Yourdon, (a software engineer computer consultant, author and lecturer; and pioneer in software engineering methodology), reviewed Constantine's manuscript; urging him to complete it. With the combined effort of Larry Constantine and Edward Yourdon, work on the manuscript continued. As part of structured design, Larry Constantine developed the concepts of cohesion (the degree to which the internal contents of a module are related) and coupling (the degree to which a module depends upon other modules). These two concepts have been influential in the development of software engineering, and stand alone from "Structured-Modular Design" as significant contributions in their own right. They have proved foundational in areas ranging from software design to software metrics, and have become a part of the vernacular of the discipline. After Constantine received the Turing Award in 1978, "for having a clear influence on methodologies for the creation of efficient and reliable software, the definitive work "Structured Design: Fundamentals of a Discipline of Computer Program and Systems Design" by Edward Yourdon/Larry L. Constantine, copyright 1979, was published by Prentice-Hall, Yourdon Press.

casey
casey

Rick - While I am certainly not an opponent to Agile techniques being used in other enterprise areas beyond IT, there is a fundamental issue with trying to go "all in" with Agile. You sum it up beautifully with the statement: "Agile thinking requires us to recognize that we cannot plan further than we can see ..." On it's face, this basic assumption is false. If it were true the world would be a much different place. Skyscrapers, hydro-electric dams, canals, space travel, bridges, etc. all rely on the premise that you can plan for what you can't see. This is especially true when we talk about investment in such ventures. The fact that Agile proponents have latched on to the idea of the "roadmap" supports the need for Agile to expand its thinking. But since the short-term view and iteration model are necessary to achieve what Agile promises, it seems unlikely that expansion would happen in the first place. Which is why Agile will remain a powerful methodology, but not the only tool in the shed. Cris Casey casey@christophercasey.com