Input on improvisation methodologies

The principles described in the article 'improvisation methodologies
' sound like software engineering at a time, when there were no standards and methodologies.

Today this may work well at end users sites, where the teams and software applications are not too big.
It won't work for development of standard software, which needs to have clear and well documented interfaces, which are used for customization, interface or integration development.
It also won't work for big systems like ERP systems, where different teams are working on different components or modules.
In other words I see problems in 'improvisation methodologies', the more you have dependencies.

I recall a time 30 some-odd years ago, when this style of programming was the norm. The programmer was king (or queen), the only criteria was did the software work, and documenation (read maintainaibility) was the last item completed (if ever). What resulted was high maintenance costs, even higher cost to respond to business change and the development of "structured" approaches to programming.

Now, I would agree that some coding environments have gone overboard with the implementation of standards and procedures. But, I would argue that this the result of disasters (financial / organizational) that occured when the "programmer" who knew the particular system left for sunnier climes.

There are reasons for using a "tiger-team" approach to software development. But, in engineering, this usually means the use of small, dedicated teams to accomplish a specific delliverable, part of a larger project, and the results are managed, all routine criteria are delivered - only the team itself exists outside the main project.

Of course, it would put a level of "job-security" back in the equation - when one or two programmers are the only source of knowledge for a business critical application.

I am a project manager in an organization that has been using the improvisational project method coupled with a variant of extreme programming for every project for the last two years. While this has allowed us to overcome some significant obstacles very quickly, it has also led to huge delays in most of our projects. We are currently implementing a solution that calls for two major cultural changes which should improve results without taking away the most beneficial aspects of the XP development method. The first change is to require much more careful documentation from our developers. One of our big challenges at the moment is to replace our legacy real-time transaction engine. Since it is entirely undocumented, we pretty much have to start from scratch. I would like to avoid inflicting this challenge on anyone else (or even myself) in the organization's future. The second cultural change is to front-load requirements gathering and needs analysis. All too often, the improvisational/XP approach leads to a "skunkworks" team leaping into a project without a clear understanding of the goals and requirements. While from time to time this is unavoidable, in most cases the practice is inexcusable and leads to misplaced expectations, delays, and disappointment. By clearly laying out the requirements and goals and then turning the extreme programmers loose, we hope to gain all the advantages of a rapid-fire development team and put away the delays and miscommunications that have previously dominated our projects.

My problem with Extreme Programming is manifold:
(1) the two bright programmers may quit, and then there is no one that understands well the program (s); no one likes to do documentation post facto (i.e., after the project is completed; the EP may become so popular in an organization that system life cycle methodologies may be discarded in part of fully.

Thanks

I do not agree with the XP methodology. It leaves the door open to exclude/miss key data elements that would have been identified if the proper analysis (Which can be abbreviated) were taken.
Also, it assumes programmers work well together, co-coding and sharing ideas. I'm sorry, but in my experience, that is a rarity.

My work place has implemented the provisions of the SEI to "strive" to get our CMM up level 5. Our customers require it -- what else would they use to insure quality?

I can see now that it pays more telling people how to write software than to actually write the stuff.

We have process comming out of our ears to the point that it takes so many hours just to spin up a project that simple jobs have no hope of getting through the ordeal.

On the other hand a lot of this is good for requirements traceability just to make sure the right job gets done, is measured and leads to information to improve our process for future projects.

Process will never replace talent, and unfortunately, I think that fact is overlooked by the people buying into the SEI-CMM ratings.

"XP" may work in some instances and "Process" may work in others. My personal opinion is when software design starts using standard engineering practice (like UML and/or patterns) and has mature, integrated, tools that go from requirement specs to test documents - then we'll finally start to crack this nut.

Right now I'd have to say the best method is somewhere between these two "extremes"...