Most organizations
don’t have a formal apprenticeship program for software developers. There is no journeyman for software development.
However, most software developers today learn as much through their interaction
with other developers as they learned during their formal education.

So if your software
developers are learning how to be software developers through their experience
at your organization, what are you teaching them? It’s time to evaluate what
the software developers in your organization are learning.

The need for education

As an industry we
still suffer from project failures, cost overruns, poor client satisfaction,
and a general malaise. Failure rates for software development projects are
wildly different depending upon who you listen to. Publicly reported rates vary
from 5% at the lowest to over 70%. While neither of these numbers is
particularly believable they illuminate the fact that there is still a problem.
If software development was steadfastly producing quality software all of the
numbers would be in the same range.

However, the mere
fact that there are failures doesn’t indicate what the causes for failure are.
Surely there are a variety of causes for project failures. Just as surely some
of those causes are causes that are well understood by those at the forefront
of software development.

Understanding the
impact of poor requirements gathering, poor project management, poor change
control, improper tool selection, etc., are all examples of classic problems
with software development which we, as an industry, know leads to project
failure. However, despite this we continue to make the same mistakes over and
over again.

The reason for this
is expressed as the gap between “state of the art” and “state of
the industry.” “State of the art” is simply the ideal world
where someone would know everything about software development that can be
known. It would be an ideal understanding, without bias, for various
methodologies, tools, and techniques. “State of the industry” on the
other hand is the average knowledge of the practitioner working in software
development. While there is no one average person, you can develop a persona to
identify the traditional software developer.

In today’s world the
typical (average) software developer has a good chance of using an integrated
development environment (IDE) and source version control. Both of these things
are important because they form the basis of effective code management.
However, the typical software developer doesn’t understand more than two
software development methodologies. The software developer is unable to
identify where one software development methodology might be more useful than
another.

Similarly, the
typical software developer doesn’t have exposure to or experience with
implementing patterns. Patterns are essentially template solutions that
describe a generic approach to a common problem. Although most software
developers have heard of patterns (most frequently the factory pattern) they
don’t have any idea what they mean or why they’re important.

Testing techniques
are generally haphazard at best. Some developers perform unit tests, other
developers test via the “throw-it-over-the-wall” method which relies
upon other developers, quality assurance professionals, and end users to
identify their bugs.

The typical software
developer continues to allow change to happen to the code without change
control either because of user requests or because of their own desire to do
something more interesting. The lack of change control creates ripple effects
which combine with other changes to become the tsunami which overturns the
proverbial project boat. They do this not from a desire to intentionally sink
the project but instead because they do not fully comprehend the impact of
these small changes.

How we learn

Before you set off to
buy a bookshelf full of the latest books in software development and a
collection of the classics, you may want to pause. Reaching a level where every
software developer reaches a relative level of “state of the art”
through reading is unrealistic. Most people can’t fully understand what they’re
reading without practical examples that they can apply quickly after reading
it.

Think back to every
article you’ve read. Have you been able to implement the suggestions or
guidelines from every article? Of course not! You can’t absorb everything you
read. Our brains are not wired that way. While we each have different levels of
being able to understand and integrate the material we read,
no one is perfect in this respect.

Humans learn best by
experiential learning. That is, we learn best by experiencing the situation
where we need the learning. Reading something is fine but the more involved we
get with the idea the better our retention of it.

When I defined
“state of the art” above I indicated it was a theoretical state. It’s
something that no one individual could ever meet. That is because of the
experience based learning that we do as humans. It’s impractical to expect that
a person has had experiences that would allow them to understand the best
practices on every aspect of software development. For instance, the best
practices for Web development, where multiple iterations and visual appearance
are important, are completely different than best practices for embedded
software where quality and reliability are important because updating embedded
software is difficult.

However, all is not
lost. We do have the ability to move everyone closer to “state of the
art” by encouraging focused reading and development in specialized areas
and then allowing this information to be spread to the rest of the team through
experience.

That’s the positive
side of teaching through experience, allowing individual team members to focus
and leverage that learning throughout the entire team. It’s something that
moves your team closer to “state of the art”. However, most
organizations are unintentionally moving developers away from the “state
of the art” by developing and reinforcing bad habits.

Double Bad

In most organizations
there are the basic processes. There’s some sort of a requirements gathering
process. There’s typically some sort of a design phase. Some folks,
particularly those who’ve used universal process (UP), Rational Unified Process
(RUP), or some agile methodologies may know these by other names such as
elicitation and investigation. Although they are typically much shorter in
duration, they perform the same fundamental purpose – to understand the problem
enough and to create a structure for a solution.

Most organizations
also have some form of documentation, no matter how minimal. It might be a
“project charter”-like email from the executive sponsor, a request
database, or some other form of information repository but it typically exists.

The challenge is,
however, that many organizations skip these steps in the process, bypass
documentation, and proceed headlong into development. Everyone instinctively
knows that this is a bad idea. There’s a mountain of evidence that the
“code and fix” methodology (if it can be called that) doesn’t work.
However, when pressed for time we eliminate the time reserved for elicitation
and investigation.

What does this teach
software developers on the project? It teaches them that these practices are
not essential (In Fred
Brook‘s words, accidental) to the development of software. Nothing is
further from the truth; however, it is the message that is conveyed. It’s clear
that these steps and the associated documentation can be skipped when speed
dictates.

Ignoring for the
moment the probable consequences of this course of action, this has a second
order effect. Developers begin to think that some of the work that they’re
doing (elicitation and investigation) are really the organization just making
them do busy work. Instead of being seen as a technique for making software
development projects successful, they are now a burden levied upon the
developer.

With a four year old
in my home, I’m painfully aware of how few times I have to do (or say)
something for it to be repeated. It reinforces my resolve that it is never
acceptable to eliminate or scrimp on elicitation, investigation, or
documentation. I know what skipping or undercutting these activities even a few
times will mean in terms of the long term retraining that will have to be done.
(If you have a child, think about what happens when they’ve been a grandma and
grandpa’s house for a few days.)

That is not to say
that elicitation must be a document heavy three month process – it does,
however, mean that the goal of understanding all of the requirements for the
current iteration or phase as good as is practical must be met. It means that
you shouldn’t relent if there’s a critical portion of the solution, which is
both essential to the operation and needed for the current iteration, which
remains nebulous after the time for elicitation has been exceeded.

Lagging

Most software
developers are lagging behind the “state of the art” due in part to
the lack of time for training, the need for more experience, but most painfully
because we are teaching software developers that parts of the process (no
matter what methodology you subscribe to) are accidental – and are therefore
not essential to the success of the project. Protecting the minimalist
implementation of the process is essential to continuing to support the
professional development of software developers.