This
article is adapted from Rich Schiesser’s book IT Systems
Management (Prentice Hall, 2002)

The IT
scourges are many: Unreasonable users constantly changing their requirements, suppliers
charging outrageous rates to fix faulty software or hardware, late night calls,
and weekend upgrades, just to name a few. But there is one scourge that is far
more insidious than these and is something many IT professionals avoid like the
plague: The dreaded process documentation. In the frequently heard words of a typical IT
analyst, “I’ve just spent months planning, testing, and now implementing this
process. You mean you want me to document it as well?”

The reluctance to document is understandable. Highly
knowledgeable IT personnel usually excel more in technical skills than in those
of writing. Some view writing as a less important, periphery part of their job.
Many struggle with how to determine the true quality and value of the
documentation they are generating. The following describes how IT analysts can
effectively make these determinations, and in the process, transform a
potential scourge into a possible blessing.

Evaluating process documentation

Many shops develop excellent processes but fail to document
them adequately. After an initially successful implementation of the process,
many of these procedures become unused due to lack of documentation,
particularly as new staff members who are unfamiliar with the process attempt
to use it.

Some documentation is usually better than none at all, but
adding value and quality to it increases the likelihood of the proper use of
the process it describes. Evaluating the quality of documentation can easily
become a very subjective activity. Few techniques exist to objectively quantify
the quality and value of process documentation. That’s why the following
methodology is so unique and beneficial. We developed this approach over
several years while working with many clients who were struggling with ways to
determine both the quality and the value of their process documentation.

The purpose of evaluating the quality of content is to show to what degree the material is
suitable for use. The purpose of evaluating its value is to show how important the documentation is to the support
of the process and how important the process is to the support of the business.
The quality of the content of documentation is evaluated with ten common
characteristics of usability. Each of these is described below.

  1. Ownership—Rates
    the degree to which the three key ownership roles—process owner, documentation
    custodian, and technical writer—are clearly identified, understood, and
    supported. For some processes, the same individual may have all three roles. In
    most cases the documentation custodian maintains the process documentation and
    reports to the process owner.
  2. Readability—Rates
    the clarity and simplicity of the written documentation. Items evaluated
    include the use of common words, terms, and phrases; correct spelling; proper
    use of grammar; and minimal use of acronyms, along with explanations of those
    that are used but not widely known. This characteristic especially looks at how
    well the level of the material matches the skill and experience level of the
    audience.
  3. Accuracy—Rates
    the technical accuracy of the material.
  4. Thoroughness—Rates
    how well the documentation has succeeded in including all relevant information.
  5. Format—Rates
    the overall organization of the material; how easy it is to follow; how well it
    keeps a consistent level of technical depth; to what degree it is documenting
    and describing an actual process rather than merely duplicating tables,
    spreadsheets, and metrics.
  6. Accessibility—Rates
    the ease or difficulty of accessibility.  
  7. Currency—Rates
    to what degree the current version of the documentation is up-to-date and the
    frequency with which it is kept current.
  8. Ease
    of Update—Rates the relative ease or difficulty with which the documentation
    can be updated, including revision dates and distribution of new versions.
  9. Effectiveness—Rates
    the overall usability of the documentation including the use of appropriate
    examples, graphics, color-coding, use on multiple platforms, and compliance
    with existing standards if available.
  10. Accountability—Rates
    to what degree the documentation is being read, understood, and effectively
    used; all appropriate users are identified and held accountable for proper use
    of the documentation.

These ten characteristics are intended to cover a broad
spectrum of quality attributes to make them applicable to a wide variety of
different types of documentation. The value of the documentation to the
environment for which it was intended is evaluated next using five common
characteristics. Each of these is described below.

  1. Criticality
    of the process—This characteristic describes how critical the process is to the
    successful business of the company.
  2. Frequency
    of use—Describes how frequently the documentation is used or referenced.
  3. Number
    of users—Describes the approximate number of personnel who will likely want or
    need to use this documentation.
  4. Variety
    of users—Describes the variety of different functional areas or skill levels of
    personnel who will likely use this documentation.
  5. Impact
    of non-use—Describes the level of adverse impact that is likely to occur if the
    documentation is not used properly.

Each characteristic for both quality and value can be rated
on a 0 to 3 scale based on the degree to which elements of each characteristic
were met. The ratings are defined as follows:

3—All aspects of the characteristic have been met or are
present.

2—A significant, though not entire, portion of the
characteristic has been met or is present.

1—A small portion of the characteristic has been met.

0—None or an insignificant amount of the characteristic has
been met.

Using this
rating system against the ten attributes of quality and five characteristics of
value enables virtually any type of documentation within your infrastructure to
be evaluated. The maximum quality rating for any piece evaluated is 30
and the maximum value rating is 15.

Benefits of the methodology to evaluate process documentation

There are three major
benefits to this method of documentation evaluation. The first is that it gives
a snapshot of the quality of existing documentation at that point in time,
particularly documentation of high value. If improvements are made to the
material that result in new ratings, then they can be compared to the current
rating.

The second benefit is
that this method gives us the ability to customize the criteria for quality and
value of documentation to reflect changes in priority, strategy, or direction. In
this way the methodology remains applicable regardless of the specific criteria
used. The third benefit of this method is that it allows comparisons between
different types of processes within an infrastructure using the same standard
of measure.

Once both
the quality and value characteristics are evaluated, the two sets of attributes
can be shown on a quality/value matrix as depicted below. Quality ratings are
shown along the horizontal axis numerically increasing to the right from 0 to
30. Value ratings are shown along the vertical axis numerically increasing as
it ascends from 0 to 15. Each axis is scaled from the lowest quality and value
ratings up to the maximum possible. The benefit of such a matrix is that it
depicts both the value and quality of each piece of documentation on a single
chart.

The matrix
is then divided into four quadrants. Points in the upper-right quadrant (1)
represent documentation that is both high in value and high in quality. This is
the desired place to be and constitutes excellent documentation that requires
little or no improvements and only periodic reviews to ensure continued high
quality. Points in the lower-right quadrant (2) signify material that is high
in quality but of a lower value to a particular infrastructure.  Documentation in this area is generally rated
as good but could be improved.

The
lower-left quadrant (3) represents documentation that is relatively low in both
value and quality. Material in this area is designated as only fair and needs
to be improved in quality. Since the value is low, improvements are suggested
on a time-permitting basis. Points in the upper-left quadrant (4) indicate
documentation that is high in value but low in quality. Documentation in this
area is considered to be at the greatest risk since it represents material that
is of particular importance to this organization but is of poor quality. Documentation
in this quarter of the matrix should be improved as soon as possible to prevent
adverse impact to processes, procedures, and services.

Figure A

Several of our clients have evaluated dozens of key pieces of their infrastructure
documentation and then graphed the results on a matrix of this kind. The matrix
became a powerful tool since it identified instantly which pieces of
documentation—those falling in the fourth quadrant—needed to be addressed most
urgently. Managers and analysts both benefit from this identification. Rather
than issuing a blanket statement that large amounts of documentation need to be
generated, managers can direct analysts to initially concentrate on those
pieces that have the lowest quality and highest value to the company. This
method of evaluating the quality and value of documentation can make the task
of improving it more appealing to IT analysts and less of a scourge.

The Harris Kern Enterprise Computing
Institute (www.harriskern.com) is a
consortium of publications – books, reference guides, tools, and articles –
developed through a unique conglomerate of leading industry experts.