This article is adapted from Rich Schiesser's book IT Systems Management (Prentice Hall, 2002)
The IT scourges are many: Unreasonable users constantly changing their requirements, suppliers charging outrageous rates to fix faulty software or hardware, late night calls, and weekend upgrades, just to name a few. But there is one scourge that is far more insidious than these and is something many IT professionals avoid like the plague: The dreaded process documentation. In the frequently heard words of a typical IT analyst, "I’ve just spent months planning, testing, and now implementing this process. You mean you want me to document it as well?"
The reluctance to document is understandable. Highly knowledgeable IT personnel usually excel more in technical skills than in those of writing. Some view writing as a less important, periphery part of their job. Many struggle with how to determine the true quality and value of the documentation they are generating. The following describes how IT analysts can effectively make these determinations, and in the process, transform a potential scourge into a possible blessing.
Evaluating process documentation
Many shops develop excellent processes but fail to document them adequately. After an initially successful implementation of the process, many of these procedures become unused due to lack of documentation, particularly as new staff members who are unfamiliar with the process attempt to use it.
Some documentation is usually better than none at all, but adding value and quality to it increases the likelihood of the proper use of the process it describes. Evaluating the quality of documentation can easily become a very subjective activity. Few techniques exist to objectively quantify the quality and value of process documentation. That's why the following methodology is so unique and beneficial. We developed this approach over several years while working with many clients who were struggling with ways to determine both the quality and the value of their process documentation.
The purpose of evaluating the quality of content is to show to what degree the material is suitable for use. The purpose of evaluating its value is to show how important the documentation is to the support of the process and how important the process is to the support of the business. The quality of the content of documentation is evaluated with ten common characteristics of usability. Each of these is described below.
- Ownership—Rates the degree to which the three key ownership roles—process owner, documentation custodian, and technical writer—are clearly identified, understood, and supported. For some processes, the same individual may have all three roles. In most cases the documentation custodian maintains the process documentation and reports to the process owner.
- Readability—Rates the clarity and simplicity of the written documentation. Items evaluated include the use of common words, terms, and phrases; correct spelling; proper use of grammar; and minimal use of acronyms, along with explanations of those that are used but not widely known. This characteristic especially looks at how well the level of the material matches the skill and experience level of the audience.
- Accuracy—Rates the technical accuracy of the material.
- Thoroughness—Rates how well the documentation has succeeded in including all relevant information.
- Format—Rates the overall organization of the material; how easy it is to follow; how well it keeps a consistent level of technical depth; to what degree it is documenting and describing an actual process rather than merely duplicating tables, spreadsheets, and metrics.
- Accessibility—Rates the ease or difficulty of accessibility.
- Currency—Rates to what degree the current version of the documentation is up-to-date and the frequency with which it is kept current.
- Ease of Update—Rates the relative ease or difficulty with which the documentation can be updated, including revision dates and distribution of new versions.
- Effectiveness—Rates the overall usability of the documentation including the use of appropriate examples, graphics, color-coding, use on multiple platforms, and compliance with existing standards if available.
- Accountability—Rates to what degree the documentation is being read, understood, and effectively used; all appropriate users are identified and held accountable for proper use of the documentation.
These ten characteristics are intended to cover a broad spectrum of quality attributes to make them applicable to a wide variety of different types of documentation. The value of the documentation to the environment for which it was intended is evaluated next using five common characteristics. Each of these is described below.
- Criticality of the process—This characteristic describes how critical the process is to the successful business of the company.
- Frequency of use—Describes how frequently the documentation is used or referenced.
- Number of users—Describes the approximate number of personnel who will likely want or need to use this documentation.
- Variety of users—Describes the variety of different functional areas or skill levels of personnel who will likely use this documentation.
- Impact of non-use—Describes the level of adverse impact that is likely to occur if the documentation is not used properly.
Each characteristic for both quality and value can be rated on a 0 to 3 scale based on the degree to which elements of each characteristic were met. The ratings are defined as follows:
3—All aspects of the characteristic have been met or are present.
2—A significant, though not entire, portion of the characteristic has been met or is present.
1—A small portion of the characteristic has been met.
0—None or an insignificant amount of the characteristic has been met.
Using this rating system against the ten attributes of quality and five characteristics of value enables virtually any type of documentation within your infrastructure to be evaluated. The maximum quality rating for any piece evaluated is 30 and the maximum value rating is 15.
Benefits of the methodology to evaluate process documentation
There are three major benefits to this method of documentation evaluation. The first is that it gives a snapshot of the quality of existing documentation at that point in time, particularly documentation of high value. If improvements are made to the material that result in new ratings, then they can be compared to the current rating.
The second benefit is that this method gives us the ability to customize the criteria for quality and value of documentation to reflect changes in priority, strategy, or direction. In this way the methodology remains applicable regardless of the specific criteria used. The third benefit of this method is that it allows comparisons between different types of processes within an infrastructure using the same standard of measure.
Once both the quality and value characteristics are evaluated, the two sets of attributes can be shown on a quality/value matrix as depicted below. Quality ratings are shown along the horizontal axis numerically increasing to the right from 0 to 30. Value ratings are shown along the vertical axis numerically increasing as it ascends from 0 to 15. Each axis is scaled from the lowest quality and value ratings up to the maximum possible. The benefit of such a matrix is that it depicts both the value and quality of each piece of documentation on a single chart.
The matrix is then divided into four quadrants. Points in the upper-right quadrant (1) represent documentation that is both high in value and high in quality. This is the desired place to be and constitutes excellent documentation that requires little or no improvements and only periodic reviews to ensure continued high quality. Points in the lower-right quadrant (2) signify material that is high in quality but of a lower value to a particular infrastructure. Documentation in this area is generally rated as good but could be improved.
The lower-left quadrant (3) represents documentation that is relatively low in both value and quality. Material in this area is designated as only fair and needs to be improved in quality. Since the value is low, improvements are suggested on a time-permitting basis. Points in the upper-left quadrant (4) indicate documentation that is high in value but low in quality. Documentation in this area is considered to be at the greatest risk since it represents material that is of particular importance to this organization but is of poor quality. Documentation in this quarter of the matrix should be improved as soon as possible to prevent adverse impact to processes, procedures, and services.
Several of our clients have evaluated dozens of key pieces of their infrastructure documentation and then graphed the results on a matrix of this kind. The matrix became a powerful tool since it identified instantly which pieces of documentation—those falling in the fourth quadrant—needed to be addressed most urgently. Managers and analysts both benefit from this identification. Rather than issuing a blanket statement that large amounts of documentation need to be generated, managers can direct analysts to initially concentrate on those pieces that have the lowest quality and highest value to the company. This method of evaluating the quality and value of documentation can make the task of improving it more appealing to IT analysts and less of a scourge.
The Harris Kern Enterprise Computing Institute (www.harriskern.com) is a consortium of publications – books, reference guides, tools, and articles - developed through a unique conglomerate of leading industry experts.