Developer

Remove the mummies before completing IT projects

Chip Camden admits to burying some so-called mummies in IT projects. He offers IT consultants tips on locating and removing mummies.

TechRepublic member Bob Eisenhardt (reisen55) sent me an email about an upgrade project in which he discovered a new category of IT problem: "mummies".

These are long dead and forgotten system settings that, once done years ago, are just lying there buried and quiet. On more than a few stations I had hard-coded DNS servers to be those provided by the ISP itself, so when they moved to the new domain with DNS on the server... NOPE, they continued to find DNS outside of the building with a ton of error issues.

Mummies, hard to find and easy to fix once you know where the bodies are buried.

I have to admit to having buried a few mummies of exactly the same persuasion myself. The DNS server is down, so you point to a public server just to be able to communicate with the outside world -- then you forget about it until it rises from the dead looking for your brains. No, wait -- that's zombies. It goes searching for its lost love... or something.

System settings such as DNS aren't the only places where mummies can lie hidden for centuries, however; they're probably even more abundant in code. Mummies are often harmless -- for instance, printf debug messages that get piped to the bitbucket in a production system. Nobody knows they're even there until someone happens to run the process with unredirected output and receives a friendly "We made it!" on the console. This is yet another reason to avoid curse words in debug messages -- or better yet, use a debugging facility that gets automatically excluded from production code.

More insipid and destructive mummies lurk in sections of code that have no output, nor any other useful function. You reorganized the code and either missed a section or didn't understand it, so you let it be. Now it's happily collecting an array of a million items and throwing it away on every transaction, just eating the processor's brains (wait, that's zombies again!). These mummies can lie buried for ages, unless someone happens to stumble across them. Even then, the stumbler may not feel qualified to determine whether they can be safely removed. Thus, they tend to multiply, as each successive maintainer adds a new solution that he or she can understand, to solve almost the same problem. These mummies make a good argument for regular code reviews and cleanup.

Another serious class of mummy includes just about anything "hard-coded": URLs (or God forbid, IP addresses), directory paths, array sizes, etc. Here's a good rule of thumb: If you're about to use a literal value, don't. Other good reasons, such as the DRY principle, exist for confining literals to unique central locations, but to avoid the mummies don't use them at all. Make mummies user-configurable or derive them from some other parameter instead. Try to keep all the configurable knobs in one place, too, so you and your client can easily review all of the system's settings when major changes occur. Make their location obvious instead of cryptic, so the mummies have no place to hide.

On a more general level, a good MAP (Mummy Avoidance Policy) includes the recognition of a denouement phase for any deliverable. Too often, we consultants move directly from "Ah ha, it works!" to "Ship it!" without allowing ourselves time to unravel all of our debugging knots and insure that the "ship" is indeed C-worthy (decide for yourself whether C is for "Client", "Chief" X Officer, the "C" programming language, or "Chip's Corny Cleverness").

Okay, time to steer back up the metaphoric Nile and rejoin our mummies.

I remember well the first time I viewed a real Egyptian mummy at the Smithsonian National Museum of Natural History. While I found that specimen's state of preservation after more than three millennia truly amazing, he certainly wasn't in any kind of shape to do anything productive.

Before you declare any project finished, take the time to remove the mummies -- at best, they're only dead weight.

About

Chip Camden has been programming since 1978, and he's still not done. An independent consultant since 1991, Chip specializes in software development tools, languages, and migration to new technology. Besides writing for TechRepublic's IT Consultant b...

31 comments
pete_nathan
pete_nathan

Chip & Bob, While I got a kick out of reading this blog post, and enjoyed the description of Mummies, this is hardly a new category of IT problem. Let?s go back to the root source of this IT problem which has been called ?Technical Debt?. Technical Debt is a term coined in 1992 by Ward Cunningham who is the developer of the Wiki concept and co-inventor of the foundations of Extreme Programming (XP). While working on a Wall Street financial application, Cunningham came up with an analogy to financial debt. Simply, poorly-written code requires the repayment of the original cost of coding with a future "interest payments" in the form of re-work. Rework here means re-coding or fixes deferred later onto the maintenance team?s effort. This type of re-work was what Bob was doing in finding and diggin up his mummies. This re-work takes place after bugs have been introduced into production and have to be eliminated, often under the pressure of time constraints. That expensive re-work would be smaller, or even non-existent, had the code been crafted more carefully, or if there were better long-term product planning and project planning. Of course, you must remember that rework is the most expensive work on a project according to the concept of the Cost of Poor Quality. It is always cheaper to do it right the first time according to Qualit guru Crosby. Technical Debt includes those internal things that you choose not to do now, but which will impede future development if left undone. Technical Debt includes such items as: undocumented code or configuration setting, hard coding, cluttered code or configuration, no coding standards, not following coding conventions, generic variable names, creating one class in a case where you should create two, failed tests, unfixed bugs, fragile code, ugly code (code that is difficult to understand, refactor, extend and maintain), code without comments or Outdated comments, unpatched security holes and deferred refactoring. There is a much greater list of items that indicate if your team has technical debt or will inherit technical debts from a previous phase/release. However, what Bob has identified as ?Mummies? is a good argument for consistency on a team or an IT department and against outsourcing. There is always the "old dog" in development who has been around a long time and who knows the ins and outs of the code, what has to happen to ?make it work? and how to fix it when it breaks. I?d like to hear more about these factors in Technical Debt and its variants, Design/Architecture/Security/Testing/Project Debt in future columns.

Sterling chip Camden
Sterling chip Camden

I think Bob mean that the term "mummies" could server as a new label for that phenomenon. I like your idea for an article exploring the concept of Technical Debt. Thanks!

reisen55
reisen55

Windows is just strange. At this account, I have WinProxy running on a single station for internet access, and it is a nice little program, easy to configure and not bad in it's day. That was about 10 years ago. When I opened up internet across the network direct, and removed WinProxy, internet direction went away from 192.168.1.1 to DHCP and DNS through a router. OK, simple. 18 months later, one system SUDDENLY REMEMBERED 192.168.1.1. Why? Heaven or Steve Ballmer knows, it just did it. To this day I cannot understand what happened but it was easy to fix. Resurrection of dead system settings. Plan 9 from Outer Space lives.

Sterling chip Camden
Sterling chip Camden

... resurrection of old settings. I've always comforted myself in the belief that a rational explanation lies in some hidden complexity for which I haven't accounted, rather than a supernatural event.

Englebert
Englebert

...sanctioned and approved by your manager. With an end-date, deliverables, delegation, resource allocation etc The mistake most make is expect this to happen on the fly. Never gets done, nor is there any motivation to do so.

Tony Hopkinson
Tony Hopkinson

The developer will be motivated to do so, in fact the reason they haven't is usally they've been specifically told not to. Or the code is so crap, they are scared to change any of it. This is a basic legacy code base refactor, if you want your code to at least not get worse every time you change it, fixing this sort of crap as you run into it, or moving towards being able to fix it, should be part of the day to day process. Once decent unit tests are in place, the overhead of doing it compared to an approved refactoring binge is trivial. Believing management are really going to allocate time to refactoring over say higher priority work like changing the colour of a button makes you sound sort of new... One of your peers (another consultant) on this site described refactoring as unproductive developer twiddling... Someone else wondered why we can't write the code properly in the first place.... You want this stuff fixed, get unit tests in place, if you can't, change your code only enough to do so. Then when you see a bit of refactoring, you can do it and test it at the same time as you are fixing a bug, adding a feature etc. The way a book I read recently, which also recommeded this approach was "Don't let best get in the way of better" Working Effectively With Legacy Code, by Michael C Feathers, have a read... The only reason for there not to be motivation to fix bad code, is you've employed incompetents, or you are employed by incompetents.

mvsenin
mvsenin

"Do not create the mummies during project" or "remove the mummies before completing" - what is cheaper? I think the 1st one, but this requires presence of a CULTURE, aka as PROCESS. So, in my humlbe oppinion it's better to consider this in the scope of all the work done during the project.

oldbaritone
oldbaritone

the "quick-and-easy" Get_registry_value and Set_registry_value point into HKCU by default. If you store program (software as opposed to user) values there, like DNS, URLs or IPs, they will "disappear" when a different user signs on the desktop and tries to use the software. I got burned many times in the early days, until I started using the full call and placing software settings in HKLM\SOFTWARE. And of course, DOCUMENT and provide a way for settings to be changed when necessary!

Tony Hopkinson
Tony Hopkinson

Current user is for user specific settings, you need to keep a wary eye out what you out in HKEY_Local_Machine as well, virtualisation for instance. My own policy is to not use the registry if I can avoid it. Bloated, flakey and very hard to manage. xml config file with the executable or in the backend if there is one are my default choices.

Sterling chip Camden
Sterling chip Camden

Hiding application-specific settings in an operating-system database has always seemed like a bad idea to me, no matter what Microsoft says on that subject. I like to keep application-specific settings in an application-specific data store, whether it be an rc file or a database.

Tony Hopkinson
Tony Hopkinson

that's why they went to .exe.cfg (modern ini file basically and such like. Registry was one of those ideas that looked good on paper, but failed big style in the real world.

Sterling chip Camden
Sterling chip Camden

Another one of those Grand Unified Theories that never work in practice. I made the very same design mistake early in my career, trying to consolidate databases. I had the good luck to learn that lesson while young -- it amazes me that Microsoft didn't learn it until they unleashed that disaster on millions of users.

Tony Hopkinson
Tony Hopkinson

is the best approach, ripping them out of legacy code can be a very problematic exercise, even if it's "trivial". I got rid of this mummy. Right where will that impact? Nowhere (fingers tightly crossed) How do we prove that from a QA perspective. &*^&^&^*!!!ing ^&*^&**T

Sterling chip Camden
Sterling chip Camden

Some languages and frameworks encourage them more than others. Especially those without good debugging facilities. It's easy to create useless code that was once used for debugging/profiling.

Tony Hopkinson
Tony Hopkinson

For instance one development logging "everything" was a requirement, so that became a question of how. There's always the good old compiler directive, unit testing, test data. 99% of these sorts of problems are self inflicted, make yourself do it right, burn in good habits instead of bads ones. Set up to succeed, define a constants unit, set up a resource file, set debug to true, code so you can unit test. Once these things become habit they don't take longer, and we all know the I'll just stick this in here for now, or I'll make that a named constant later because I'm in hurry is just a faster way to definitely go wrong sooner. 60% of the time you spend debugging or maintaining code is comprehension. There will be bugs, there will bodges, there will be changes, it's an investment with a guaranteed return.

maclovin
maclovin

[And this is not meant to offend, but] it comes down to knowing your environment. Documentation (and memorization) will allow you to know exactly what needs to be changed when moving sites, making changes etc. I deal with developers for a very specific piece of insurance software that are always seeming to get lost in the fray, and then blame it on my network! (Which probably explains my deeply rooted hatred of programmers) I understand how one could forego the documentation, because I think we've all been there before! It could be time constraints, a predecessor that wants to be secretive about this magical IT stuff, or you might be that predecessor! XD I used to be that way for a while (I still have my moments....probably inexperience and general childish habits) but if you document everything it seems to me that you would further solidify your position, since they know exactly how much you have to take care of!

Tony Hopkinson
Tony Hopkinson

It's called good enough. Documentation, particularly code annotation is an absolute classic, it takes time, it can double your maintenance burden, "customers" are very reluctant to pay for it and out of date documentation is actually less helpful than none. The most cost effective approach for everyone is to not put yourself in a position where you need the extra overhead. That's replacing if(totalMinutes == 1440) { ... } with Const NumberOfMinutesInOneDay = 1440 and using it from then on, literals of any form are constants, make them so. If they could change e.g ErrorColour or DefaultDateFormat, then make them resources or configurable options and put them in some logical place and document that.

maclovin
maclovin

Good enough is so true. and is understood to be the de facto standard, it seems. It's still quite unfortunate, though!

Sterling chip Camden
Sterling chip Camden

... keeping things simple to begin with. Complexity builds, no matter what you do. So you try to minimize it, and document the rest. That's in a perfect world. In the real one, we end up doing a lot of digging.

Sterling chip Camden
Sterling chip Camden

... for keeping your projects free from the mummies?

oldbaritone
oldbaritone

More than any other single point, many programmers are guilty of this one! First, there NEEDS to be documentation and a manual for the project, and it NEEDS to be accurate "as-built", not just as originally specified. All of the mummies buried here probably fall into that category. Hard-coding is a no-no, but even if the setting is buried in a registry hive, is there documentation about what it is, where it is, and how to re-set it without using REGEDIT? We love to tell users "RTFM!" but if the manual isn't accurate, it's our fault, not theirs.

Sterling chip Camden
Sterling chip Camden

... is the real challenge. Usually, you write fairly detailed requirements to get customer sign off, but the process often doesn't include incentives for updating the docs as the requirements change. That's something you just have to enforce.

Bill Ward
Bill Ward

Or rather, no magic numbers in the code proper. On the projects I've worked on, I've developed a huge aversion to "magic numbers", i.e., hardcoded values that "just make the code work" or hard coded limits to the code. You can't ALWAYS avoid the basic REASON for the hard coded number (a 64K limit due to a two byte index field is still a 64K limit, after all), but it's good to at least TRY. So I always define any such number as a constant, and put it in either a header file or constants file, the program headers themselves, a default configuration file, or in the actual configuration files. Yet even with the mantra, on code I've written in the last two years, I'll occassionally come across someplace where a magic number that got inserted "just to test" before being properly broken out got left in. It's maddening, because a full 30% of all bugs come from magic numbers, in my experience, where the wrong number is being used. That 64K number I mentioned for example.... was that 0-64K-1 or 1-64K for the allowable range? It makes a difference when you put the magic number as 65535 or 65536; one might give you a rare, difficult to find exception violation or out of bounds error, for example, even though most of the time (since you're being consistent in using it!) the code works correctly.....

Sterling chip Camden
Sterling chip Camden

by color-coding literal values in an awful shade of pink or something equally noticeable.

sysop-dr
sysop-dr

Maybe use a computer and language that uses 32 bit and your problem goes away. And while you are at it maxint or interger.maxint or whatever the defined language item is for max int and your magic number goes away. For other magic numbers like pi or Planck's constant you may still have to define them. Other than that there should be no "magic" numbers. Try to avoid pointer arithmetic and test for end of indices is a better way to keep bugs out of your software. I have seen so many places where people are incrementing counters within loops that also increment the counter it's not funny (like incrementing x in a for x loop.) Test your inputs for bounds and always check/embargo user input. Nothing beats code reviews and proper unit testing for making code secure and bug free.

Sterling chip Camden
Sterling chip Camden

I can think of two off the top of my head: Text of an edit control in VB6. Data references in Synergy/DE on 32-bit platforms. In both cases, a 16-bit value is used for the length, even on 32-bit platforms. But your second piece of advice is right on: try to derive the max if you must have it, instead of coding it as a literal anywhere.

Charles Bundy
Charles Bundy

A long long time ago an install routine with a free hard drive check was written by yours truly. Code was tested and deemed bullet proof. A client called and said the software wouldn't install on his new 386 PC because the installer said he had no free space on his new hard drive. How big is your new hard drive I asked? 600Meg he replied. Lo these were the days when a 40Meg MFM/RLL drive was considered huge. After picking my jaw off the floor I changed the code so that two's complement overflow would not return a negative on what had seemed at the time to be a gi-normous temp variable. Blasted hardware progress!

Editor's Picks