Banking optimize

Avoid getting buried in technical debt

Ward Cunningham wrote in 1992 that shipping first-time code is like going into technical debt. Chip Camden explains that the usefulness of the technical debt metaphor is awareness.

In an experience report on the benefits of object-orientation for OOPSLA '92, Ward Cunningham observed:

Another, more serious pitfall is the failure to consolidate. Although immature code may work fine and be completely acceptable to the customer, excess quantities will make a program unmasterable, leading to extreme specialization of programmers and finally an inflexible product. Shipping first time code is like going into debt. A little debt speeds development so long as it is paid back promptly with a rewrite. Objects make the cost of this transaction tolerable. The danger occurs when the debt is not repaid. Every minute spent on not-quite-right code counts as interest on that debt. Entire engineering organizations can be brought to a stand-still under the debt load of an unconsolidated implementation, object- oriented or otherwise.

Cunningham's debt metaphor has since become popularized as "technical debt" or "design debt". The analogy applies well on several levels.

Debt service. Steve McConnell points out that where there's debt, there's interest. Not only do you eventually have to implement a correct replacement (paying back the principle), but in the mean time, you must work around what you implemented incorrectly (that's the interest). Doing it wrong takes up more time in the long run, although it might be faster in the short term. Deficit coding. Some organizations treat technical debt similarly to how some governments and individuals treat fiscal debt: they ignore it and just keep borrowing and spending. In terms of technical debt, that means ignoring the mistakes of the past and continuing to patch new solutions over old problems. Eventually, though, these patches take longer and longer to implement successfully. Sometimes, "success" gets redefined in terms of the minimum required to get by. More significantly, the entire system becomes more brittle. Nobody fully comprehends all of its dependencies, and even those who come close can't make major changes without breaking things. Users begin to wonder why problems take so long to get fixed (if they ever do) and why new problems arise with every release. Write-offs. You always have the alternative of declaring technical bankruptcy. Throw out the project and start over. As in the financial world, though, the consequences of that decision aren't trivial. You can lose a lot of credit with users and supporters during the interim when you don't have a product. Furthermore, a redesign from the ground up is a lot more work than most people realize, and you have to make sure it's done right. The worst possible scenario would be to spend millions of dollars, years of effort, and end up with only a newer, shinier pile of technical debt. The very fact that you're considering that kind of drastic measure indicates strongly against your success: the bad habits that got you here have probably left thousands of critical system requirements completely undocumented. Good luck discovering those before you ship something to customers. It's not all bad. Strategic debt can leverage finances, and the same holds true in the technical world. Sometimes you need to get to market more quickly than you can do it the right way. So, you make a strategic decision to hack part of the system together, with a plan to go back later and redesign that portion. The key here is that you know that you're incurring a debt, and it's all part of a plan that won't allow that debt to get out of control. It's intentional, not accidental.

That's the main benefit of using the technical debt metaphor: awareness. Too often, after a particularly bloody operation on a piece of unmaintainable code, a developer will approach his or her manager with "We really need to rewrite this module," only to be brushed off with "Why? It's working now, isn't it?" Even if the developer possesses the debating skills necessary to point out that all subsequent changes to this code would benefit from taking some time now to refactor it, the manager would rather take chances on the future, because "we've got enough on our plate already."

By framing the problem in terms of the debt metaphor, its unsustainability becomes clear. Most professionals can look at a balance sheet with growing liabilities and tell you that "somethin's gotta change." It isn't always so apparent when you're digging a similar hole technically.

Thanks to TechRepublic member pete_nathan for suggesting this topic.

Note: This TechRepublic post originally published on January 27, 2011.

Keep your engineering skills up to date by signing up for TechRepublic's free Software Engineer newsletter, delivered each Tuesday.

About

Chip Camden has been programming since 1978, and he's still not done. An independent consultant since 1991, Chip specializes in software development tools, languages, and migration to new technology. Besides writing for TechRepublic's IT Consultant b...

180 comments
mslizny
mslizny

"Debt service. Steve McConnell points out that where there’s debt, there’s interest. Not only do you eventually have to implement a correct replacement (paying back the principle) principal - the loan amount interest - the amount paid for the use of the money Science and finance use different words which sound the same.

wordworker
wordworker

@Tony >>academic guff or, MBA style drivel Pot, meet kettle. >>all designs are a series of compromises based on (hopefully well identified) priorities in an ever chnaging environment True, and the better identified, the better the requirements >>design and code quality are perceived as very low value. Design and code quality perceived as low value? What are you smoking, brother? I work with 20- and 30-something year old ASP, .NET, C programmers who think they're smarter than everybody else, and even they wouldn't make such a ridiculous statement about quality of design and code. Not having THAT, tony.

gechurch
gechurch

I hadn't actually heard of this metaphor before. It's a nice easy way to think of and discuss the problem. I've got one comment to add: never rewrite your code from scratch. Looking at a poor codebase it is easy to feel that it is an unmanageable mess that needs throwing away and starting again. It is always (without exception) better to refactor. Your codebase is the documentation of your products lifetime worth of knowledge and fixed bugs. Starting from scratch is throwing away all that knowledge, all those lessons learnt. You will end up having many of the same bugs again. Any time you had written code to work around a bug in a third-party component, or some strange behaviour when the OS is in a language other than English, or when the screen DPI is set unusually low or whatever. It also means you can't fix new bugs that existing customers find. You can't ship a new version to respond to changing customer expectations, or to keep up with a feature a competitor has brought out. You will end up taking a long time to end up with a feature-identical, but more buggy version of the software that you already had shipped. Never start from scratch. Netscape taught us this lesson.

Tony Hopkinson
Tony Hopkinson

firmly wedded to. On a new project they want to get from 'just spending' to 'start earning' as quick as possible. Now you have an existing project where they went as quick as possible, and you start saying things like to get this right we'll have to redo a substantial amount of work... It's not just code, it's upgrade, data transforms, prerequisites, testing, finding something saleable to justify the fixes to the client(s)... Hands up who's seen this one. We are going to stop doing unit tests for test build test, because they are taking too long.... No dev worth the name suggested it... The famous undeliverable deliverable...

techrepublic
techrepublic

"Strategic debt can leverage finances, and the same holds true in the technical world. Sometimes you need to get to market more quickly than you can do it the right way. So, you make a strategic decision to hack part of the system together, with a plan to go back later and redesign that portion." This is the great trap. If you "get to market" with crap, you'll simply disappoint your customers/clients and ruin your reputation. Also, the "re-design it later" almost never happens and even if it does, it is always cheaper and faster to have designed it right in the first place. This is all assuming that one knows how do good design/implementation at all. I have a saying: "If you cut corners, the corners will cut you."

jkameleon
jkameleon

It's nothing wrong with technical debt accumulation, really. Fair warning is in order, of course, like "I can do that quick and dirty if you want, but next time we are going to change this code, it will take more time". The answer is always "Who cares about the next time, just do it fast". That usually makes sense. It's a good chance, that the company, the code, the manager, and especially you will not be around long enough to see that next time. In any case, your employer is your customer, and customer is always right.

Tony Hopkinson
Tony Hopkinson

"Change this application, so a customer can have more than one order" They'd written version one for a land grab, didn't need multiple orders all they wanted was the first one, to prove the concept. All the way through the design, order and customer a 1 to 1 relationship, in real terms there was no such thing as order or customer, it was some unholy confabulation. It gets better though, they gave the intial go to one of your newbie clueless types, in fact worse straight out of academia, nice and cheap. It was a telecomms application, it had no concept of provisioning, the second order some customers wanted was to change their address..... On top of that the UI design was a multi tab single window, designed from the worksheet they collected the data on, in the same order, which I can assure you was not suffering from any sort of logical organisation. Better still the guy had obviously missed the class on coupling and cohesion, If you referenced the "customer" module, you had to linlk to another 89 modules for it to build. There were only two more in the entire solution. Just to cap it off, these pratts did a link to their billing system for producing the entire one invoice for the entire one order, by customer surname... So come again? Would you like to here about a highly succssful american business that tried to go multi-national and hired the R&D managers's mate who'd never written a serious piece of software in his life, never used the chosen language , knew nothing about databases, or client server, had two development teams seperated by the atlantic, who weren't allowed to talk to each other, to come up with a system that communicated via xml documents through a flat file only aware mailbox system, that hadn't figured out that we don't routinely use dolloars over here and that america and the UK are different countries? I've been dealing with this crap since 1987, that's just two examples, glaring ones I'll admit, so still not having it. Don't tell me my contention is ridiculous by comparing me to people you don't have any respect for either. Go ask them if they think design and implementaton quality have a higher prioity than the colour of the buttons or getting it done yesterday, I'm guessing you won't like what they tell you. Whether it should have a bit more consideration, that;s a different question, in fact it's the entire reasn behind this thread, something which quite obviously went right over your head. Go back and read what technical debt is,pay attention to the responses where denying that it exists, doesn't really help much. Start being part of the solution.

Tony Hopkinson
Tony Hopkinson

A good design with bad code, can be refactored. If the design is bad (doesn't matter why or for how long) rescuing it with more code is extremely high risk, at that point you do need to look at start again, though not necessarily in total. The thing either strategy needs to take into account, is believing either approach will be a simple refactoring (no additional functionality) is a total pipe dream. So, do you keep the two versions in step, ie can you afford to freeze the old code. Where is the resource coming from? Can you justify it to the business? Will your IT managers have the balls and your business managers have the brains, to accept that the current software is fubar? Another key question is what sort of transistion / migration pain will there be. Can you get your customers to pay for the "upgrade". Initial loss versus having to offer new stuff as a hook. I personally prefer a combination of start again and refactor, an evolutionary approach. It will mean the benefits will take longer to come through, but it also reduces cost and risk, start again looks cheaper, but you f'ed up once and in my experience the business will not learn from that, so they'll do it again. Treat the thing like legacy code, improve to move functionality into a new design / code base. Find logical partitions, e.g. if your database layer is a mess, start pushing it into interfaces, and then re-implement them. No separation of concerns, same a again, move the bad code to an implementation layer, abstract it, reimplement it. The only extra technical debt you should take on, is that which reduces your exposure. Look at it as extending your loan, to reduce your 'monthly' payments. Refactor where you can, rewrite where you have to, and refactor to do that transaparently as far as your customers are concerned. Above all remember which ever approach you were going to take including do neither, iterate towards what you nedd and review constantly, because the thing that got you in this mess is still true. Change is a given....

apotheon
apotheon

There is a smart way to start over from scratch. Create a new, essentially competing version of the application. Offer them both side-by-side. By simple virtue of the fact that it is better written (that's why you started over -- right?) and more maintainable, the new application will quickly overtake the old in terms of its suitability for general use. Drop support for the old version when nobody wants it any longer. You just have to make sure you actually support the old version, too, during this time.

herlizness
herlizness

> oh no no no .... some code is SO bad it'll take several times longer to do revisions than it will to just toss the whole mess and do it right ... you don't want to throw good money after bad and too many people do just that in the interests of "preserving their investment." They should have thought about that while they were making the initial investment.

Tony Hopkinson
Tony Hopkinson

Ending up starting again is the cost of not paying off your debts. It's on the amass technical debt side of the scale, not it's opposite. Why would collected bug fixes be valuable, I means you weren't planning on recreating the bugs in teh new versions were you....

Tony Hopkinson
Tony Hopkinson

While I agree that redesign never happens, which is really the practical aspect of amassing tecnical debt after all if redesign did happen, it wouldn't be an issue, would it. Getting out to market first in a new market fdor a landgrab, or a loss leader type approach to gain a market share are hugely valuable, indeed success is waht would give you the revenue to resource doing it right. Designed right in the first place... What are you working on sun dials and water clocks? Define right, then tell me how long it will be right for, then tell me why what you are doing has any value at all, if it's existence won't change what right is.... No one with any familiarity with software design would argue that not paying it off is a poor option, not getting into debt in the first place, is rarely optional.

jkameleon
jkameleon

Your reasoning is correct, but people generally aren't reasonable, and somebody always makes a profit out of it. Living on credit is bad. Credit card fees and consumer credit interest increase your cost of living for about 30% or more. Yet, almost everybody does it, and banks make a profit out of it. Similarily, incurring technical debt is bad, for obvious reasons you described so well in your post. Yet, almost all big software producers do it, and software developers (sometimes unwillingly) make a profit out of it. Time you spend redesigning the application and untangling the spagetti code is not yours if you are employed. It's paid for by your employer. So, if your employer tells you to cut corners, you cut corners, because the corners won't cut you. They will cut your employer, and keep your job. Your resoning applies only if you write software for your own use, for your own business.

apotheon
apotheon

1. You can't change a vote on TR, as far as I've been able to determine. 2. I'm kind of offended by the idea that doing something poorly to ensure later job security is somehow good. It's actively harmful to innovation and advancement.

Sterling chip Camden
Sterling chip Camden

... most providers can't afford to support two versions of an application of any complexity.

gechurch
gechurch

If you are talking about small snippets of code then I absolutely agree. Perhaps my definition of refactoring differs to others, but to my mind refactoring includes rewriting the poor parts of the code. It's just important to do it piecemeal and keep the code working. If however by "some code" you mean "some entire programs" I would disagree, at least for projects of any size. Small projects can be rewritten in a more appropriate language, or with a better design in mind after afirst build. There is little risk for small programs because the investment in time is so much less, and all the lessons you learnt writing the small program will be fresh in your mind. I can't imagine any programming effort of significant size being all bad though. Even if it started off that way, as it grew it would become obvious that things are unmanageable and people would begin coming up with standards and coding styles. If the coders were so hapless and unable to learn I find it hard to imagine them creating a worthwhile program that people would pay for. That level of incompetence would doom the project before it got to such a large size.

Sterling chip Camden
Sterling chip Camden

... what I said agrees with Joel here. You can rarely, if ever, afford the "bankruptcy" option. Just as in finances, people often mistake bankruptcy for a free pass -- but it has steep costs.

apotheon
apotheon

Yes, the first to market "wins", at least at first. Of course, the "right" thing to do after that is jump ship to someone else who's trying to build a better version of the same thing and offer it at a competitive price, competing on quality. Consider the example of W3Catalog, arguably the first major public Internet search engine -- which did nothing but mirror listings from site indices, a quick-and-dirty approach to building a search engine index. It went the way of the dodo thanks to the efforts of people who later did things with an eye toward sustainable quality. It's probably no mere coincidence that AltaVista appeared on the scene just under a year before W3Catalog shut down. What you describe as a business model has little bearing outside of a race to market for the first-time entry in a whole new market niche. More to the point, it is just as easy to be first to market with a well-written, focused offering as it is to be first to market with a poorly-written, unfocused offering. Why not take the former approach instead of the latter, and ensure some resiliency against future attempts to outdo you -- giving you room to grow and improve, rather than trying to compete on the strength of your crufty bloat?

jkameleon
jkameleon

getting into technical debt is business, not technical decision.

apotheon
apotheon

Predators taking advantage of the weak are a poor example. Sure, you do what your employer (who never wants anything but quick and dirty garbage) tells you to do -- but if you are not a completely spirit-broken waste of space, you also look for a better employer.

jkameleon
jkameleon

1. I wouldn't know. IMHO the voting system is stupid, and I don't intend to use it nor take it into consideration. This is supposed to be technical, not political forum. 2. What is good depends on project requirements. The classical example of misplaced goodnes is WWII German King Tiger tank. The best tank at the time, and, for the same reason, in wider context, a failure. The same principles apply to software development. When you expect application to be around for a while, the most important aspect is readability, or maintainability. I imagine that's what you consider "good". In different circumstances, however, it's time to market, or certain functionality. Which of the above has priority is business, not technical decision. In any case, good developer must be able to find a proper balance. If I'm told "I want this application readable & maintainable, because it will go through revision", I'll write it readable & maintainable, and won't care much about performance, and such. If I' m told "This part of the code is critical, It's our front end, it must not fail, it must be absolutely secure", I'll write it nice and easy, and pay attention to every detail. If I'm told I want this done fast (which is usually the case), I'll do it fast, and won't pay attention to anything else. Never mind if it's going to be crashable, never mind if it has to be rewritten- the customer wanted it done fast, and got what he requested. And I love such requests, because they mean more business for me later.

Sterling chip Camden
Sterling chip Camden

... if any intelligence can be found. For the universe, Intelligent Design explains a lot of things... by its absence.

Tony Hopkinson
Tony Hopkinson

of what will be done was the basis of the whinge and moan. I was talking with one our marketing boys t9oday and he actually admitted that pressure from them and sales had forced us down the inflatable dartboard development path, in the past That was a first, mind you we had him surrounded. :p

apotheon
apotheon

I thought we were talking about what should be done, rather than what would be done, though.

Tony Hopkinson
Tony Hopkinson

Where did the assumption that the new version isn't going to be as big a mess, if not bigger than the one it replaces. All the "reasons" that the first attempt went nipples over bum are still in play, if not even more so. No business head is talking major changes to existsing software or total rewrite because you and I don't find the code aesthetically pleasing. :( They are looking at it because of can't. Not Vista compatible, different language, not HTML5, can't ever be web based. Some real road block to any future continuance of the code base. There's a deadline associated with that, if new version will take you past the deadline, or is more resource than they hoped, expect at least 1/4 to a 1/2 of the time you could have had to do it to be piffled away in a desperate denial of reality, especially if the muppets who authored the orginal disaster are still in power. It won't matter why you got to where you are, a good deal of effort must be spent in definitely not being seen to be responsible for it. Worse still these muppets will have left an impression that the old code could be rescued, they may even initially set an expectation that it will be, usually without discussing ot with any one vaguely technical to avoid them injecting large doses of reality to their convenient hallucination. At this point it's spit roasting time, best thing you can get is to decide who's in the middle, followed by choosing which end of you big Fred is gonna be at...

Tony Hopkinson
Tony Hopkinson

The revenue generated from the old product is the wherewithal and the justification for further work. It's real money not some marketing boy's projection. Even if you can get away with freezing it in terms of features, database server upgrade, Office Upgrade , OS upgrade, new version of a browser, legislative changes... Any of which can drastically effect the bottom line. If you let you get to the point where rewrite starts becoming attractive, you are hell already, unless the business planned for obsolescence via neglect...

apotheon
apotheon

> Windows XP was the first release of this line of OS aimed at home users. Actually, Win2k was intended to be the OS that unified the home and business lines of MS Windows. The problem is that home users rebelled at stupid little things like the login process. It was "too hard". That's why Microsoft rushed that atrocity WinME out the door in such a hurry; to provide something to tide over the stupidest of home users while developing a unifying OS that would allow them to drop the worst of the two codebases. The big differences between XP when it was first released and Win2k were: 1. the fact that the login process was gutted by default, though configurable to use a proper login if the user so desired 2. the Fisher-Price widget set for those who preferred big, primary-color buttons, like infants do It was not until Service Pack 2 that XP really became a different OS from Win2k under the hood.

apotheon
apotheon

> It's been well established that the reason the old codebase doesn't get fixed is invariably "time". Note that I never said this is the way updates to a system should be written. I'm talking about the case where building on what you currently have is not an option -- where your options are basically these: 1. stop further development to do a complete refactor starting from the foundations of the system 2. throw it away and start over 3. spend exponentially more time on every feature 4. keep on going exactly as you have been, but lose all your customers because they already hate it and it's only getting worse 5. throw it all away and don't rewrite it from scratch because it's not worth it any longer Under such circumstances, the approach I described is a best-case "compromise" -- except that, like all truly good compromises, it is no compromise at all. In short, I'm not talking about spending more time than you have because you have no time to do it right with what you've already built. I'm talking about spending more time than you have because all the time in the world will not fix what you already have while keeping your customer base. This is the option that might have saved Netscape -- but Netscape didn't try it. Goodbye, Netscape. It's the option that is currently keeping Perl relevant even while there's a (so far vaporware) complete rewrite of Perl underway. When time is the smallest problem with your legacy codebase, it doesn't matter any longer whether you think you have no time. You have to spend something, and if time is cheaper than all your other options (including customers), time is what you try spending to dig yourself out of debt. > allow you to spend thousands of manhours to create a product that has nothing new to offer customers. If your new version has nothing new to offer customers, you're doing it wrong. > And even if you suddenly have so, so many manhours become available, why is it desirable to re-create the program you already have instead of going through the existing code and refactoring? . . . because WordPress is a tangled mess of BS spaghetti code that is doing more harm than good, and should be replaced (for example). It has gotten to the point where the only people who are really properly served by it are people who would be about as well served by LiveJournal and Blogger -- which is to say that it is now completely useless to almost everyone with any technical chops at all who wants a private domain with any custom platform design at all. It seems singularly designed to break with every upgrade if you do anything other than treat it as a one-size-fits-all cookie cutter application. You are, technically speaking, better off getting a LiveJournal, Blogger, or WordPress.com account and paying too much for too little to get a private domain at one of these account is if you really feel that you need it, than you are deploying a new WordPress install at your own hosting account to be able to have a custom setup for a reasonable price given what you can get out of it. . . . and, if you have the ability to do so, you're better off writing your own little CMS software from scratch to replace it, essentially taking on the job of the guys who should be trying to replace the WordPress codebase for the world anyway, because it's such garbage under the hood. With WordPress, "going through the code and refactoring" is not going to solve the problem. If you try "going through the code and refactoring", and end up with something worth keeping, the end result will be a complete rewrite, but with a lot more agonizing about whether you should keep parts of it along the way. News flash: the answer is "no, you should not keep that piece of code," regardless of which piece of code from WordPress you're examining. It's all garbage -- with the possible exception of a couple well-written pieces here and there whose only purpose is to make up for the deficiencies of other parts of the code, thus making themselves obsolete once you solve the underlying problems of those other pieces of code by throwing them away. On the other hand, I admire your optimism.

gechurch
gechurch

Yeah, nice examples. I would argue that Windows is an exception to the rule (well ok, you've got me - I said no exceptions! I guess it goes to show that you should never make blanket statements). Microsoft have/had some pretty unique attributes with Windows: - The OS was near-ubiquitous - They have oodles of cash and developers - They go to great pains to maintain backwards compatibility - The switch-over period was several years. NT started around the same time as Win 3.1, yet Windows XP was the first release of this line of OS aimed at home users. Giving your users and third-party developers a decade or so to get used to the new system isn't something most program developers can do. Still, it's a great example of a company making the "start from scratch" decision, and it being the right one. As for Apple, I've got no idea how they managed to create a new system that broke compatibility with the old. Their users just don't seem to care. I don't agree that OS X v iOS is or will be another example. That's not a case of rewriting the same product because the code is awful. It's a case of writing a new (although similar) product to fill a different role.

gechurch
gechurch

This sounds like the worst of both worlds to me. You've got the cost of rewriting from scratch, along with the burden of maintaining the old system. In fact, I think it is worse than that - I think it is impossible. It's been well established that the reason the old codebase doesn't get fixed is invariably "time". Your solution requires the same amount of time maintaining/updating the old codebase, PLUS the number of manhours it takes to write the entire program again from scratch! Where is this time going to come from? A management system that doesn't allow time to tidy up your code as you go surely isn't going to backflip and suddenly allow you to spend thousands of manhours to create a product that has nothing new to offer customers. And even if you suddenly have so, so many manhours become available, why is it desirable to re-create the program you already have instead of going through the existing code and refactoring?

Justin James
Justin James

... Windows NT vs. the 3.X/9.X branch, and Mas OS vs. OS X. Both were huge successes with a great payoff for both their vendors and the customers, but both were extraordinarily expensive and risky. One might argue that OS X vs. iOS may well be another example in a few more years. A third example is SunOS vs. Solaris (the jump from BSD to System V). Some might argue that this was a success, but it seems like everyone in the know at Sun regretted it later. J.Ja

apotheon
apotheon

That doesn't change the fact that's the right way to do it. The truth of the matter is that if you don't do it that way, but it needs to be done, you're almost certainly dooming your application altogether. If you throw it away and start over from scratch, you'll find you can even more-ill afford that over the parallel maintenance approach because users will flee to competitors while the current release stagnates (just as Netscape), and if you stick to the broken codebase you'll just become obsolete. Basically, if you don't become the competition that draws your customer base away, while still being the target of that customer base poaching, someone else will fill those roles -- and you're going to have a tough row to hoe even staying afloat in the long run. Parallel development is the best case scenario under most circumstances. (There are always exceptions.)

Tony Hopkinson
Tony Hopkinson

I implied that the whimps who steer clear of resolving the efforts of the 'best' devlopers who just left to be the 'best' developers on an other green field project are in fact not the best developers... The fact that they don't want to do the hard bit, means they are a waste of space, and more often than not because they are a lot of the reason it is hard.

herlizness
herlizness

To say that the best developers are more interested in greenfield projects is not the same as saying that all greenfield projects get the best developers.

Tony Hopkinson
Tony Hopkinson

Given we are talking about the usual situtation where a new development has gone into a maintenance cycle and then the wheels are found to missing.. In what way are they best? The more technical debt you have at the end of version 0, the more likely it is the business will see little option but to borrow again for version 1, and then 2 and 2.4 and 3 and .... A good experienced developer borrows at a much lower interest rate, and when I say experienced I mean experienced at paying it off, not accruing it. Every project I've worked on where the technical debt was high ,which is all but one ,and I designed that myself, a significant percentage of the cost of addressing the debt, was juvenile and or sloppy coding practice. Things no one who's had to fix their own 'errors' more than once would ever do again..... The real heart ache of working on a legacy code base, is only being allowed to make it worse. Being given license to fix it intelligently and effectively (not often unfortunately) is one of the most challenging tasks there is. The reason business keeps employing these incompetent wusses to do new software is their really valuable guys are still trying make the last effort work in the real world...

herlizness
herlizness

Perhaps you've been blessed to have never seen a true mess. You wouldn't think it would happen in prominent, very large organizations on major systems but it does. In some cases, it's a function of too many iterations of incremental patches and add-ons leading finally to an unmanageable and dysfunctional mess. In other cases, it's because the database schema was very poorly designed, it has to go, and accordingly all the code targeting it is useless. In theory, you might be able to retain and use some code .. perhaps some UI or business logic but, speaking broadly, tasking a team with sorting through a legacy mess tends to deplete their energy and enthusiasm for the project. Moreover, and again speaking generally, it's often difficult to attract the best talent to these kinds of salvage efforts ... the best people usually want new development projects and they can get them. I'm sure you're right that there are some situations where mods are the way to go but I think you were asserting that they are usually the way to go and I cannot agree. That said, I am fully aware that a lot of organizations do it your way and I think too often they never really get it right. I would note though, that I have also seen some complete re-writes fail completely as well .. a $60m fiasco comes to mind ... but can't name names.

apotheon
apotheon

> Who told you the business objective was to produce quality software? For publicly traded corporations, producing quality anything is certainly not the business objective. I never said it was. Quality is certainly the basis of many business models, however. Given enough time at the hands of short-sighted stockholders in publicly traded corporations, any business model will be corrupted into short term pursuit of market dominance, of course -- but many businesses do great for long periods of time focusing on quality as their distinguishing characteristic, rather than on fleecing their customers. The point is not that producing quality is the business object. The point is that it is a perfectly valid business objective that lends itself to a strong business model, but that fact is all too often ignored because of the perverse incentives pressed onto CEOs by the public shareholder ownership model currently in vogue.

Tony Hopkinson
Tony Hopkinson

Who told you the business objective was to produce quality software? The objective is to produce something you can sell. "Quality = good enough" Quality is seen as an overhead in business, yes that's stupid by any long term criteria, but business doesn't have a long term, it has a continuing short term. I quoted the business arguments, which I'm unfortunately all too familiar with, and while we might not like them, they are valid at least in terms of the fact that if you ignore them you will fail to communicate with business, and therefore achieve nothing.

apotheon
apotheon

> what you asking me business questions for? You're the one who compared trying to produce quality to working on technologies that have been obsolete for hundreds or thousands of years, as a means of ridiculing quality over quickness as the basis of a business model.

Tony Hopkinson
Tony Hopkinson

what you asking me business questions for? My own take, would be it would be harder to get that new job on the strength of 'your' current 'success' In my experience, it's basically down to control (giving your techies the power to define good enough) is anathematic to a business, and quality of people. Lost count of the number of 'successful' projects I've seen initiated by the inexperienced and then handed over for 'maintenance' to even less experienced cookie cutters. Aside from one period where I was effectively in charge, my entire career has been resolving or coping with technical debt. First casualty in a any non trivial commercial software development is always quality, Quite often before development starts as well...

apotheon
apotheon

There's a lot of complacency in the world. People, for instance, seem to think that there's some kind of vast conspiracy in government to protect the little guy, so they keep electing Democrats or Republicans (depending on their particular perspectives on that conspiracy) in the hopes these evil, corrupt manipulators will do what's right for the people. It's absurd to anyone who hasn't bought into it, of course, but that's shockingly fewer people than it should be. That complacency covers the banking industry to some extent too. I suspect the same is true of where you live, though it might manifest differently.

jkameleon
jkameleon

Where I live, mistrust towards the banks is something that goes without saying.

Sterling chip Camden
Sterling chip Camden

Predators on trust. A lot of people trusted their loan officers for financial advice. Granted, in retrospect that was stupid, but historically loan officers were more conservative than those seeking a loan. So when the loan officer said, your monthly payment will be $N, the people applying for the loan said "Hey, we can do that!" without having the sense to notice that they weren't paying down the premium at all and that the rate would adjust. Yes, they were stupid. Yes, they should have known better. But the loan agencies were still predators on their ignorance.

jkameleon
jkameleon

Nobody forces nobody to take a bank credit. People do it by their own free will. And, for the most time, they even bitch against the bank if they don't get it. Nobody prevents nobody from writing good software. Take part in an open source project, and do it. Just don't expect to be paid for it, because this is not what market wants. For some reason, don't ask me why, Q&D is what people are willing to pay for.

jkameleon
jkameleon

Chasing and beating the riot police another. And, because political system doesn't work as it should, there's nothing in between.

Sterling chip Camden
Sterling chip Camden

... unfortunately has too much to do with everything now. And yes, it was bought and paid for.

apotheon
apotheon

I guess we might as well all just lie down and die.

apotheon
apotheon

Life's too short to go through it like a jobsworth drone.

jkameleon
jkameleon

... has nothing to do with anything anymore. It was bought and paid for a long time ago.

apotheon
apotheon

When government starts setting rules for how business should be conducted, so that a legal framework for consolidation of power under immortal bureaucracies with legal "rights" equivalent to those of actual human beings, the market is not free.

jkameleon
jkameleon

... in a free market way. Everything that makes a profit for my employer, and salary for me is ethical. Everything else is just evil propaganda, propagated by evil mother f'ers trying to reduce competition in the field of evil mother f'erness.

apotheon
apotheon

Bringing down corrupt bureaucratic power structures is not the point of being ethical. Not being an evil mother f'er is the point of being ethical. Bringing down corrupt bureaucratic power structures is not the point of doing good work. The ability to be proud of your work, and to improve your skills and yourself, is the point of doing good work. The fact that quality work and ethical behavior can create competition by offering the market an alternative, siphoning away some of what supports those corrupt bureaucratic power structures -- maybe just a little, but every little bit counts -- is a nice side-effect, though.

apotheon
apotheon

Have you ever tried educating an ignorant client?

jkameleon
jkameleon

... are generally not eliminated by ethical behavior, and especially not by writing good software. The power structures, bureaucratic and otherwise, tend to stay in place, you know, and keep ethical human beings down. The best method of their elimination is illustrated in that video I've posted before. And remember, you are supposed to chase & beat the riot police, not the other way round.

jkameleon
jkameleon

If the customer wants tight deadlines the tight deadlines will be, together with everything that accompanies them- poor quality & maintainability, incomplete and/or false functionality, and so on. The customer is supposed to take that into account. If not- fie on him. Who doesn't know, that it's impossible to have all at once (deadlines, quality, price, reliability, and whatnot), doesn't have a clue about software development. Clueless customer probably won't notice most of the deficienciencies. Low quality software is therefore always the proper thing to do when deadlines are short.

apotheon
apotheon

It's in the best short-term interests of everyone except a rare few who are using the system as it stands for their own benefit at the expense of everyone else, and in the best long-term interests of everyone, to eliminate the corrupt bureaucratic power structures in place. It is in my own best interests (as well as those of essentially everybody else) to behave responsibly as ethical human beings.

apotheon
apotheon

I find it strange that you're so proud of your ability to perform poorly on a deadline.

jkameleon
jkameleon

Economics profession has market price, just like everything else. Economic systems it produces and markets are designed to direct wealth to the purchaser. > As for playing by the rules someone else made -- to the extent you refuse to even question the rules, you become part of the problem. When political power & money is at stake, rules are not to be questioned. They can only be changed employing this and similar methods: http://www.youtube.com/watch?&v=Uc000YDVY5o Looking from the less radical, market oriented angle, it's better to be part of the problem than part of the solution. Market presumes, that everyone acts out of self interest. By being part of the solution for idealistic, philanthropic motives, you are therefore distorting the market, and thus preventing the market from correcting itself in timely manner.

jkameleon
jkameleon

There is no deadline I couldn't out-Q&D.

apotheon
apotheon

When was the last time you faced a deadline that effectively mandates a quick and dirty "solution" that could be significantly undercut with ease, without compromising on the core requirements of the project? In my case . . . never. You either rush to meet the minimal requirements by deadline, or you turn in something complete that falls well short of minimal requirements because you gave up early. In large part, the growth of agile methodologies is an attempt to solve that problem, by allowing (apparent) quick and dirty to grow into something well-designed that meets all requirements. When you try to meet all requirements in a single development iteration, you end up having to choose between two options: minimally required functionality or under deadline. The two do not fit together, otherwise.

apotheon
apotheon

One of my biggest disappointments with Friedman -- in general, a brilliant economist -- has been his personal inability to recognize the incredibly simple fact that the institution of the corporation is not necessary to private enterprise. In fact, it is a limit on the freedom he espouses, because the corporation as an institution simply cannot exist without direct governmental interference in markets. Upon creating that institution, government has then created further interference in markets by proxy. If you realize that simple fact, the core failure of his point as expressed in that choral performance becomes clear; the amoral, impersonal corporation as a legal "person" is indeed amoral, but is not the best custodian of services such as education, even if Friedman's point about privatization being the way forward for freedom is valid and true (which it is, once divested of the language of corporatism). Sole proprietorships and partnerships -- not corporations -- are the ethical choice for privatization. As for playing by the rules someone else made -- to the extent you refuse to even question the rules, you become part of the problem.

jkameleon
jkameleon

But good ideas won't pay my bills. > That is precisely why I prefer to spend what limited time I have writing software of which I can be proud -- or, at least, that advances my skill rather than merely causes me to practice the skill of writing garbage far below my skill level, so that I can do a better job of writing software of which I can be proud in the future. Good luck competing with NOSOTEK guys.

apotheon
apotheon

You think that the fact the corporation makes up its own rules excuses what it asks you to do. I've said already that I can understand making a living while you're there, but that basically any self-respecting person should be looking to get a job where that is not necessary. If you've given up all hope of bettering those conditions, and further try to use such ethical bankruptcy to justify itself, there's nothing left for me to say to you about it.

apotheon
apotheon

You brought up half-assed work for half-assed pay, and now you say it's not "low pay". You said "quick and dirty", and now you're saying there's no (deadline) stress. I'll come back when you settle on a topic of discussion. I'm not going to try to stand on these shifting sands for a discussion with you any longer.

jkameleon
jkameleon

It's writing Q&D software and live.

jkameleon
jkameleon

... which doesn't matter a bit. From the legal point of view: If I'd write a good software job despite of being specifically told to write Q&D crap, it would amount to industrial sabotage. Classical example of such sabotage is Boyd's advice about dynamiting silk: http://www.iww.org/culture/library/sabotage/sabotage2.shtml From the moral point of view: If writing crappy software keeps my job, maximizes my salary, and increases my employer's profits, it's a good, moral thing to do- by today's moral standards. That's certainly not the way I was brought up, actually, I don't like it a bit, but I don't make the rules here. Mega corporations do.

jkameleon
jkameleon

... and not necessarily low-pay. Q&D is what business requires most of the time.

apotheon
apotheon

Of course one's ability to say "no" depends on economic circumstances. I'm not talking about what's possible -- I am talking about preferences. Given the option, even if it is not quite as strictly lucrative in the short term, I would rather say no than yes to a half-assed job for half-assed pay -- both because doing half-assed jobs rubs my work ethic the wrong way (among other problems I have with such an approach to writing code) and because taking less than I'm worth devalues my work in negotiations with potential future clients. One or the other of these things needs to be better than half-assed to make it worthwhile for any but the most desperate of circumstances, from my perspective. If you want high-stress, low-quality work from me, you may just have to pay me extra.

apotheon
apotheon

You not only try to justify planned obsolescence as a means of essentially holding customers hostage to your future work to save them from the damage you visit upon them; you then apply the same principles to their health, as if that somehow proves that you're doing a good thing by hurting people enough to make them come back for more.

apotheon
apotheon

Considering nobody, as far as I'm aware, has ever acquired enough skill and experience, and spent enough time, to write perfect software, but many people write quick and dirty garbage software every day, I do not see how you could reasonably believe that it takes more skill and experience to do the latter than the former.

apotheon
apotheon

> Chances of achieving anything remotely meaningful by writing programs are pretty slim. Chances are much better than in the vast majority of other possible career-related tasks out there, given that at this point writing software is the single most effective form of contributing to advancement of the state of the art of automation, automation is the core activity of technological advancement, and technological advancement is the key to solving darn near every problem humankind faces that does not involve people simply learning to be honest and kind. > That's the long-term stockholder's problem, not yours. You appear to have a very self-centered approach to determining what is, or is not, a good idea to do. > Employment in software development is everything but long-term, and even if it is- if you are employee, if don't own the company, the business isn't yours, profits aren't yours, and neither are problems. Obviously, if you're a rank-and-file employee, this entire discussion is moot -- unless and until you get a job that does not ask you to produce cheap garbage on a regular basis. It should be equally obvious that I am talking about cases where you get some say in the matter of whether or not you have any say in the kind of work you do right now. > Value is determined by the market. Yes and no. I was talking about a different kind of value than the purely economic sense of the term -- though the software quality I described as having a "complete lack of value" has significantly less economic value than something crafted with a bit more care, too. Anyway, I guess I should blame myself for this misunderstanding, since I was not clearer about the type of "value" to which I referred. I should know better. I meant intrinsic value, in terms of the ability of something to contribute to the long-term benefits of advancing technology and to productivity via rare investment in maximizing the benefits of automation, as contrasted with a throw-away effort that could be (and often is) duplicated by any schmuck with five months' experience toying with technologies he or she barely understands. When everyone and his kid sister are all building essentially the same exact piece of software, marketing may create a short-term market value that can keep you in Cheetos and Pepsi, but it contributes no intrinsic long-term value to anything; it's just spinning the wheels of technological status quo, wasting millions of man-hours on pointless duplication. It's even worse if you aren't even producing a decent quality example of exactly the same thing as everyone else. > I prefer not to do things Q&D either, but- in order to pay my bills, I have to. I certainly do not fault you for that. It is not the need to pay your bills that I fault. It is your apparent insistence that writing garbage software on command -- regardless of whether the job is undertaken to pay the bills or just to get a new $300 pair of sunglasses -- is somehow just as noble a pursuit as inventing a better privacy technology infrastructure framework to which I object. > Oh, yeah, the Tiger tank... It was by far the best in the WWII, nearly impenetrable, it outgunned every other thank. But, its design took too long, and its manufacturing was too demanding. Consequently there was not enough of them around to influence the outcome of the WWII. I'm aware of this. See my previous comment for my response to that fact. > When you design software, your time matters. As a matter of fact, it's usually one of the most important design parameters. That is precisely why I prefer to spend what limited time I have writing software of which I can be proud -- or, at least, that advances my skill rather than merely causes me to practice the skill of writing garbage far below my skill level, so that I can do a better job of writing software of which I can be proud in the future.

jkameleon
jkameleon

If you are paid to slap things together in a half-assed manner, you slap things together in a half-assed manner. As far as I can tell, job market for half-assed programmers is far better than for the purist ones. Slapping things together so, that they work just enough, and are still reasonably maintainable, requires far more experience than writing perfect software.

jkameleon
jkameleon

Sometimes you can afford it, more often not.

jkameleon
jkameleon

And continuous disaster is a good recipe for continuous employment. Basically, it's a sort of planned obsolescence. It's not unlike pharmaceutical industry. The drugs it produces have side effects which induce customers to buy more drugs (with the side effect of their own) to alleviate that side effects. Making side effects just enough adverse to avoid lawsuits and/or loss of customer base is quite challenging, as you can imagine. It requires close cooperation between business & marketing, and highly skilled professionals working in R&D.

jkameleon
jkameleon

> to achieve something more meaningful Chances of achieving anything remotely meaningful by writing programs are pretty slim. > regardless of the long-term damage to the company's valuation That's the long-term stockholder's problem, not yours. Employment in software development is everything but long-term, and even if it is- if you are employee, if don't own the company, the business isn't yours, profits aren't yours, and neither are problems. > or the complete lack of value of the produced technology. Value is determined by the market. > I guess the world needs people who take quick and dirty projects. I just don't want to be one of them. I prefer not to do things Q&D either, but- in order to pay my bills, I have to. People generally get paid for crappy things they'd rather wouldn't be doing. Things people like to do are done for hobby. Oh, yeah, the Tiger tank... It was by far the best in the WWII, nearly impenetrable, it outgunned every other thank. But, its design took too long, and its manufacturing was too demanding. Consequently there was not enough of them around to influence the outcome of the WWII. When you design software, your time matters. As a matter of fact, it's usually one of the most important design parameters.

apotheon
apotheon

Considering my current practical handicaps when it comes to software development skills, I should probably avoid projects that will instill bad habits.

apotheon
apotheon

Maybe you can explain to me how the tiger tank being made using a more "quick and dirty" style would have been better for the world. The truth of the matter is that the US won the war for a lot of reasons, and its tank technology was not central to that fact -- especially given that the US had the economic power to do much better without damaging its output in the least. > I always enquire about priorities beforehand, I alway ask "Do you want to have this done Q&D?" The answer is usually yes. As I said elsewhere, when that's the answer to that question, I usually choose to pursue a different project -- one that is meant to achieve something more meaningful than a single-quarter profit on the corporate balance sheet regardless of the long-term damage to the company's valuation or the complete lack of value of the produced technology. I guess the world needs people who take quick and dirty projects. I just don't want to be one of them.

apotheon
apotheon

I'd rather just say "no".

Sterling chip Camden
Sterling chip Camden

When you repeatedly allow yourself to slap things together in a half-assed manner, you become a half-assed programmer.

jkameleon
jkameleon

... you will realize, that making things Q&D is usually better for you, and sometimes even for the rest of the world. King tiger tank was perfect, and the best. Sherman tank was crapware. The USA won the WWII. I think I must emphasize that "sometimes" and "usually", because I'm not arguing for crapware here. I always enquire about priorities beforehand, I alway ask "Do you want to have this done Q&D?" The answer is usually yes.

apotheon
apotheon

Regardless of what bites me, I would much rather spend my time making something enduringly valuable than waste it throwing crap together so it can blow away in the first stiff wind, leaving me with a legacy of naught but squandered hours.

Sterling chip Camden
Sterling chip Camden

... to do even the smallest project half-assed. That will be precisely the one that lives forever.

apotheon
apotheon

I like to think beyond next week, and about the broader and deeper consequences of my actions.