IT Employment

Sloppy code: Why it's not (always) the developer's fault

With programmers complaining that managers are forcing them to push unsatisfactory code out the door, TechRepublic looks at whether developers are getting a raw deal.

Does this conversation sound depressingly familiar?

Programmer: "There's a security hole in this code, can I fix it?"

Project manager: "No, we don't have enough time, just ship it anyway".

This type of exchange embodies what some readers feel is wider problem in software development - that programmers are not given time to write the code they feel is up to scratch.

A repeated complaint during a recent TechRepublic discussion about security holes in code was that the main concern of managers is to get code out of the door, and nothing short of a show-stopper of a problem will hold back its release.

The reality of software development is that commercial pressures will often drive suppliers to favour rapid delivery above other considerations, according to Michael Azoff, principal analyst for software and IT solutions at Ovum.

"There is pressure on developers as often in a customer-supplier relationship penalties kick-in if something isn't supplied by a certain date," he said.

"In those instances it's easier for the supplier to deliver and then fix known issues later, which had they more time they would have dealt with. From the developers' point of view it's unsatisfactory, but that is the reality out there."

And as another TR reader pointed out, when "big profits are made out small cost savings" there can be an incentive for organisations not to fix poor code.

Given that companies supplying software to large organisations often don't have the incentive to stop sloppy code being rushed out the door, Azoff said it is up to the customer to fix these issues. Bringing in a third party test the supplier's code will deal with many of these problems, he said.

"The more mature a customer is and more educated they are about these kind of issues, the more likely they will bring in these testing services to deal with these problems," he said, adding that sensible customers will also have in-house staff who manage the quality assurance of the software and the external testing service.

Could agile save the day?

Changes in the way that software is developed are also reducing the tendency to ship unsatisfactory code, said Azoff.

As agile software development methodologies become more popular, and the use of waterfall methodologies decrease, Azoff said that the pressure on developers to rush out a sub-par release should also fall.

"Waterfall tended to deal with testing and QA towards the end of schedule, and that's when things get very pressured and where shortcuts are taken. That's been a long term problem with waterfall.

"The reason why agile has increased adoption and has displaced waterfall is that agile deals with testing and quality much better."

As a result of agile allowing developers to do more unit testing and continuous checks and integration "the quality of the product is improved while it is being developed", he said.

Don't forget the bigger picture

Even though developers may feel as if managers are forcing unnecessary compromises, it may be the right decision for the business.

"Good engineers focus on engineering and sometimes lack that bigger picture to look at the business - [to realise] that if you don't ship this then we're going bust," said Andrew Clymer, a founder of software consultancy, training and development specialist Rock Solid Knowledge.

"If you're going out of the door with a security hole, depending on what you're exposing, that's a risk I guess the PM feels the organisation should take, and they have access to the bigger picture.

"At the same time I'm sure there are times when the PM doesn't see the deep engineering side of it."

Software can never be perfect or 100 per cent secure, and Clymer said that ultimately the decision on when to push the code out of the door needs to be taken by someone who can appreciate the wider commercial context.

"You're always under pressure to ship and PMs always want it ready to go, I'd say it's a necessary evil."

About

Nick Heath is chief reporter for TechRepublic UK. He writes about the technology that IT-decision makers need to know about, and the latest happenings in the European tech scene.

82 comments
MyopicOne
MyopicOne

...so a general "well done" to the commentors. I'll just add that I know of a system undergoing a much needed major revision that shipped with three show-stopper and four critical bugs by their own quality group's admission, but no heads rolled...

krismeister
krismeister

This article reminds me of a talk by seth godin. http://vimeo.com/5895898 The idea is that when you run out of time or you run out of money you ship. Seth Godin worked at Microsoft on many development projects. If you don't know him you should.

Jaqui
Jaqui

That puts the lie to most of the reasons for pushing sloppy code out. openBSD, forked from freeBSD with 1 focus, secure code, always. in the 14+ years it has been around, they have had 4 security issues in the default install / config. no other project even comes close to THAT record.

NJKotze
NJKotze

Stop paying testers peanuts and start investing in them. As a software tester I have never experienced anything different than what stated in the article. This is the consumer market. The problem tends to start with the users who are willing to pay for more features that is unstable. That drives the client to push the dev house with unrealistic expectations. No software development life cycle trick will work to get you out of this. Many dev houses neither have the balls to tell their client that that it is not how software is developed. Chances are pretty good that if you have a tester in the team he was told to shut up because he makes the dev house sound bad. You will never get around the budget and schedule problem unless you have a client who is willing to negotiate. Then start doing reviews of the requirements WITH the client so their eyes and ears start bleeding by the questions raised from poor requirements. Get your tester in there to ask some more questions because if they have the experience they will follow the simple rule "If it's not testable we don't develop it". Doing reviews with the client should also invite contractual re-negotiations since specifications are cleared up. Then developers and architects need to learn to develop testable code. No time to develop testable code is the general excuse from developers. Then, please for the love of sleep and weekends off, please use tools that work together and not "oh we can hack it to integrate it". So many companies invest in dev tools but testing have to use substandard open source tools with no developer to help integrate the systems. Pull the client in for testing and have a decent change control process and perform impact analysis. This article is an excuse to poor discipline from all parties involved. Oh, managers and marketing must stop making terminology and buzz words up as they go along. There are terms used in the industry so that everyone can understand what is talked about. Feels good to get that off my chest. thanks guys :D

deneventer
deneventer

What is the point of BETA versions? Also, why is it that release version 20 million still has the same nagging problems of the original?

GregoryLHart
GregoryLHart

Once a defect leaves your desk, the cost to detect, identify, and resolve that defect increases exponentially. The best solution technically and economically is to deliver the highest quality product.

Dr_Zinj
Dr_Zinj

And it's usually the managers, sales and marketing, that royally screwed up in not allowing for the contract to have enough time to ferret out and fix security holes. "You don't see the big picture" is ALWAYS their lame excuse for their own incompetence.

arthomas152
arthomas152

Remember the Y2K bug brouhaha? One company had a great client. As I was writing new modules on a project, I noticed there were vulnerabilities that needed to be fixed. I changed the date on my PC and the code froze. When I reset the date, the code worked fine. I asked for permission to change the flaws and estimated it would take about a day and a half. “Nope! They didn’t want to pay for it.” I cleaned the code anyway and alerted that client manager to have all his projects rechecked. No one had mentioned to him there was a problem. He would have funded the changes if he had known. Most people don’t want shoddy software and even fewer want to be deceived. Whether that deception is committed deliberately or just by omission doesn’t matter. Some managers need to study ethics and then go to a hospital and see if a spine and ??? can be implanted. I had too much experience correcting code that was written without an awareness that the code might need to be modified when circumstances changed. Cute names for variables such as “Spock” “Kirk” “Jim,” “Bones,” and “Sanguineous,” variables that were not strongly typed and would pass whatever ??? was sent to it, impenetrable code, and code with no comments were not unusual. The original managers paid no attention or just didn’t care. I know there are people on both sides of a project who do not want to hear an unpleasant truth. I took pride in the work I did and the people I supervised. Our reputations depended on the quality of the work we did. So did our honor and our integrity. We were hired to do a job to the best of our abilities. Surprise! Some of us insist on doing our best.

ProjectJourneyman
ProjectJourneyman

Often biggest problems occur when the decision-makers don't take responsibility for their decisions. When an organization wants employees to be "empowered" to take responsibility for their work, but not empowered enough to make decisions about the work, you have a rift. I agree that agile or other communication-intensive methodologies can help bridge the gap between management understanding and management decision-making. The more management understands the consequences of decisions, the more those decisions will be informed. In the end, though, it is an integrity issue - if management doesn't care and nobody holds them responsible for their decisions, the organization is broken and no developer can fix it.

dougswamp
dougswamp

My approach was always get a ruff version running and JAM the H*LL out of it. Design bedamned. When things become a problem, do a flowchart of that section. Below is the flowchart for the line-wrap / hi-lite logic of my search engine. http://www.telusplanet.net/public/stonedan/pict01.jpg http://www.telusplanet.net/public/stonedan/pict02.jpg http://www.telusplanet.net/public/stonedan/source.txt The above code is sloppy by intent. The perfect perfects hate it. Little do I care. It is the only program I need EVER. It can randomly select a video then pick a random start point and play for a few seconds then the next short video clip. When you have lots of video like I do (2549 on YouTube) random is critical to stay in touch with what you have. Perfect code pisses off the coders and seldom matters; plus it takes way more time... Boo to the Lord-it-over-yous and their coding standards. My hope; is that the sloppy code gives some of you too-goodies a heart attack.

RMSx32767
RMSx32767

It's sometimes incomplete data validation and associated error trapping. Some years ago I was asked to debug a crashed program. The crash was caused by the string NOPHONE in a telephone number field. The original programmer had assumed a phone number would always be numeric.

flood_specialist
flood_specialist

There is already a movement to make software companies liable for their product. I agree. No more open ended EULA's. You make it, you sell it, you're liable. Simple to me. We hold many of our product manufacturers liable for this and that. So much of our life now is around software doing it's job, metro rail services, air traffic, automobile systems, and medical systems. For the sake of profit is crap. You knew the mission was dangerous when you accepted it. When people quit bending under the pressure and or promising something just to get the job, stop!!!

Tony Hopkinson
Tony Hopkinson

Would you describe Mr Godin as famous, or infamous?

Deadly Ernest
Deadly Ernest

ship the product before it's ready because the powers that be screwed up the estimates of resources needed as well as mismanaging the project.

Tony Hopkinson
Tony Hopkinson

and will pay for it even if it is unstable, badly documented and buggy, whether you've used your marketing skills to 'erm convince to do so or not. It wouldn't be business smart to argue them into not wanting it, would it? Commercial Software in a nutshell, by Tony.

Deadly Ernest
Deadly Ernest

but I wonder WHO is pushing that, the consumers or the marketing people? I often see people in the shops buying portable consumer items asking if the shop has something that a lot more basic as they don't want or need all those other things. One I hear a lot is I don't need Internet on my phone at all, I got a tablet for that, the phone screen is too small to be of any use reading emails or web sites - yet almost every phone has more and more Intern capability for the same miniature screen.

Slayer_
Slayer_

Betas are also often used to generate hype about a new program or product.

Deadly Ernest
Deadly Ernest

1. beta versions are for real time testing to try and identify issues that occur in production that the people who wrote the specs and test sheets did not think would happen . 2. Nagging problems often continue because the people with the decision power decide the cost of fixing the problem properly is too much, or they have a hidden agenda that the problem promotes, or the cause is so well hidden down in the badly written code it can't be found. I'm sure there are others, but they're what spring to mind right now.

Tony Hopkinson
Tony Hopkinson

that there is a salable and marketable product. Are you new? :p

cece1012003
cece1012003

Managers employ developers to deliver a high quality product, but almost never heed the advice about making it an actual quality product. There's hardly enough time for developing code, adequate testing, cleaning and review of the code half of the time, because the people in charge don't pay you to explain this to them, only deliver the best product ever presented in no time flat. Never mind that technical specs and deliverables of the project change all the time and even overnight. I understand that businesses must be the best in the competition, and they have to keep costs down as well. However, if the planning and risk assessment is done correctly, it's possible to bring out a quality product without it having plenty of security flaws, produced at a reasonable cost. To be honest, I feel it's incredibly irresponsible for software to be purposefully launched with security flaws in it to beat the competition. Depending of what kind of application it is, it can endanger people (if it was safety-critical) or it leaves users vulnerable to their systems being compromised, i.e. financial info/passwords. I've learned that the first product is not the preferred product every time. If there was an application that was better than the competition (not necessarily first on the market), I'd buy it.

Tony Hopkinson
Tony Hopkinson

We wouldn't be having this discussion, or the one that kicked it off. In fact we probably wouldn't have TR. Hastens to add, you are correct about the cost of detect, identify and resolve, but there are two HUGE assumptions in your statement. It's going to be fixed. It's going to be fixed correctly. Even if all "sloppy" coding no matter who was at fault was resolved. If all specs were right, all tests plans were right and comprehensive and used, technical debt which is an inescapable reality of producing commercial software guarantee the result would be percieved as sloppy anyway, so why bother... :(

Deadly Ernest
Deadly Ernest

photo of money flowing into their private account from commissions

Tony Hopkinson
Tony Hopkinson

Developers of commercial software do not have the right to do something management say they don't want. Doesn't matter whether it's sticking a backdoor in, a logic bomb, an easter egg, and extra piece of functionality, a bit of elbow room for potential future functionality or fixing a known security flaw. If you don't want to do what they want, leave. It's their product, it's their money, integrity is respecting that.

Tony Hopkinson
Tony Hopkinson

In fact perfect code is less maintainable than sloppy code, after all how can you "improve" perfect. You need to work on your message, you aren't doing yourself any favours with it.

Tony Hopkinson
Tony Hopkinson

The programmer assumed no one would enter a non-numeric phone number, that's called developer testing, been about for ages. PS nophone is 6674663 :D

Tony Hopkinson
Tony Hopkinson

Boss, I'm not doing what you said, because of insert technical mumbo jumbo here. Right new boy, your predecessor struggled with this, but I'm sure you have enough incentive (read a job) to meet the "customer" need. WTFU

Deadly Ernest
Deadly Ernest

sloppy code gets released into the wild because; 1. the person writing it doesn't know better; sure it's rare today but still happens with some people just out of training; 2. management comes along and walks off with a copy before you fix the problem; 3. the problem isn't spotted straight away due to a team being involved and a change by one affects earlier work; 4. the specs and the test requirements are well thought out or badly written and the coders don't suggest the best way to do it; 5. the new code is to be stacked on legacy code and you get no chance to properly test the two together; 6. sloppy or flawed code is done deliberately by the coder or on management orders due to hidden agendas. Those are just a few of the situations I've seen in real life, and I'm sure there are others. Sadly, today the people responsible are very rarely held to task for the results

Deadly Ernest
Deadly Ernest

it was reinforced that when you established a data input field you also included the code to ensure the data was checked for validity before it was used for anything else - I was told that's the professional way to do it right.

Charles Bundy
Charles Bundy

[i]All we've got is overdrafts and crap software... [/i] 'cause consumers like 99 cent apps and free checking! :)

Tony Hopkinson
Tony Hopkinson

We me and you, not mega corp.... May be if you read what I posted instead of what you wanted me to post so you could reply to it and show how wrong I wasn't, you'd make more sense. You are getting f'ing irritating.

Deadly Ernest
Deadly Ernest

if a company's management can see a financial advantage from crappy code, in most cases today they'll do it. Microsoft has been pushing their Palladium agenda for almost 20 years, and have been sneaking it in by stealth a little at a time - and each new bit they add is done under the heading of 'stronger security.' Now I've no evidence they're doing it, but there is a correlation between in

Tony Hopkinson
Tony Hopkinson

If business believed that it was better economically not to put bugs in sofware, there would be less of them. There aren't, so they don't and they have impressive bank balances to prove it. All we've got is overdrafts and crap software...

Tony Hopkinson
Tony Hopkinson

to pay attention. You don't test assumptions,you question them. Your position seems to be if I meet your assumptions then the software meets your requirements... Breakout a dictionary you aren't going to see a reference from one word to the other... By the way I was a total failure in the military, I questioned the asumption that those further up the food chain were giving me orders I should follow... After dealing with NCOs, pointy heads don't bother me in the slightest.

Deadly Ernest
Deadly Ernest

strictly obeying orders on the testing, even when you know they're wrong or insufficient. That's the same as doing bad or no testing as it isn't a proper test. Also, blindly following orders is expected in the military but wasn't accepted at Nurremberg either. I'll accept that the people ordering the poor or bad testing bear the most responsibility for it, but also if any of the development team spot the problem, and they should, I believe they have a duty to raise their concerns with the appropriate people and state why - failure to do so makes them culpable as well. we may have to agree to disagree on this one

Tony Hopkinson
Tony Hopkinson

is the root cause of 99% of the problems. Your contention that the name field should be searchable is an assumption. I wouldn't assume that on implementing a design. Making a field efficiently searchable is a cost. If I'd been doing the analysis I'd want to make sure that you were not expecting it. As a dev If it wasn't mentioned I'd only ask for clarification if some other requirement depended on it being so, and that would only be to allocate extra resource to implement it, or that the dependent feature be dropped. If there was no inconsistency, I wouldn't do one jot of extra work to make your assumption valid. If I did I wouldn't be meeting your stated requirements, I'd be meeting MY assumptions... I'm not testing my assumptions, I'm not testing your assumptions, I'm testing against the requirement. So what do you as a customer need to do? Same thing as I. Don't make an ass out of u and me...

Tony Hopkinson
Tony Hopkinson

and definitely the customer. It was the shop floor boys doing the data entry who 'erm spotted the difficulty, wouldn't be the first time the customer stakeholder, didn't know the job to that level of detail would it? At that point the classic defence, "it meets the spec" is highly effective, except of course your customer still thinks you are a set of overpriced wankers.

Deadly Ernest
Deadly Ernest

Yes, it is an assumption only a numeric data input is needed, however, there would be other information on hand about the relevance of that assumption. First, the majority of data input fields usually have an example beside them, second the location where the program is likely being used will also add information - all phone numbers in Australia are numeric only now, and I think that's the case for the USA too - fifty years ago they were alpha numeric, but all numeric now. Thus you would be able to make a lot of very valid assumptions from the info available, despite not having a proper set of tests. Sure, it would be better to conduct tests based on properly defined specs; but it's better to do tests based on assumed specs that you document what's being done and why, than not to do any tests what so ever. Today many databases programs have built-in data validation that kicks in based on the type of field declaration, but not all do, and sometimes you want more than what they offer. When I was first taught programming and developing applications there was no built-in data validation, so we were taught to first take the input data and run it through a data validation routine prior to passing it into the program for further action. Not all that hard to do, but something you had to remember to do and do right. Often with names it was an automatic process to have all the characters converted to the same case so they would always be handled the same way later and were easier to match the strings properly. Testing for this sort of thing should always be a matter of course, and it used to be, but isn't any more. Today we get caught between the devil of 'only do what we tell you to do' and the deep blue sea of 'but my personal ethics requires I do it right.' Many years ago I got faced with a major case where I could see that who ever prepared the specs had no idea of what they were talking about and management weren't prepared to accept the job as written was undeliverable due to inbuilt errors. When ordered to do the job that way or else, I handed in my resignation and took the or else. I soon found other work in related fields in the same organisation. The people who refused to listen to me were terminated within weeks of the finished garbage being presented to the client and proven to not work as expected. The project resulted in the client and terminating some of the people as well, for writing the specs the way they did. I know in these situations some people will deliver the garbage and some people will deliver the resignation, and some will fight like hell to get the specs rewritten the right way. Which ever way a person goes will be their own choice; but those who insist the garbage be done and sold should be held responsible for it's production - that's my thoughts on it. Below is some of the sort of issues that can give data validation checking staff major headaches: I spend a LOT of time conducting database searches while helping people with their family history research now, and one of the most annoying things is when you find a database that will NOT allow any form of wildcard search, thus a search on ROBERTSON will not even look at ROBERTSONS etc, to make it worse, some others do soundex searches thus a search for PEARL will include PEERL in the answer as they sound similar - and until you've played with the database you don't know which way it goes. But what's most annoying is those that do an exact match no matter what, thus Delray is not matched with DelRay and older style spelling of the same name - further aggravated by people making typos when inputting the data.

Slayer_
Slayer_

It was defined as a number. So you code the box to only accept a number. The testers can only verify that it only accepts a number. It goes to a client and they complain cause they want to enter text.

Tony Hopkinson
Tony Hopkinson

All you are doing there is swapping one set of assumptions for another. The guy who kicked this thread off either omitted key information, leaving us assuming, or has passed on their own assumption. Should the telephone number box only allow numbers or should whatever expects only numbers be changed. There's absolutely nothing in the code as we've been presented it, to say which bit is wrong Now I'll agree that comprehensive brute force testing (inefficient and unreliable) would find the inconsistency, but any fix without disambiguating the specification with the stake holders would be a guess based of the fixer's assumption. That's removing the inconsistency, worse still from your point of view about quality software, which end would be fixed? Least effort on the vendor's part, obviously....

Deadly Ernest
Deadly Ernest

for most software you can usually work out what is or isn't acceptable data for the input fields from their field type and / or their field name. Heck, even how they display ion the informs will often tell you a lot about what should be in there. However, you go about the application does need to be tested or you have no way of showing it's completed and you can be paid. The worst problem I've ever seen in a test environment was a tester was in a hurry to pass a program so they only tested it with the info provided by the client, all of it good data that had everything right, so of course it passed - the program concerned was returned ten days later as having failed client field tests because they sat some of the data entry clerks in front of it and said to enter data, as soon as a typo happened the whole thing crashed. In the rush to get it out the door people involved did not incorporate any data input validation processes. Needless to say that team were unemployed by the end of the week, while some of us were busy fixing the code. The program was good apart from the few routines for data validation prior to using it. That's a classic example of not proper testing - any software test should include what happens when the data is invalid.

Tony Hopkinson
Tony Hopkinson

Before you can test, you need definitions or what right and wrong are. Telephone number is or is not numeric is an assumption. A test of an assumption, proves or disproves the assumption not the code.

Deadly Ernest
Deadly Ernest

the wrong type of information entered, for what it does with insufficient information entered, for having the right information entered - to see what it does with it. It should also be tested to see that it performs as expected and as intended and as wanted - they aren't always the same by the end of the project.

Tony Hopkinson
Tony Hopkinson

The phone number issue is classic. Put a number in the box and it worked, job done, on y va. It is what it is. :(

Deadly Ernest
Deadly Ernest

of the reasons I moved away from working on code was it was getting harder and harder to get people to provide specs at all, let alone decent specs. Part of the reason for that is if the specs are well done, they have to work a lot harder to move the goal posts on you and introduce changes. I got fed up of having the specs sound more and more like airy fairy wish lists than a spec sheet, so walked away to avoid having my ethics compromised by being forced to produce crap.

Slayer_
Slayer_

Cause the spec would say to test a numeric field.

Slayer_
Slayer_

I didn't have access to change the database, so instead I changed the code to convert the entry to Hex before saving to the database.

Tony Hopkinson
Tony Hopkinson

It's why we invented testers... :D And analysts, you say number to a dev, he's already made a whole range of unconscious assumptions Now if it was going to an int field in the db, not masking the entry field for numbers, would be a poor effort. If it was string in the db, then something went wrong somewhere, because it having non-numeric data in it shouldn't have mattered. In fact that would mean the assumption that number was numeric had been foolishlt made in more than the data entry. This is one of those deceptively simple ones we've all fell over at least once. Now when someone says it's number, I start checking definitions, not often number = integer. Even if it does in the uderlying data, I've seen outputs called numberofClients that were populated with "You have 1 client" as such. It's an easy mistake to make. No documentation, no unit tests and no time, it will get made, I guarantee it. last contarct I had I was given aspect where Invoice Number was defined as an int I built the entire app inclusing database based on the at spec provided to me by the IT manager no less. Bug 1 I can't get the invoice number I/198767 into the box...