Discussion on:

Why clean code is more important than efficient code

Does anyone (other than Jaqui) disagree with the message of this article?

Bloatware is a bane. It annoys the hell out of me to have to upgrade my hardware because the new version of the software I use requires more RAM. And non-efficient code is slow. The one thing that annoys users more than anything is having to wait for software to do something whilst the computer copes with the bloat.

I agree that code should be well documented. And if it is, then complexity and efficiency should not be a problem. It still boggles me that although the power of computers is far far greater than that of the early 1980's technology, software today slows the performance down to the same agonising die-of-old-age-before-it-does-anything level.

Getting it wrong, is much more irritating.
Doesn't matter how long it takes...

from Chad writing "it can't improve if it's not clear" to you reading "let's write bloatware from dusk till dawn"?
Bloatware isn't even on the chart, and to my understanding is often caused by lack of clarity on one level or another.
You also mistake optimized from efficient. Optimized means "tweaked for performance within a specific context", whereas "efficient" only means "lacking of unnecessary delay"... I may be wrong but I have come to understand that optimized code may (maybe even will) become inefficient if the context it was optimized for is no longer the case.

I know jack all about programming, but I know enough to think you got this all backwards.

> Bloatware is a bane. And non-efficient code is slow. I agree that code should be well documented. It still boggles me that although the power of computers is far far greater than that of the early 1980's technology, software today slows the performance down to the same agonising die-of-old-age-before-it-does-anything level.

I agree that this sucks, and I prefer to write small, efficient programs. This in no way contradicts the subject of the article.

Maybe you are confusing bloat with unoptimized code. Optimization does not reduce bloat; bloat is when too much crap is crammed into the program. The way to reduce bloat is not through optimization of the functionality included in the program.

The way to reduce bloat is to throw away features you do not need. Write programs that are small, elegant, and each do one thing well. When you need to do more than one thing, combine programs at runtime.

Probably the biggest runtime bloat factor is extra stuff happening within frequently looped or called code.

This should be a strong indicator of what needs to be optimized, if anything, and the problem gets worse when code is re-used in ways that increase the frequency it is invoked. This applies to the original poster's example, too.

Worse than bloat is exploitability, and this too is often an issue when code is re-used in unexpected ways that bring it to the surface. Writing source code for fast runtime execution rather than source code readability is one good way to increase exploitability risk; another is the use of lean languages that lack sanity-checking, such as C's "copy until nul" functions.

There is a gradient from edge-facing code that has to be protected against exploits, down to deep code that has to be fast and must never crash. Anything that brings the exploitable edge down to this critical code is a big problem, e.g. web browsers that allow web sites to (near-)directly call 3D driver code.

I expect to replace edge-facing code frequently, and I expect vendors to rapidly patch exploitable defects as soon as these are found. Ideally, I'd like vendors to code edge-facing software safely, so it isn't exploitable in the first place. All of that favors source code readability over runtime efficiency; a faster JavaScript interpreter doesn't interest me if every web site can throw junk at it, and it contains exploitable defects.

In contrast, bad driver code can wreck system usability and eat data, as well as bog down performance. So I want that code to be written efficiently, and do not exepct to change it unless there is a VERY compelling reason. The key to that is limiting the ways in which the code can be invoked, to reduce availability for exploitation, and simplify its connections to other code so errors can be avoided.

Where you need speed, you have to optimize, at the risk of source readability and errors. Where you are "close to the metal" e.g. device drivers, core OS code etc. the same applies, with the added constraint that errors become intolerable. To make that work, the code has to be kept trivial in complexity.

is feature bloat.
Functional bloat, e.g. writing unchanged data to a store, or triggering a calculation pass even though nothing's changed is a potential optimisation target, but like all optimsiation it's not free. Detecting that the data has been changed might be more expensive than just storing it anyway, for example, it will without doubt be far more complex.

The split between say driver code and perhaps the UI for a CRUD application is huge.
Driver code is almost certainly optimised, changing it after that is an iffy procedure at best, UI code fior a crud application, you expect to change almost all the time, so only a complete pratt would try to optimise it....

Agree whole heartedly on the last paragraph, to me the essence of programming is addressing complexity, far too many seem to view it as generating complexity though.

misunderstanding on every single point

Honestly, I don't know what causes bloatware, except that the companies that make it only make money from adding new features to existing software. So rather than reconsider the original design, they just add on to what they have, without consideration for design.

I wrote custom business apps. for several years, I shot for clear, maintainable code. I also shot for elegance, meaning simplicity of design--this was all contingent on if I had time for it in the budget/schedule--and my code wasn't bloated.

Your point that commercial software now runs just as slow or slower than software from 20-30 years ago is right on, but clear code does not correlate to bloatware. You're barking up the wrong tree.

Software gets slower faster than hardware gets faster.

It's because software vendors want to sell one package, yet their users have various demands. Thus, they pile all the features that are important to anyone into the one package. And they have to keep adding more, so they can sell upgrades. Microsoft Office is the poster child for this phenomenon, but it is by no means limited to that application or vendor.

The article is talking about writing routines and applications not an entire OS.
Clearly he says optimization matters but not allways and what he is stating is that what really matters everytime you write code is cleanness.
Wich i think is right, and you'll agree with me one never write all efficient code at once but as a result of a number of certain development cycles.
So you write it, you test it, you see where your program needs to improve performance and you enhance your code until you think your code performance cover your needs.

Personally i sometimes find myself spending precious time investigating wich way is the best to accomplish certain task just to acknowledge i will write it that way one hour ago. That is a way of premature optimization i'll have to abandon.

In fact, highly efficient code is often much more difficult to parse with the eyes. Duffy's remark about laziness is true, but I like your take on applying your energy to creating the right abstractions. Specifically, decoupling underlying implementation from exposed interfaces. If you get the interfaces right, then you can always optimize the implementation later.

Deferring optimization actually makes sense as a conscious engineering choice, in that it aids in grouping things together in a more clearly delineated manner, using smaller "parcels", ensuring that it's easier to experiment with different means of organizing the internals of a system until the final form of the system's architecture and API is nailed down. This assumes, of course, that you are not using a waterfall-style, BDUF approach to project management. If you are, well, that changes things -- like the likely quality of the end result (ugh).

Back in the day, I remember a developer who preferred a single, spaghettified, GOTO-ridden routine over a more structured approach simply because he considered the overhead of CALL/RETURN to be too great, both in performance and stack usage.

Sounds like a good person to know . . . by sight, so you can avoid him.

It's easy to poke fun at that approach, but that developer made a choice, even if it was unconscious: "When it comes time to change this code, it may take me (or someone else) longer to adjust it, resulting in late delivery, but that's not nearly as important as making code that will run as fast/light as possible. Better to deliver something late that works than something on-time that doesn't."

I don't think that's a priority that generalizes well, but I can certainly understand how someone might choose that route in certain circumstances.

... might be in a highly critical piece of real-time programming. In this case, we were writing accounting applications, which benefit much more from maintainability than from a few milliseconds of performance improvement.

That's the point of the article: make that choice last. Only sacrifice maintainability for performance if you must.

As any experienced developer will tell you, the most sophisticated developers create the greatest problems by trying to be too damn cute (efficient).
The dumb developers programs rarely seem to have problems, but the smarky ones always do.
The classic example is this :
If the Input is 1, make the Output 2.
If the Input is 2, make the Output 1.

The dumb developer will code as above.

The experienced developer may consider a table for future enhancements

The smarky developer will code :
Output = 3 - Input
which is efficient, but no one understands it and almost certain to cause future system problems.

There had better be at least an assert before it though...

Otherwise the peer review is going to be painful...

... that the only inputs will ever be 1 or 2. It would need at minimum a catchall for other values, and an explanatory comment.

and if done right explanatory as well.
if 1 & 2 are the only valid outputs they should be the only possible outputs.
there's a potential argument for
if input = 1 return 2
return 1
At least it's clear.

even return (input ==1) ? 2 : 1 reads better than the tricky math.

C is still the best (IMHO)

C definitely shines in some areas. Not in all.

return ((~input) && 0x3);

(logic instructions take fewer machine cycles)

( for you vBasic folks, that's return((NOT input) AND 3)... )

of course, it suffers from a lack of handling the cases other than 1 or 2

but it's "efficient", in terms of "fewest machine cycles to complete"

and it should make the case-and-point about "efficient but unintelligible"

If the point of the code is to return the complement of the two low-order bits, then this code is self-documenting. If that just happens to be a coincidence of the numbers involved, then it is obscure.

Shudder, never been a fan.

I mostly find myself using the ternary operator any time I'm tempted to use "unless/else".

I came up with my own class framework once for transferring data back and forth between a web page and a database in ASP.Net 1.1, and I found the ternary operator really helped make it less complicated. It was a situation that helped me figure out why some .Net and Java programmers preferred using code generators, because there was a lot of repetitive code involved. One of the things I did with the ternary operator involved value conversion. I'd write something like:

objFieldValue = (record["field"].IsNull()) ? 0 : record["field"];

A lot more efficient than having to type this kind of pattern 50 times as:

if (record["field"].IsNull())
{
objFieldValue = 0;
}
else
{
objFieldValue = record["field"];
}

The curly braces could've been left off in this case, but that potentially leads to logic errors if additional code needs to be added to a clause.

I probably would have just wrapped that in a method rather than write it fifty times. Is there something I missed in your explanation that explains why you had to do it fifty times?

clever b'stard C code, and don't have a lot of time for that sort of thing. Personally I don't see if else that hard to read, whereas because I use it so little the ternary operator trips me up.
Not using it is a habit I've developed, same as in C
I never use
int i = 0;
If (i)...
Always if (i==0), I put no value at all on terseness in compiled environent, and very little more in interpreted one.

An aesthetic objection, like where do you put your opening brace in a block as opposed to a technical one.

I was generally in the mood to avoid code repetition. This was 5 years ago. I was using ADO.Net (I probably didn't give that impression with the code I used. I forgot the exact method names and such. The example I used was pseudocode), I had DataSets, as I recall, so I was extracting out DataTables, then Rows from them, and then selecting out fields. So the issue was I was using Microsoft's framework. I think to really have made this better, I would've needed to subclass off of Data Set and DataTable, and produced my own custom Row type out of that, which would've required internal access to Microsoft's ADO.Net data structures. There was a tool at the time (probably "Reflector"), now that I think about it, which I guess I could've used to decompile Microsoft's code, but there's no guarantee I would've been able to pull this off even with that, given that they could've closed off access to their data structure to subclasses. With a modern version of .Net this would be more doable, because I could just add some class extension methods, and (probably) have access to the internal data structures (though that's still a question in my mind). I didn't think of doing this at the time, as I remember I was already behind schedule with that project. It took a while to write the code I did, but it didn't take a lot of learning time to pull off. As I recall, I did some cutting and pasting of code (which I would then go back and revise for specific field names and such) to help save time.

Like I said, this gave me perspective about why some developers preferred to work in a meta-language, which would generate C# code, or what have you.

The answer is, yes, I think there *might* have been a better way to pull this off. The issue was time.

That explains it, they are a near guarantee of a big f'ing mess as soon as you want to do anything useful.

@Tony:

Yeah. In one of my other .Net 1.1 projects I eschewed datasets altogether, and just wrote my own data access code, which looked more like JDBC code, still using ADO.Net, but I maintained more control. It took longer, I think, but I was more satisfied with the results.

(which is only when I can't avoid it) and that's to a class we've written.

Never use that horrible schema crap, it's about as clean as incontinent tramp's underwear.

I can still appreciate the problem.
Sounds like a brutal solution, like using a firearm like a club... but sometimes that's all a person has time for, and if it keeps the lion from eating you, that's great!
Not as elegant and Alex and the Gordian Knot, but then, maybe he did in fact hack at it fifty times before he got through... legends take liberties with that sort of thing.

... is functional vs imperative. In a language like C, if you compute something in an if/else, then in order to pass the result into another function, you must (a) call the function more than once, (b) assign the value to a variable, or (c) create a new function that contains the if/else and returns the appropriate value. The ternary operator can be embedded in a function argument.

Of course, in languages like Ruby, all forms of "if" can return a value, so in those languages the functional aspect is the same.

If I wasn't straight I would propose.

There's no process you might want to do a piece code that isn't aided by it being clean, including messing it up....

Once you've optimised a piece of code, which certainly doesn't necessarily mean you have to make it messy, there is very little else it will do, you've optimised for a task, quite possibly for a task in a particular environment. The most regular mistake isn't premature optimisation, it's reuse abuse. That's when you take your super efficient code and get it to do something else as well. You can't have one piece of code that will do two things efficiently, any attempt to do so will make it inefficient AND messy.

Do a clean version, write the tests, prefereably the other way round, if you are fortunate enough.
Then optimise with your unit tests to prove you haven't broken it, any other approach leaves you with a highly efficient collection of bugs.
Keep your unoptimised version, then when you want to optimise something slightly different or even for something different, that's your starting point.

Optimisation is a lossy process, you lose flexibility, and seeing as in our industry change is a given....

Clean code is efficient, it's optimised for developing.

... then it may be possible to efficiently reuse parts of the internals. I agree with you, too often people try to reuse a monolithic piece of code by passing in a new flag or something. If the functionality of that thingy is divided into components that each do one thing well, then the way to reuse those components will be obvious and will not require them to be changed at all.

and then when I could reuse that code then I'll look at making it external, generally that is not simply a matter of just changing the access modifier.

It's the people who make the 'internals' public just in case they might want to reuse them that drive me barking mad. That's a lot of scaffolding and extra work to do it properly and seeing as you don't really know what it could be reused for you are working blind, waste of effort and an increase in risk. If you are definitely going to reuse it, then at least you how much of it you need to expose and can deal with the consequences of doing so.

The 3 -Input case, if it's internal and is only called with 1 or 2 then not brill but okay. If you then think I nedd that somewhere else and make it a public method without validating the input, you are some sort of idiot.
An experienced developer would validate the input just in case six months later when they did need it agin, even internally they 'forgot' and ended up looking like an idiot.

... is only wasted effort if your language punishes you for doing so. In a language like Java/C# where that means creating a whole class with public vs private members, then I see your point. In a language like Ruby or Haskell where "API" can just mean well-defined functions, it's not much effort for a big potential gain.

This brings to mind a couple things I've read. One is Richard Gabriel's article The Art of Lisp and Writing , specifically where he talks about the practice of coding something up and then revising it. He equates this to the process of writing (in English). I extended his ideas into my own world, writing my own blog post , quoting his article, if anyone's interested. This was my own thinking about current software practice and what needs to change about it.

What I realized as I was reading through this material again is that your example of a routine, and how you reasoned about it, has implications for whole programs/systems, and thereby, projects.

Here are what I thought were the relevant parts that Gabriel talked about:

Great writing is never accomplished through planning followed by implementation in words, because the nature of the word choices, phrasings, sentence structures, paragraph structures, and narrative structures interact in ways that depend [on] each other, and the only possible plan that can be made with which a writer can reason about the piece is the piece itself.

...

[T]he requirements for a system come not only from the outside in the form of descriptions of behavior useful for the people using it, but also from within the system as it has been constructed, from the interactions of its parts and the interactions of its parts separately with the outside world. That is, requirements emerge from the constructed system which can affect how the system is put together and also what the system does. Furthermore, once a system is working and becomes observable, it becomes a trigger for subsequent improvement.
I think this bolsters your point, that clarity of code is important, because you make it more difficult to reason about your code if it's already optimized. He makes a larger point about our dev. tools, that they've promoted premature optimization of design in order to get optimized executables, but this has made software/project development more difficult to reason about.

Interestingly, Gabriel used the same quote that Knuth used, though Gabriel says it comes from C. A. R. Hoare (and has been misattributed to Knuth): "Premature optimization is the root of all evil in programming."

Your article also brought to mind something Alan Kay told me about coming up with new architectures:

[W]e need to ask about the architecture of the proposed model . . . before worrying about details. Some of these questions are algebraic: i.e. are there abstractions that could simplify the number of concepts needed to span the model? It seems there are: both structural and behavioral. Can we separate "meaning" from "optimization"? Yes, in many if not most parts of the system.
An idea that seems to have stuck with him for a long time is that code should represent "goals," rather than commands, and it's up to the underlying system or dev. environment to formulate implementations of those goals. The idea being that you can have clarity, and optimized implementation, but to have that, you have to have a compiler or runtime that doesn't take your code at face value. This is "pie in the sky" for me at this point. I don't know what it means yet, but I felt it had relevance to the subject.

> Interestingly, Gabriel used the same quote that Knuth used, though Gabriel says it comes from C. A. R. Hoare (and has been misattributed to Knuth): "Premature optimization is the root of all evil in programming." An idea that seems to have stuck with him for a long time is that code should represent "goals," rather than commands

That seems to match up nicely with something else Alan Kay said -- that he wished he'd called it "message oriented programming" instead of "object oriented programming", because now everybody's obsessed with objects when it's the messages he thought were important. Messages, if you think about it, are requests for goal-fulfillment. Objects are the things that fulfill the goals for you, by executing their methods in response to your messages. As such, it seems the way to handle things (according to his goal-based design) is to figure out how to organize your code so that requests for goal-fulfillment are the central factor in your design.

You're right that he's regretted the term "object-oriented programming," and wishes that he'd come up with something better. I think he'd mentioned hearing the suggestion of "message-oriented," but I haven't heard him settle definitively on a term that he felt would accurately convey his ideas about it. I agree with "message-oriented," because I think to today's developers it gets the idea across pretty well. He's still possessive of the term "object-oriented," even though just about everyone else has some very different ideas about it than he does, and it seems like he sees it as representing his legacy. He sees himself as being known for it, and perhaps it's not so easy to change that.

What you've said about messages being goals I think is one way of interpreting the idea. In the same conversation I quoted from above, he also said this:

[A] metaphor that I used in the 70s was that regular programming was a single chef (a very high bandwidth chef) zipping around fashioning with ingredients, and object oriented programming was negotiating with agents -- that agents were ambassadors, etc. (I got talked out of trying this metaphor in my 1977 Scientific American article by the editor Dennis Flanagan -- maybe this was good.)

But the deepest analogy was one of the earliest for me: biological organisms made from cells - and they definitely do negotiate.

Also, I would characterize the ARPA (and then the concentration at PARC) as mainly interested in a "no-centers" style scaling architecture (personal computers, Internet, Ethernet, objects, "no OS", "no Apps", etc.).

So a lot of the interest was how to get things done without having to concentrate the knowledge in one or a few places (because this would require these places to have to control "ingredients").

Based on this, "goals" seemed more like, "I want to get to X," or, "I'd like to accomplish X," but whether the object would actually do that for me is another matter. So the message represents my goal, but that doesn't mean that the goal will be fulfilled. I'm not saying that any current OO system should necessarily operate this way. I'm just trying to bring ideas together in a way that makes sense.

Jumping off what he talks about with "no centers," he's said in the past that the internet, the physical infrastructure and TCP/IP, is a good example of an object-oriented system. I think it could be argued that it's a better OO system design than Smalltalk. The thing is, this way of doing things is hidden from most people. The internet experience for most is not that different from interacting with a mainframe.

Re. the quote about premature optimization

In the book "The Dream Machine," one of the things Waldrop mentioned is that when Licklider and his buddies would get together in what I'd call "jam sessions" (I think in the 1940s) where he and his colleagues would swap and discuss ideas, they'd come away with some great new ideas, but nobody knew who came up with them, because it was all a synthesis of what each person contributed. There have been some quotes I've attributed to Alan Kay in the past that now I wonder about, because, for example, one I've used I've also seen attributed to Dan Ingalls. I know they've both said them. They may have come up with these ideas amongst each other, and so attribution is problematic.