Apps

Thoughts on Shortening the Release Cycle


I have been spending a lot of time lately talking to Scott Abel from Spiceworks. One of the things that I keep hearing from him and his team is how quickly they are able to add new features to their product. Long term readers with sharp memories will remember that I am against short release cycles. I still am, too. But for some products, particularly Web based ones, can benefit from a release cycle oriented more towards individual features and less about entire versions.

A Web application is typically a collection of miniature applications which work together coherently. Why wait six months to upgrade one part of the site just because you want to upgrade everything at once? It does not make sense to me either.

On the other hand, products that are sold to customers as a comprehensive package really cannot work this way. Why not? Because the customer has paid for a particular feature set. What is the vendor supposed to do? Sell a feature set that is not finished yet, and tell the company that the product will “eventually

About

Justin James is the Lead Architect for Conigent.

34 comments
butterpot
butterpot

Some customers are more clear on what they want and are good at documenting this. Sometimes this is not the case. It would help to give them "foretastes" of the final product just to ensure that both parties are in sync with each other. This beats springing a surprise on them when it is too late to take any action.

Deadly Ernest
Deadly Ernest

Design the main program to provide all the basic tools and things, and allow for the easy addition of new bits, like browser plug ins. Then provide the new features as individual customising plug ins at a low price each. Client buys what they want and adds them to the main package. Much the same you do a car and its factory or dealer fitted accessories. Much the same way as you put the main kernel for Linux on your system and then add the desktop and applications you want. Sure it means a long list of items, but it gives you total flexibility and the ability to keep adding new items forever.

Tony Hopkinson
Tony Hopkinson

but no customer is going to be happy with constant changes to functionality. Rapid fixes is one thing, plug ins is another possibility. But if you go too far down the route of functionality changes especially given installation of them is optional, the vendor and customer is going to end up in a total tip. This is one of those things that are technically possible that turn into practically improbable. You'd have to have a very good code base, an extremely modular design with a very powerful versioning system built into deployment to even consider being successful with it in the real world.

Jaqui
Jaqui

a short release cycle in software or operating system. the short cycles introduce bugs in the released version since the time to test and fix the bugs wasn't there.

Justin James
Justin James

Let me know, I'm curious to hear from others on this! J.Ja

Justin James
Justin James

I have always found that it is best to mockup the project in a low cost (in terms of time), high impact (in terms of how close it looks to the final product) tool before beginning the code writing. This process helps show the developer potential pain points ("gee, that probably won't work!"), exposes shortcomings in the project spec ("we are missing something important"), and let's the customer say, "yes, that is actually what I want" before starting work. I have used sketches on paper, Visio diagrams, static HTML versions of the site, JPGs with rectagular boxes showing the screen layout, whatever was really easiest to do and let the customer see what we interpreted the request to be. These visual tools are also invaluable to the developers for when they begin to actually code. J.Ja

Tony Hopkinson
Tony Hopkinson

is very difficult. Dependencies implicit and explicit between modules can eventually get just as complex as a tradition C++, guess the pointer and there's useful side effect in amongst this spaghetti manoeuvre. It's the age old problem all coders have had, if you can remove side effects and you've got the interfaces right and you avoid dependency chains, then you are going to look like a star, until the interfaces change. Make the interfaces nebulous, (pointer to a structure) and you end up with an unmaintainable pile of crap, subject to all sorts of bugs.

Justin James
Justin James

I rreplied to Tony's post along those lines above. :) J.Ja

SnoopDoug
SnoopDoug

You are missing the point. It's not change to existing functionality, it's additional functionality. Do you really think folks are going to complain about more? Not if it makes their life simpler. Suppose we have a simple XML editor. The next time you open it up, it asks you "There is a new validator available, would you like to install it?" Would most users install the new functionality? Heck yes they would. doug in Seattle

Justin James
Justin James

Tony - While I agree with you, that level of modularity is really tough to code, particularly in a secure and stable fashion. Firefox is a great example of that; even the most diehard Firefix fans will readily admit that it is the 3rd party extensions which bring it down or cause problems. When I was doing a lot of VBA work at my last job, those macros could thrash Excel worse than anything else. Modularity at that level is NOT a trivial task, sadly. The easiest/cheapest way to do it is to use a language that supports an eval.() type function, and be careful about what you expose to that eval.(). J.Ja

SnoopDoug
SnoopDoug

A cycle does not mean just writing the code and throwing it at the user. Anyone who does that ends up with the proper fate--in the dustbin. I don't recall whether Agile was mentioned specifically in this article, but this topic is exactly what Agile is proposing. Divide your development into discrete steps and write tests that confirm each step as they are rolled out. There is no reason you cannot write additional code that is just as good as existing code. Dev teams need to commit themselves to a quality metric and stick with it. Better late than crappy. doug in Seattle

Justin James
Justin James

Another problem is regression testing. The faster your development, the more fragile the fixes, making regression testing even more important. There is a certain inflection point in maintenance where testing is a much bigger cost than the coding, and I am positive that agile projects hit this point much sooner and harder than more traditional, "waterfall" type approaches. J.Ja

stew
stew

Isn't part of the problem that you're confusing short release cycles with short development cycles? That is, just because one uses Agile techniques to add features and fix bugs in a project doesn't mean that one must have short release cycles. Besides, the point of Agile techniques is to have a product that is continuously releasable. Then, when one decides there's enough functionality, one just cuts a release. Since everything has been tested from the start, one has complete confidence -- insofar as the tests are trustworthy -- that the product can be released when it meets the release criteria. Obviously, if major problems are found in the product, the tests are faulty and must be corrected. The good news is that the same problems will never reappear unless someone breaks the new tests in the future. Using Agile techniques to develop code day to day gives great confidence in the finished product and emboldens one to make changes. That makes development simpler and smoother.

SnoopDoug
SnoopDoug

Good grief, Charlie Brown, you can never change a public interface once a product has shipped. Even first year CS students grok that. If you dev team changes a published interface, they deserve the fate (massive code rewrites) they get. doug in Seattle

Deadly Ernest
Deadly Ernest

but I have written some small business programs in Access, Oracle, dBase 3 & 4, C++, and Cobol. In all cases I wrote modules and then linked them in, often making changes to the main menu to add in new modules. In some cases it required great care in slipping the modules into the correct place within the major program (very much so with dBase), but Oracle, C++ etc, it was easy to add a new module, insert a few things into the main menu, and recompile. yes the bigger the program the more work and testing required. But it's not as difficult or impossible as some would make out.

Tony Hopkinson
Tony Hopkinson

which may or may not be installed, and may require fixes to other modules which also may or may not be. Descisions like if you want new super duper y, then you must have latest version of x and this module z. It's the granularity that counts, teh finer it is the more flexible and complex.

Tony Hopkinson
Tony Hopkinson

a good fist of it the big bucks :D If you want it it you have to build it from the ground up, you have to keep it at the forefront of your design strategy, and you have to expect to rework your design foundation every now and then, because of something you didn't thing of. This is why most organisations don't do it, while it will save and make money in the long run, the start up cost / lead in time is too scary for the bean counters. I'm thinking more in the lines of meta data than eval.(), but that's not a stroll to the water cooler either.

Justin James
Justin James

I agree with that. But Agile also asks for something more, which is a continuous feedback loop and deliberate lack of long term planning, which I find potentially dangerous. Too many projects I have worked with that were handled like that quickly because difficult to maintain at best. J.Ja

SnoopDoug
SnoopDoug

You state: There is a certain inflection point in maintenance where testing is a much bigger cost than the coding, and I am positive that agile projects hit this point much sooner and harder than more traditional, "waterfall" type approaches. Based on what? Experience or conjecture? How the hell does making smaller changes incur a greater cost than moving from alpha to beta to shipping? Each of these steps adds a tremendously larger number of changes to the code base. Your logic makes absolutely no sense. I think the major difference is that in the Agile method you fix bugs much sooner in the process instead of letting them waterfall forward until the final "fix" phase just after code freeze. If anything the waterfall model falls on its face as you commit to some arbitrary ship date, the date rapidly approaches, and the fix-test-fix cycle gets shortened. I've seen it so many times. The triage team meets more frequently toward the ship date, pushing bugs from one severity level to a lower, less-critical level; changing bugs to "as designed" or "won't fix" to reduce the bug count; deferring a multitude of non-priority 1 bugs to the next release. They then proudly display the rapidly reducing bug count to the clueless upper-level managers. Wow, what a team! doug in Seattle

Tony Hopkinson
Tony Hopkinson

regressions testing is more of an integration test of various combinations of your modules. My experience is the less modular a design, the more regression testing you have to do, because it's much more likely that you've broke something that used to work. That tends to be more of an issue at the code level that the product one though. Course you can't have a modular product without modular code.

Jaqui
Jaqui

customer given a choice: fastdelivery, bug ridden and unstable or slow delivery, higher cost, fewer problems. most customers will look at the downtime costs and say they will take the better quality product. open source is well known for rapid release cycles, as well as rapid patches when needed. when an app takes longer between releases it gets fewer patches. and Firefox is an excellent example of this, several years development and under 30 patches required for the first year after release. Most bug fixes were included in incremental releases over the course of the same year, patches were only done for critical or severe security issues.

Justin James
Justin James

... requires an interface change, other wise the user will not be able to use the feature. The only way to have a new feature not change the interface is to display non-working interfaced elements from the get-go, like emtpy table columns ready to be filled in when the new feature is added, or buttons that do nothing when clicked. The trick is, developing a roadmap to the features such that adding the new functionality adds to the interface without changing or eliminating what has already been deployed. No one complains when you add a button. They complain when you move, replace, or remove a button. Having a well thought out roadmap like this is a deliberate "not going to happen" with most Agile programming methodologies. J.Ja

Tony Hopkinson
Tony Hopkinson

Coping with not changing them causes as many problems as not. In fact in the plugin situation, not changing the interface should it be required introduces an even greater level of complexity somewhere. In fact that was my main, thrust modular product is not modular source. If your new validator takes extra parameters that can't be squeezed in teh existing ones. So now it's change interface on existing, or create a new version of the plug in. One of teh new parameters comes in damn handy for a third level module the validation results viewer. Now you've got two bases, four modules , eight combinations only two of which are legal. Never mind intermediate products, side effects, os specific dependancies. Even Linus would need a comfort blanket, if you take it a few levels further. Can happen easily with bean counter input, in fact will and does.

Justin James
Justin James

I am facing the same challenge... an application to automate workflow that gets customized on a customer-by-customer basis, and each customer can have multiple workflows based on initial input! In my fantasy land, I could use XAML + WPF for this, but the development was done in Java, so at best, UML (that monolithic, massive, difficult to work with beast...) may have to get used... J.Ja

Tony Hopkinson
Tony Hopkinson

I'm pushing meta data hard. The software has to change at least twice a year for legislation, so hard coding the logic inside the presentation layer has finally been proven unacceptable. :D Hoof***ing ray.

Justin James
Justin James

Just as "adult content" is a major driver behind technological adoption, "will I get a bonus?" drives far too many business decisions. Unless modularity is a sellable feature that will increase someone's bonus, you won't see it. On the project I have with my current job, modularity can become a bonus-able feature, because it means that we can sell a short integration cycle to the customer as well as internationalization because the text could be split out from the presentation. But the developers are scared of this, because it was never an original project spec. Tough for them, because my job is to drive this project on a technical level and make sure that it is the best piece of code possible. And we all know what a fascist I am about getting things right. :) J.Ja

Tony Hopkinson
Tony Hopkinson

is that it's not a one to one translation with modular software design. This is simply because effectively you only have one version of a module compiled in code, but you can have many compiled modules at run time. It's not hard to sell the benefits, technical and commercial to business types, the lead in design time to get it right before they see something with a commercial application though, that is a tough sell. In my opinion, the key is to get a definition of the core product. Some of us might think from a technical point of view base level firefox should have had less in it, but from a business point of view they got it down pretty well. Can you 'sell' it without the extras, what extras have user appeal. It's when you confuse software modularity, with product modularity you get either something overly ambitious, or something that's really monolithic without the benefits (such as they are).

Justin James
Justin James

3 jobs ago (but only two empoyers ago), I spent the bulk of my time talking to people in Europe (primiarly the Netherlands) and Australia. It was an intensely awesome, and eye opening experience into the world of how other people speak English. The most interesting thing that I saw was how people in Europe in non-English speaking countries spoke this grammatically perfect English that was actually tough to understand due to its lack of slang. It felt like walking into the bathroom at the gas station, and finding it tastefully decorated, pleasant smelling, and filled with exotic antiques. Served the same purpose, just felt too "clean". My favorite lesson I learned was that the Australian accent that Americans are so familiar with (Paul Hogan, Steve Irwin) is actually the Queensland regional dialect, a very rural, informal accent, along the lines of the deep South in the US. A lot of the Australians I talked to were always baffled about how Americans had this specific accent in mind when thinking of Australians. On the flip side, many of them had the impression that most Americans spoke like Jole PEsci, Robert DeNiro, and other actors famous for New York accents (although to be quite specific, many of the actors we think of having a "New York" accent actually have a Northern New Jersey accent, and there *is* a difference!). When we'd goof off, they would put on Queensland accents, and I would revert to my New York accent (I was born in New York City and both of my parents are from there, but I have spent so long away from there that my accent only comes out on rare occassions), and we would just have a heck of a time trying to confuse each other with regional slang. J.Ja

Deadly Ernest
Deadly Ernest

some are due to cultural baggage - in Australia we're now metric but many pre metric expressions are still in use as they are part of the culture. Eg, how many kilometres to the litre is still the car's 'mileage' performance. We use margarine when we 'butter' bread. A miss is as good as a mile, give them an inch and they'll take a mile. Many more like that, carried on from prior usage. Others are adopted from other cultures - gung ho, big bucks, etc. They continue to get used as they now have a universally accepted meaning, and constitute a linguistic short hand. Big bucks is well understood to mean 'they got paid extremely high rates of pay, in excess of what they really earned' much shorter to say big bucks.

Justin James
Justin James

At the end of the day, with the exception of some far sighted organizations, modularity at that level is considered a "feature" and only written in if it can generate a revenue stream. Bean Counters are like that. And on that note, does anyone in the UK get "big bucks"? I would think that they would get "big pounds" instead. ;) That is a linquistic oddity I've noticed, a lot of phrases get used in places where they are not culturally valid, like "going the extra mile" in a metric country. :) J.Ja

Justin James
Justin James

"Based on what? Experience or conjecture? How the hell does making smaller changes incur a greater cost than moving from alpha to beta to shipping? Each of these steps adds a tremendously larger number of changes to the code base. Your logic makes absolutely no sense." Both. The logic is actually quite simple. With a longer release cycle, you have certain points where you test everything, and continue on. Those tests are expense in terms of time, energy, and effort. With a shorter release cycle, you are performing those tests much more frequently. The bigger the project gets, the longer the tests take. Therefore, the more often you run them (and agile runs them constantly, compared to waterfall), the longer your release cycle becomes. "I think the major difference is that in the Agile method you fix bugs much sooner in the process instead of letting them waterfall forward until the final "fix" phase just after code freeze." Yes, this is true. But again, you are forgetting that running the tests themselves have a cost. Running them a few times is much cheaper than every 4 days. "If anything the waterfall model falls on its face as you commit to some arbitrary ship date, the date rapidly approaches, and the fix-test-fix cycle gets shortened. I've seen it so many times. The triage team meets more frequently toward the ship date, pushing bugs from one severity level to a lower, less-critical level; changing bugs to "as designed" or "won't fix" to reduce the bug count; deferring a multitude of non-priority 1 bugs to the next release. They then proudly display the rapidly reducing bug count to the clueless upper-level managers. Wow, what a team!" This is a problem with PM's, salespeople, PR types, and other people who commit to a ship date. Agile experiences this just as often, the difference is that instead of saying, "Version 10 will be done in 3 months" they promise a set of features. Either way, you declared that a set of features (a whole version, or individual items) would be done by a certain point. With Agile, at least they get *some of the features*, but it depends a lot on when and where the hold up occurs. J.Ja

Justin James
Justin James

While I agree with a lot of what you've written here, I take exception to part of this: "You are mixing apples and oranges. It is not fast delivery. It is frequent delivery of updated functionality. And it has nothing to do with bugs. Bugs are from many sources, and of course one source is lack of testing time. The lack of testing time is just as endemic in long development cycles as short development cycles. All else (same eventual product, same development team, same tools, etc.) being equal, a shorter cycle with fewer changes should expose fewer bugs and only bugs in the added code. After all, there is less code to run through regression tests than in the "throw it all together in the end" model." Adding code in one area DOES NOT only create bugs in the added code! Without going through and testing everything all over again, you have no idea in many cases what you may have accidentally trampled all over, particularly when the change required a modification to a core code component. J.Ja

SnoopDoug
SnoopDoug

You are mixing apples and oranges. It is not fast delivery. It is frequent delivery of updated functionality. And it has nothing to do with bugs. Bugs are from many sources, and of course one source is lack of testing time. The lack of testing time is just as endemic in long development cycles as short development cycles. All else (same eventual product, same development team, same tools, etc.) being equal, a shorter cycle with fewer changes should expose fewer bugs and only bugs in the added code. After all, there is less code to run through regression tests than in the "throw it all together in the end" model. Let's face it, all product development mixes code together and runs regression tests all the time. The difference is that the quality of code in short cycles must be higher than the quality of code in long cycles. Why? Because you are exposing it to the user. If you put buggy code out to a user, you deserve to be chastised. It's the choice of the product team to decide when to release any code to users. Would you prefer to get functionality A in a week or two (and B... in two week cycles) or wait for a year for A through Z? The point of short cycles is to get the really important stuff out quickly, let the users tell you that you got it right, then go to the next most important big of functionality. You adjust your sights as you go. The biggest issue is cultural, both on the dev and product side and on the user side. I suspect that internal and free products are much more likely to go Agile than consumer products. Although internally I'll bet Microsoft and their famous "we eat our own dogfood" philosophy is more akin to Agile than we realize. doug in Seattle

Justin James
Justin James

... drive me INSANE. I really cannot bear the management of it many times. It seems like every other day, a huge new release to some critical part of a FOSS install is getting upgraded. Even worse is when a core library that other items rely upon is in the short release cycle mode, and are not fully backwards compatable. You end up with a JVM-like mess, but at a much deeper, and faster scale. Ugh. J.Ja

Editor's Picks