Leadership

Commandments for Distributed Collaborative Development

Independent software development consultant Chip Camden spends much of his day worrying about Distributed Collaborative Development. These are the DCD rules he has learned throughout the years.

As an independent software development consultant who works mostly from home, I spend a large portion of my day worried about DCD. No, I don't mean Data Carrier Detect (I stopped using dial-up more than a decade ago), nor do I follow Dead Can Dance (that band split up in 1998). I'm talking about Distributed Collaborative Development: software engineering that defies the unities of time, place, action, and project management.

I've been doing this sort of work since I began consulting in 1991 -- over a 9600-baud modem back then, and across the Internet today. Nobody ever handed me a "DCD for Dummies" book -- my clients and I discovered the rules as we went along. Out of the goodness of my heart (and because I like playing Moses), here are 10 commandments for DCD, carved in the hard tablets of experience.

1. There shall be only one authoritative repository of source code. Nothing leads to more needless wheel-spinning than splitting off a version of the source for developing a feature that will eventually need to merge into the main source. Rather than creating a separate copy, use the power of your version control system of choice to branch and merge specific files as needed. This brings us to the second commandment.

2. Thou shalt use version control in all of thy projects. No matter how small or how prototypical the project, version control software is your friend. It not only prevents developers from walking on each other's work, it also documents the project history and acts as a layer of backup. For DCD, you need an option that works well over long distances. A traditional VCS repo on a shared drive over VPN will add many hours to a project of just waiting for archive access. You need something designed for distributed work, like Mercurial or Git. These products allow you to clone a local repository that you can periodically sync with the central repo -- minimizing the bandwidth required to do intensive development. That also encourages frequent commits, which is a good practice for when you have to backtrack.

3. Thou shalt not kill the build. Don't push changes to the central repo until you're reasonably sure it isn't going to cause grief for your other team members. If the project gets ported to Windows, Unix, and OpenVMS, then you'd better test your changes on those platforms first. If you can't easily do that (waves hand for OpenVMS), then at least review the code with that platform in mind. Remember that breaking the build causes even more grief for remote developers, because every change they subsequently have to pull to fix it has a heavier time cost. Besides, if they're on a different time zone, they might not discover your mistake until you're snoring in bed, so they have to wait until your next day, which is their next night.

4. Thou shalt defend the orphan. That includes you and anyone else working remotely. Development procedures often evolve without consideration for their impact on DCD. Someone decides that it would be a good idea to insure that a particular file is always up to date, so they add an unconditional version control "pull" to part of a build procedure. That doesn't present a problem for developers on the gigabit LAN, but it brings builds to a screeching halt when you're getting less than a megabit down (or not even connected to the repo host). Remote developers have to insist on opt-in-only access to the central repo -- or at least a single opt-out to all automatic access.

5. That which is united shall not be put asunder. Separate concerns carefully between teams. Too many interdependencies will waste just as much time as a broken build when one team has to wait for another to complete a component or make a decision before they can move ahead. Oddly, finger-pointing also becomes more prevalent when the rightful target of the finger is not well established.

6. Small and frequent shall be thy deliverables. It's always a good idea to break projects down into smaller components, just to ease project management. But it's even more important for DCD, because the impact of a commit/merge on the main product becomes more manageable and predictable. In contrast, a huge and as yet unseen deliverable looms silently and ominously on the horizon.

7. Thou shalt not leave thy fellow developers in the dark. Communicate, communicate, communicate. You may think you know what you're supposed to deliver, but the other teams may have a vastly different expectation. You don't want to find that out when you're done. Remote developers may not be able to be present at daily meetings, so they have to use other ways of staying up to date, and keeping everyone else updated on what they're doing. Video conferencing still hasn't achieved wide acceptance, so regular emails and phone calls often fill the gap. A private wiki can be helpful, because it holds the entire topic of conversation together for posterity.

10. Thou shalt not use octal. It confuses people.

About

Chip Camden has been programming since 1978, and he's still not done. An independent consultant since 1991, Chip specializes in software development tools, languages, and migration to new technology. Besides writing for TechRepublic's IT Consultant b...

45 comments
Jaqui
Jaqui

to non open source projects, but it sure does to open source, respond to ALL feedback asap. not just for clarification if needed, but if someone is making a comment, they will be much happier in getting even a simple thank you for it. if you have presented a test release of the project to the client(s) and some of their people are making comments, don't just take hem and address them in the project, respond to the people directly. it helps with getting them to buy into the project completely, and the positive feeling from hearing from the dev team about their comment carries over to the project itself.

dawgit
dawgit

Easy to over look too. Today I had an interesting meet up with a friend / college, and the main theme was 'Language'. That is there must be a common Spoken language between all parties. English is preferred, (because it's easier?) but there must be one standard established. I know it might seem trivial to most used to working in an Homogenized Society, But most of us are used to being exposed to at least 3+ languages per day... That alone can get any program rather mucked up very quickly. Otherwise, good points. :)

Sterling chip Camden
Sterling chip Camden

Same day, if not sooner. Otherwise, you soon won't be getting any more feedback, or it will be drastically reduced. And that's a negative for anyone, commercial or not.

AnsuGisalas
AnsuGisalas

English is not, most certainly, an easy language to learn. Nor is it easy to use. It's also not a very usual language, in fact it's almost perverse in it's desire to deviate from normalcy. I blame Henry 10, with his papacy and his beheading of the wives. Deviance bred into the bone. English is simply the language of the present Hegemony. Like Latin of the european dark ages, it's the language spoken by the powers that be, and it is inundating the world through "cultural imperialism", as the french like to put it. So, while it is difficult, deviant and opaque, it's what we're given.

Sterling chip Camden
Sterling chip Camden

You're absolutely right -- except English isn't easy: we use it because more people speak it than most languages, and most Americans don't know anything else. There's also a need to get on the same page in technical jargon, programming languages, and programming style. Sure, you can (and often must) employ more than one language, but consistency of application helps.

bergenfx
bergenfx

... or even a crazy 8. So, I will not offer this as an addition to that list, but rather as a homily. Maybe it could be put in the same book or something. Blessed are the collaboration architects for their tools and methods shall keepeth everyone on the same tablet. And by the way, I am not overly generous with praise. Otherwise, I would have called you out as the best TR contributor, both in content and style, long ago. But this --> 7. Thou shalt not leave ... 10. Thou shalt not use octal. It confuses people.

Jaqui
Jaqui

your entry has the example of time zone differences that can make that impossible. ;) that is why I just used asap. :D you get the feed back, spend say 30 minutes looking into what it's about and then respond, using your look at the code as a foundation for the response. danged typos

Sterling chip Camden
Sterling chip Camden

That deviance in the English language goes back way before Henry the Octal Order of Magnitude. Perhaps Henry was more a symptom of English than the other way around.

Jaqui
Jaqui

as in code layout? shouldn't that actually be in the SRS for the project? things like spaces vs tabs for indenting code blocks number(s) of each all need to be the same across the project. I also think commenting of code needs to be clearly defined, for any project, not just a distributed team. [ use an example of the type of commenting and code structure / layout that will be expected for all code in the project. ]

AnsuGisalas
AnsuGisalas

"ain't"? :) But, that was fun. Octal took me two seconds, filtering through potential reasons. Exactly the right amount to trigger the funny centre when the answer came bubbling up out of the morass of my mind... longer would have been embarrassing, while shorter would have lost the pun.

AnsuGisalas
AnsuGisalas

Would that be "Thou shalt answer thy questioner, for so shall also thy question be answered"? Or is it simply "Thou shalt not be a pompous asse!"

Sterling chip Camden
Sterling chip Camden

After 30 minutes you should be able to respond, at least with "I'll need to do more research" but hopefully with "it looks like it might be..." and always "thanks for notifying me".

bergenfx
bergenfx

... scaring the other children

Jaqui
Jaqui

but the commenting needs to be specified in the srs / project docs. that gives it the "no excuse" for when it isn't done

Jaqui
Jaqui

and don't /* * This Function displays * the output of the * program formatted * for the console display */ or something equally inane. :D but putting a descriptive comment block before a function that is doing a whole mess of work and might not be clear on reading the code is also a good thing. the hello world main function for C, n comments needed, really. the main function for say, V(im), could use some comments ;)

Sterling chip Camden
Sterling chip Camden

There are certain semi-universals. Don't comment a for loop with // for items 1 thru 10...

AnsuGisalas
AnsuGisalas

But I see in front of me an image, that is my world. I know (having taken the leap of faith that the Daemon forces), that the image contains a representation of something that, for lack of a better word, I call real. I also know that my perception is warped, and that warping is systemic. So, when I spot a shibboleth of systemic warping, I grab onto it, pull it out, roots and all, to bake it in the sun. After that the image realigns, and I see new things, and it's a marvelous feeling. Of course, I know, it's infinitessimal progress, ever only approaching the real, for what it's worth.

bergenfx
bergenfx

but I guess I did. I love cross threading -- well done. Count me as a defender.

bergenfx
bergenfx

And since chaos (or the perception, thereof), seems to be a matter of frame of reference, you get recursive ascent, (While I have used the phrase 'recursive descent' hundreds of times, I have waited decades to use its counterpart. Thanks for providing the opportunity to use it).

bergenfx
bergenfx

Everything is a metaphor for ... uh ... (can't think of a better word, even though it is looked at unfavorably these days) ... truth.

santeewelding
santeewelding

Once known, all else is known. The Upanishads (?)

Sterling chip Camden
Sterling chip Camden

Most people treat it as only interference. It's part of the "nonsense" they need to filter out of their world. We geeks delight in it, which is one of the reasons why we're so poorly understood by non-geeks, and only partially understood by semi-geeks.

AnsuGisalas
AnsuGisalas

Out of the chaos rising on the tide a pattern that, once seen, is found everywhere. That pattern forms a basis for a progressive simplification of all processes, clearing out chaff and smokescreens, and then the freshly cleaned machine rips roaring into pristine territories formerly behind the curtain. At least, that's how it feels like.

bergenfx
bergenfx

Oh, please don't put me in the corner ... anything but that. I especially liked the fact that I could use it against the teacher. I mean how could I be responsible for classroom content when I was in the corner?

bergenfx
bergenfx

Interesting view on it. Sounds to me that introducing self-reference into to a system, processing self-referentially, is like John Malkovich jumping into his own portal ... everyone he sees is John Malkovich. Okay, not that clean as an analogy, but I really like the notion of feedback loops acting harmonically, and certain stimulus creating interference patterns to that. And then there is the whole issue of what is the result of that. In what ways can the interference stimulus (to a higher level thought process) create expansion or enlightenment? I'd better stop now.

bergenfx
bergenfx

... here is a different direction. There are two kinds of people in the world. Those who apply finitely patterned, trans-abductive, para-cognitive states cross-referenced with meta-phenomenological bifurcative pathways in transformational extent and those who are not semioticians.

AnsuGisalas
AnsuGisalas

That's where we all hang out. They say there's a water cooler there, but seriously, I've never seen it for the crowd :p Self-reference is exactly how we think. The brain works by way of feedback loops. A neuron (or other functional brain cell) can "feel" itself downstream, infinitely looping back into it's upstream one way or another. Like standing between two mirrors, seeing one's selves myriadly facing off, all the way down the corridor ahead/behind, curving off into the vanishing point. So, self-reference does affect our thinking, and there are often interesting harmonic/disharmonic interference patterns when taking one feed-back system into the domain of another. But what I really meant was, that if someone read commandment number 10, and was extremely in cue with Octal, then the octal10=decimal8 realization might come too quickly for it to be funny. If it takes too long, reversely, the usual response is a feeling of "ugh". I guess this is a case of knowing one's audience, mostly, and that's as important to a humorous writing as to a serious.

AnsuGisalas
AnsuGisalas

That's true of course, and accurate, but the other one was, I think, sufficient. Of course, the other one can be given an arbitrary number and still be true. "There are two kinds of people in the world: those who divide the world into thirty thousand groups and those who don't" That's the problem and beauty of logic with integers, exactly being the unlikely default.

Sterling chip Camden
Sterling chip Camden

There are two kinds of people in the world: those who divide the world into two groups and those who either disagree with this statement or have no opinion.

Sterling chip Camden
Sterling chip Camden

... then you, I and a few others would never get a chance to post anything.

santeewelding
santeewelding

In the corner is my base of operations. It's how I can tell when Chip is doing it. Come. Have a front-row seat.

bergenfx
bergenfx

Good one. Here is one I wish I could take credit for, but it was a friend... There are two kinds of people in the world: those who divide the world into two groups and those who don't.

bergenfx
bergenfx

"trigger the funny centre" For me self-reference is much more of exciting the cerebral outback, which is inhabited only by a few Dr Seuss-like characters, although it does feel kind of funny-centre-ish because it typically results in a humo(u)rish reaction, typically followed by a brain re-organization. Godel,Escher, Bach had a major restructuring effect, while anything written by Charlie Kaufman was more of a scrambling effect, (I guess that would be brain de-organization). I am being semi-serious here. I think there is something about self-reference that "messes with" the conventional ways we think; kind of like Zen koans breaking the walls of logic. Any thoughts on this? (Does one get put in the corner of TR for this kind of talk?)

Jaqui
Jaqui

the reference to the "there are 10 types of people in the world" line? :D [ those who can. and those who can't being the types. ]