Software Development

Business analysts: Clear communication may curb application development failures

Application development success depends on the clarity with which business analysts -- those liaisons between the development and business sides of the house -- explain each side's needs to the other.

By Amy Rogers Nazarov

Despite decades of advances in programming languages and testing tools, application development projects still fail at an uncomfortably high rate.

While The Standish Group reports a lower rate of outright failure -- 19% of software development projects in 2006 as opposed to 31.1% in 1994, when the organization's flagship CHAOS report was first published -- there is still substantial room for improvement.

If more progress is to be made, experts say, it will hinge on the extent to which business requirements evolve in tandem with the application; the degree to which requirements are clearly communicated to developers; and the clarity with which business analysts -- those liaisons between the development and business sides of the house -- explain each side's needs to the other.

Looking to the future

"The key with a BA is not to just understand business needs at that moment, but rather to ask questions -- what could happen, why it could happen, and when," says Dan Walker, group leader for supply-chain business systems at International Truck and Engine, Warrenville, Ill.

Walker notes that his company began evolving the BA's role several years ago, when it realized that IT was not fully synchronized with other business areas. "We didn’t want IT to be the black hole" into which insufficiently described business requirements are lost.

Joe Marasco, CEO of Ravenflow, an Emeryville, Calif.-based company whose products seek to align requirements with development, says that at least half of the errors in struggling software projects come from "poorly understood requirements."

Indeed, some 93 percent of readers of Requirements Development Journal (http://www.requirementsdevelopment.com/) complained that users expect functionality they did not originally request, while 89 percent cited incomplete requirements as a source of aggravation and 85 percent of respondents said they had to struggle with requirements that were ambiguous. Given these miscommunications, "it’s a bum rap to blame [a failed project] on the developers," Marasco says.

Further complicating matters, requirements are in a state of constant flux. Yet Ian Knox, a product manager for Microsoft’s Visual Studio Team System, notes that the more changes tracked and collected, the more accurately the resulting application should reflect the needs of the organization.

Set a baseline

While every change has a time and/or budget impact, establishing a set of baseline requirements for the application will help all stakeholders gauge the effect of the inevitable changes.

"You have to spend a lot of time in the design phase," says SeemaPhull, founder of the Arsu Group, a Phoenix-based firm that consults with clients on supply-chain business process reengineering. It’s at this time, when applications are first conceptualized, that discussion is most critical among representatives from business and from software development, she says. "Otherwise the IT people may run off and start putting the technical specs together, perhaps picking the technology with the most bells and whistles, not the one the organization needs most."

People give short shrift to the process of soliciting and documenting requirements, says Ravenflow’s Marasco. What’s more likely to happen is the business analyst will generate a set of requirements as thick as "a Manhattan phone book" -- requirements that are often handed off without being validated.

Others believe that in those early meetings, it’s incumbent upon the BA to help management grasp the range of options in a new application. "Senior executives are not stupid," explains Kevin Brennan, vice president, body of knowledge for the International Institute of Business Analysis and a business analyst at blue sands, an IT process consulting firm in Toronto. "Show them what they could do [with new software] and why they should care."

Business problems may sometimes be solved by non-technological solutions. It’s up to the seasoned BA to recognize when that might be the case.

"If you come from an IT background, you tend to think the answer to everything is software," says Barbara Carkenord, who was a programmer at General Motors before co-founding B2T Training LLC, the Knoxville, Tenn.-based BA training company where she serves as president. "But some [fixes] may be procedural; others may be organizational."

Michael Blechar, vice president and research director at the Gartner Group, Stamford, Conn., says that the place in an organization from which the BA hails -- and where they are ultimately housed -- can affect the success of the development project.

"Usually it has been the lead analyst from the IT side who is responsible for gathering requirements," Blechar says. "But as business process management and improvement initiatives have been springing up in business areas, the role has been moving from IT over to the business units." That’s because as many as two-thirds of business processes, Blechar contends, are "human-to-human in nature. They do not involve computer systems."

Obscure no more

Historically, those in the BA community -- who may hold a variety of titles, but all serve as that link between technology and business -- have "labored in obscurity," says Marasco. Yet their stature is being elevated through new certification and training options.

The IIBA recently created the first industry certification exam for business analysts, to be offered around North America this year. And about 5,000 people per year go through B2T’s two-week training curriculum for business analysts, Carkenord estimates. Among other issues, they learn that "it almost always takes longer to gather and document the requirements than for a coder to create the application," she notes.

In an attempt to rectify this disparity, software vendors are changing the way they communicate with developers and the types of best practices they advocate. "About five years ago, we realized we couldn’t focus [exclusively] on the developer anymore," says Microsoft’s Knox. "We had to include BAs and [application] testers."

Ashok Reddy, director of Rational platform offerings at IBM, confirms that the "waterfall" approach -- in which developers attempt to gather all business requirements at the outset, put together an application, then unveil for users who have until now had little or no idea of what they were going to get -- has fallen from favor. Projects that take an iterative approach -- in which chunks of code are developed, then demonstrated to the line-of-business people, who test it against their requirements and suggest needed changes -- "let you flush out risk as you go," Reddy contends.

The more risk is expelled along the way -- in other words, the more opportunities users have to give input into development projects -- the more likely it is that the finished product will meet real business needs, and expensive re-writes minimized.

Setting and meeting iterative proof points will go a long way toward ensuring that software works as expected, says the Arsu Group’s Phull.

"Define the business processes, then define the end state," she says. "Too often, companies focus on simply automating what they are doing today."

8 comments
Dan.Neunaber
Dan.Neunaber

This is a great article. The guru of machine-human interfacing is Donald Norman, a cognitive psychologist, MIT electrical engineer, and former Apple VP in the Applied Technology Group. His books describe how bad design flows from a failure to consider how the product will be used. He even advocates eliminating the word 'user' and instead calls for use of the word 'people.' Too often, those of use who use hardware or software must bend to the whims of the designer, when in fact it should be the other way around. Anyone who reads Don Norman's books will find example after example of poor design flowing from a failure to consider who uses the technology. His books are quite amusing and a great follow up to this discussion. He can be found at www.jnd.org

Tony Hopkinson
Tony Hopkinson

Clear communication may curb application development failures. What do you mean may, how does unclear communication facilitate success? Developer to BA to Customer can help, especially in specialised domains. Customers do not know what they want, found this out and it will change before you've wrote it down. As Wayne said limit your risks, take small steps, review your goal. Never invest so much in part of the development, that you have no choice but to keep it whether it turns out to be garbage or not. Change is a given, if you accept that, plan for it, use it, you will end up with what the customer really wants, not what the developer thought the BA wanted who thought that was what the customer wanted who didn't know what they wanted in the first place. The only way to cope with change is to communicate, regularly, frequently, both formally and informally through all levels and all parties. I've found as many if not more design or implementation flaws at the 'watercooler' as I have in a formal three month review. Information must flow up and down. It's not that communication is unclear, it's, it not happening enough. Otherwise someone would have pointed out that the communication you did do had gone wrong because of a dumb assumption like order number is an integer, or some such.

Wayne M.
Wayne M.

Unfortunately, this article is a mixture of somewhat contradictory quotes. I would focus on two areas that affect the development of software systems: Communications and Cycle Time. The introduction to the article gets it two-thirds right, "If more progress is to be made, experts say, it will hinge on the extent to which business requirements evolve in tandem with the application; the degree to which requirements are clearly communicated to developers; and the clarity with which business analysts -- those liaisons between the development and business sides of the house -- explain each side's needs to the other." Communications between the software users and software developers is critical. The answer, however, is not to hope for more and better trained intermediaries, but to implement direct, face-to-face communications between the two. By the time we add in Business Analysts, Project Managers, System Architects, and Software Designers, it is incredible that anything requested by the users is ever accurately heard by the developers. These added levels of communication limit how fast the development team can iterate cycles, slowing down the feedback loop between users and developers. Thus we get comments, such as, "You have to spend a lot of time in the design phase, ... Otherwise the IT people may run off and start putting the technical specs together ..." Clearly, we can't have a speedy response to a request! The keys to success in a software development effort are: improve communications by removing as many of the middlemen as is feasible and mitigate the affects of miscommunication by quickly turning around implementations in small scope iterations.

Tony Hopkinson
Tony Hopkinson

red rag to a bull that one. That the designer who was told to use access, to integrate with third party btrieve file, that it must use XML at either end of a fixed format file VPN mailbox? That sort of failure happens far more often than poor user assessment, though I've seen some absolute classics on that front as well. My favourite was the one with a piece of information that was missed out and so has to be entered manually. On system up three flights of stairs once every two minutes.

Dr Dij
Dr Dij

such as iRise or Prophesy which show reqts visually are getting so popular. The flow of use cases, screens and menus can be shown to end user before coding begins. Instead of 500 pages of reqts which are hard to read and hard to find missing parts as we think visually. One problem currently is that management thinks reqts analysis takes so long they want coding to start right away. Even if it takes longer than coding for all but insignificant projects, it is necessary. It would be like replacing parts of your car before going thru diagnostics. You might have the wrong reqts or replace wrong part. In the end you may have to allow for some scope creep. But having a fixed deadline for the iterations hashes out what is most important feature from end users use cases. a hybrid waterfall and iterations approach may be best. This also gets working software with some of most important features done early in the game for end user feedback.

Suzie B
Suzie B

I agree that you should remove 'as many of the middlemen as is feasible', but you need to ensure that the middlemen eliminated are the right ones. I am a business analyst of 17 years and my business customers tell me I provide value to them most days. Key to this is an understanding of how they work at a individual level,the application architecture and empathy to their needs. I often see BAs that instead of facilitating what the business customers really do want, spend their time telling them what they can and can't have. A BA with these attitudes provide no real value to business customers at all, and often my customers just turn off to these people. I'm recognised for my additional 'technical' skills as well as business expertise, but it often amuses me that many other BA's see this as a different skillset to what they need to have. You've got to have the right people with the mindset needed for the job, or at least those willing to develop themselves. The BA role has eveolved so much in the last 17 years, but some BAs are being left behind in the dark ages simply because they are unwilling or, dare I say it, lazy.

Tony Hopkinson
Tony Hopkinson

Most often they mislead. They tend to be a considerable investment. They tend be sold as the silver bullet. After such an investment, people feel a need to use them, suitable or not. All the successful developments, I've worked, both in terms of customer expectation and achieving cost and schedule deadlines have been iterative. As you say get something usable into the customer's hands as fast as possible. When I say usable, I don't mean works with up to ten customers who order just one product either. Don't try an conjure a whole cloth solution and then find out your wand has broken, or more usually the case was bought from the firm who sold you those X rays glasses when you were a pimply panting youth. :p

Tony Hopkinson
Tony Hopkinson

for cost cutting purposes for a long time in the UK. In fact the first time I worked with a domain analyst was for an american company. A good one is worth her weight in iridium. The good ones I've worked with have always not needed to be told that it's not obvious what the customer wants. As a developer, I need the requirements explained in a 'language' I can understand, otherwise we get too far down the track and then you have to explain to them it's not possible. That language isn't code though, if that level of detail appears on my desk, it makes me wonder what the analyst missed while they were doing my job for me. I needed an analyst who knew that I would assume a number is numeric, unless explicitly told otherwise. Must confess most of the time I go back and check now, not all my analysts have been good ones. :(

Editor's Picks