This article originally appeared in Technology & Business, a ZDNet Australia publication.
Software vendors keep telling us that Web services are the answer, but what’s the question? Are there business drivers pushing for Web services, or are software vendors just trying to drive their sales?
Business and IT managers like to think in terms of distinct projects with clearly defined deliverables and a relatively certain return on investment. Vendors, on the other hand, are keen to preserve their revenue growth and are loath to concede that things sometimes work well enough pretty much as they are.
This eternal struggle between innovation and pragmatism is rapidly coming to the fore as software companies follow a well-worn path by hyping Web services into the stratosphere. This year, anybody who's anybody in the software world is building a Web services strategy that will dictate the composition of their next major version.
Customers are exhorted to buy into the technology. New and improved Web services standards are rolling off the treadmill. Observers are jumping on the bandwagon with heady market shares and take-up projections that will no doubt prove over-ambitious in time. Analysts are spewing out grandiose projections about the potential of the Web services market that belie the torpor plaguing the rest of the IT industry.
All are adding further fuel to the Web services fire.
While other highly hyped technologies, like CRM and ERP, promised specific outcomes, Web services is little more than a collection of standards without an actual product to encapsulate them. It‘s a means to no particular end, rather like giving a box of Lego to a child and hoping it’ll build something useful.
Because everybody recognizes its importance, the Web services industry has become the latest battleground in IT's perennial battle between dark and light—although this time the players are different.
Microsoft, which by now has become used to being universally distrusted because of its chronic desire to differentiate its products, has been working closely with IBM to rapidly develop consistent and workable Web services standards under the auspices of the 106-member Web Services Interoperability Organization (WS-I).
"We're trying to take the Internet and make it into an enterprise-class platform," says Charles Stevens, vice president of Microsoft's Enterprise Partners Group. "Customers like the building-block approach because it lets them pull these systems together. But we can't just talk the talk: we've got to prove that we can be a world-class integrator using XML across business-class platforms."
Microsoft's vision of Web services is based around its .NET strategy, which is rapidly gaining substance with the recent release of the Visual Studio.NET development environment and the anticipated launch of the Web services-enabled Windows.NET operating system next year.
Then there's IBM, which has heavily promoted both Linux and XML-based Web services but which favors the use of J2EE (Java 2 Enterprise Edition) as an environment for building and integrating Web services.
Sun, the creator of J2EE, has consistently refused to join the WS-I, although it finally conceded that joining might position itself as a Web services leader. However, Sun’s offer to join was contingent on the creation of two new board positions that would put it on an equal footing with IBM and Microsoft in charting the course of Web services. IBM thinks Sun's Java expertise would be beneficial; Microsoft, typically skeptical of its ideological nemesis, is against Sun's membership. So far, the decision as to whether Sun will be admitted—much less whether it can secure itself two board seats, which are coveted by dozens of other WS-I members—is still up in the air.
With vendor sniping rampant and the final pantheon of Web services still unformulated, customers may be forgiven for remaining skeptical of the whole concept.
Like it or not, Web services are coming
Political machinations aside, Web services will soon be a part of your life whether you like it or not. They’ll be ubiquitous in major operating systems, and within a few years all major enterprise applications will use Web services interfaces.
If nothing else, Web services have the potential to end the headache of application integration, a plague on all manner of businesses. Because vendors recognized the need for backwards compatibility early on, they've positioned Web services as a way of developing completely new functionality while integrating existing code from all manner of environments.
"The justification has to be to unlock the value that people have already got," says Mark Chrimes, managing director of Avanade Australia, which has already engaged several customers in .NET discussions and has helped drive early Web services implementations with New South Wales Department of Employment and Workplace Relations and Sydney's Taronga Zoo.
"There's a lot of dissatisfaction from current implementations, but the cost of replacing legacy systems is phenomenally and prohibitively high. There are many more moving parts in the IT world now than before, and the real value of Web services is going to be in reducing the cost of development of those applications."
While openly praising the concept of Web services, analysts universally urge caution on customers who want to jump on the bandwagon. That's because everybody agrees Web services are a good thing, but there’s still no consensus on exactly what they are. Add to this a lack of available products and a business world with far bigger concerns, and you've got a Web services market that even vendors concede is set to go through a period of disillusionment as customers cotton to the fact that two buzzwords and a good idea won't improve their bottom line overnight.
The specifics of each vendor's view of Web services vary based on each company's ideological perspective, but it's safe to define Web services as online modular applications that use consistent standards to provide seamless interfaces between multiple and distributed business processes.
There are, necessarily, common elements to each vendor's vision. Because it has already been recognized as the most desirable standard for interoperable data representation, XML is accepted as the most desirable method of data representation.
TCP/IP is the common transport mechanism, allowing seamless connectivity between internal networks and the Internet. HTTP is used for data flow control.
SOAP (Simple Object Access Protocol) has dominated efforts to provide a consistent control interface between Web services. SOAP includes a number of specific HTTP headers designed to ease the flow of information through firewalls and proxies. It also includes an XML schema that forces Web services components to comply with the tightly defined parameter and exception values that facilitate communication between software modules. In many ways, SOAP is an improvement over the COM and CORBA modular application architectures that have become clumsy facilitators of integration between applications within local environments.
While COM and CORBA saddled developers with the not insignificant burden of carefully specifying inter-application interfaces, SOAP-based Web services obviate much of this requirement by using a significantly easier vocabulary based on standard Internet protocols and common methods for data representation.
If that sounds a lot like an API for tying an application into a standard environment, you're on the right philosophical track. Instead of providing an interface into a specific program, however, Web services encapsulate discrete business processes—hiding the specific back-end functionality and making it accessible to other applications through a raft of standardized connectors.
Web services are effectively the next step in the evolution of object-oriented development, which allowed developers to standardize module interfaces within an environment, but failed to provide a viable method for linking disparate environments.
"Object-oriented techniques like CORBA focused very much on procedure interoperability and nothing whatsoever on data interoperability," says Hal Stern, chief technology officer with iPlanet, the onetime Sun Microsystems-AOL joint venture that in March set out on its own to develop Sun's Open Net Environment (SunONE) Web services architecture.
"If I have complete gibberish, I can have all the objects I want but I won't be able to exchange data," Stern continues. Using existing OO models, he says, developers "ended up building an awful lot of context; there's not enough structure out of the box to make those things useful right away. By leveraging the Internet as a way of doing delivery, and by leveraging the Java environment as a way of providing a robust development and deployment platform, we overcame some of those. We've reduced the tare weight of [OO development]."
By taking over many of the arcane aspects of data exchange and presentation, Web services sugarcoat the challenge of truly distributed computing that has been so difficult to achieve in widespread practice. Hooks into standards-based, server-side logic should ease many once unfathomably difficult integration tasks. Business leaders, therefore, can use Web services to integrate applications based on well-understood business functions rather than having to get bogged down with the vagaries of distributed application mechanisms. Typical Web services components might include modules for adjusting inventory, paying contractors, calculating and lodging tax, and so on.
Wading into mUDDIer water
In early Web services implementations, modules will be hosted on a company's intranet and used to assemble and interact with local enterprise applications. This will be possible through use of WSDL (Web Services Description Language) and UDDI (Universal Description, Discovery, and Integration), two other evolving Web services standards that will eventually provide a mechanism for describing Web services and documenting the way in which applications interact with them.
WSDL describes the protocols and formats that each service uses, while the proposed WSFL (Web Services Flow Language) describes workflow relationships between those services. UDDI, on the other hand, provides for searchable directories of WSDL information, which will allow Web services-enabled applications to find and use the components they need.
While XML and SOAP define the inner workings of a Web service, UDDI is the real enabler for interoperable Web services. It will provide a means for software to search out and find the services it needs, weighing competing offerings to identify one that's best suited for the business's requirements.
As in your normal phone book, Web services are organized within a UDDI directory as white pages (by addresses and contacts), yellow pages (by industry classification), and green pages (where services are listed based on descriptions that include XML version, encryption type, and an XML Document Type Definition).
Use of UDDI within an enterprise environment may promote reuse of individual components by allowing disparate development groups to leverage the work and experience of other business units. Ultimately, this could enable far faster application development if, for example, an internal logistics organization published hooks into its system for other business units to use. In this way, Web services could ultimately prove to be an enabler for component reuse in ways that simple object-oriented development has never managed to achieve.
That's because object reuse within OO environments requires a working knowledge of each module's interfaces and design. WSDL, on the other hand, effectively provides a consistent way of documenting those modules and publishing that documentation in a readily accessible format.
Consistent with the hype that has accompanied Web services in general, starry-eyed vendors believe widespread public implementation of UDDI will eventually lead to the development of online marketplaces where access to Web services is bought and sold with impunity. The theory goes something like this: Vendors and business partners, having developed what they feel to be best-of-breed application components offering some novel take on a specific purpose, will decide to offer them to the global market by publishing them as Web services.
For example, an inventory management system needing to pass data to a standard shipping module could look up Australia Post's available Web services and pick the one corresponding to the type of delivery required. A payroll service provider, on the other hand, could expose its systems to the Net for trusted (and paying) customers to link to directly. If the application can't decide which is best, a UDDI browser will allow humans to manually search, compare, and specify services to be used based on listings of the services contained therein.
Because they'll be hosted by the Web service provider, use of the modules will involve all manner of extra due diligence to address quality of service, disaster recovery planning, and even contingency plans should the Web services supplier go out of business. It's like the application service provider (ASP) delivery model married equally unsuccessful B2B exchanges and produced Web services as their offspring.
A good example of the applicability of Web services can be found in the ATO's recently announced Australian Business Register (ABR) project, which builds on Microsoft .NET infrastructure to provide a centrally managed platform for business authentication that will be made available as a Web service to other government departments and relevant organizations. Two years in the making, ABR is not completely finished and has not been publicly discussed. An ATO spokesperson, somewhat irritated that Microsoft and Accenture had spoken publicly on the project, declined to comment on it for this article.
Maintaining a central register of business information, accessible to trusted outside parties' applications on an as-needed basis, is an excellent example of the way that Web services can hide the complexity of an organization’s underlying infrastructure whilst allowing it to be polled for relevant information.
Of course, the ATO is a known and trusted entity; buying access to modules from smaller or unknown organizations introduces a whole new range of issues.
"More and more we rely on modules to do stuff," says Peter Pritchard, Asia-Pacific regional manager with development and testing tools giant Compuware. He believes necessity will drive Web services' earliest adopters.
"Vendors are going to get a reputation for providing good, solid modules that work. You do a lot of things that have no competitive advantage, and I think SMEs are going to do this first because they often can't afford to build entire applications themselves."
While it's reasonably well understood in theory, UDDI is far from completion. A number of vendors, including Antarcti.ca Systems, Cape Clear Software's ClearConnect, Mind Electric GLUE, Systinet WASP, and the open-source jUDDI, now offer UDDI development tools. For managing UDDI registries, look to Peregrine Systems, RealNames, ResolveNet, and Riskebiz Internet Services. Sun will release its first UDDI product soon.
Such products may provide early frameworks for implementing UDDI directories but do little to address concerns about Web services security. That’s because the standards for enabling Web services security are still largely unwritten. In April, Microsoft, IBM, and VeriSign finally debuted the first: WS-Security, which enables encryption of data being passed between Web services and integrates SOAP with the W3C's existing XML Security and XML Encryption standards.
Microsoft and IBM plan to release six more Web services security standards in the next 18 months. WS-Policy, WS-Trust, and WS-Privacy will allow data to be certified based on the sender's identity or internally defined business policies whilst meeting privacy obligations.
WS-Secure Conversation, WS-Federation, and WS-Authorization will provide a high-level security interface designed to promote seamless communication between secure environments.
No matter how good the security standards may be, potential Web services customers must work particularly hard to make sure their data remains safe at every point within the loosely affiliated universe of Web services that are envisioned to make up next-generation applications.
Although this naturally requires considerable technical skill, it's also going to demand involvement of business experts with the knowledge to build robust security. And this, as is always the way, will take considerable time and effort.
"Web services absolutely will create new security weaknesses. These services are not being designed by bankers," James Molini, chief executive of security firm Brink's Internet Security, recently told CNET.
"Many services we see, especially those built by smaller firms, are not actually built using real financial security people. As a result, sometimes they don't really know how to even comply with federal regulation regarding the security of their system."
Predictions about UDDI's future—and the fact that vendors are promoting Web services even though they're still very much works in progress—suggest the current hype over the technology is driven more by vendors desperate to escape the current downturn than by genuine customer need.
"We're very close to being there," concedes Doug Tidwell, IBM's Web services evangelist. "One of the main ideas behind Web services is that they give you a fairly lightweight way to move structured data from one place to another. It's the next logical step for XML, and we're working very hard to make sure there are standards that people can work to."
"There are still some issues," he adds optimistically, "but it's much better today than it was six months ago. The next step for us and the rest of the industry is the [UDDI] registry piece, where there's a lot of work to be done. Right now, the main thing we're doing is encouraging people to just start working with this."
Although this conceded level of immaturity might introduce enough uncertainty to have risk managers shaking in their boots, developers are rapidly warming to the ideological potential of Web services. Indeed, many have already spent long days building ad hoc Web services around XML and SOAP, which are far more mature than the more fantastic aspects of Web services such as UDDI.
Just which vision of Web services wins out in the end is anybody's guess. Although J2EE proponents are quick to tout that environment's benefits, Microsoft's heft in development circles—and its commitment to support a variety of development environments—has won considerable interest among those investigating Web services.
A recent survey of 813 North American developers by Evans Data Corporation showed that .NET had 28 percent of the market while J2EE was neck-and-neck at 27 percent. “Custom code" ranked third at 11 percent, while proprietary EAI products were named by just 4.2 percent of respondents. Fully 82.5 percent of respondents planned to use XML interfaces in at least some of their applications, and 19 percent will use XML in all their applications.
Another survey, this one conducted by Merrill Lynch, confirmed Evans Data's results. Two-thirds of the 100 European CIOs surveyed report investing in Web services, but their allegiance is evenly split between .NET and J2EE.
Sun, true to form, has been openly critical of Microsoft's strategy for enabling Web services development. Sun's Stern, for one, criticizes C# for what he sees as a relatively lax security model that lets developers circumvent built-in protections if they know what they're doing: "these are programming conveniences, but a disaster for people who manage deployments. We're trying to drive some developer discipline here."
Another Evans Data Corporation survey found Microsoft's C# language was already being used by 12 percent of North American software developers, up from 7 percent of developers six months earlier. The language's penetration is expected to pass 24 percent within the next year, portending massive acceptance of .NET as a concept and C# as the facilitator of that concept.
Just what effect the introduction of a major new software infrastructure will have on businesses remains to be seen. Certainly, the facilitation of full distributed development—and the potential integration of external code as necessary—will force companies to become even more stringent with their development and testing discipline.
Development industry guru Grady Booch, the long-serving chief scientist at XML and tools developer Rational Software, believes Web services are pointing developers in the right direction despite the attendant complexity they introduce.
"Software development is fundamentally, wickedly, pervasively hard, and no one technology is going to change that," Booch says.
"Reuse at the small scale has always been oversold as a benefit of any coding technique; at best, we can whack away at some inefficiencies, which means doing more model-driven developments. There's sound technology behind [Web services], which I see as a natural progression in raising the level of abstraction of systems. Business experts are very well versed in arcane rules of [business], but you'll never train them to be programmers. Instead, Web services let you bring the code up to them."
Reality rains out the Web services party
Interoperability, standardization, modularization, and distributed computing are all noble causes, and analysts have universally lauded the efforts of Web services vendors in bringing the cause forward so far. At the same time, all agree that it will be some time before companies start seriously using the stuff. That’s because real applications simply aren't built like this. Application interfaces must be coded, data interoperability assessed, scalability tested, and configurations optimized. Source code must be examined, optimized, and upgraded. And, while it may be no small technical accomplishment, depending on external and unknown Web services providers for any important business component is a risk that would set off alarm bells during any due diligence process.
The alternative, establishing vendor trust before integrating modules, doesn't make sense since it would obviate the "discovery" phase of UDDI and could be better addressed by simply installing the software internally.
Businesses want stable, predictable, and well-understood applications that will consistently perform a specific function as they expect it to. So while Web services may have caught the attention of developers—Evans Data Corporation found that more than half of respondents planned to develop XML Web services this year—business managers have been far less enthusiastic.
"Customers are more preoccupied with running existing IT and business alignment than they are concerned about investigating every new bit of technology," says Merv Langby, chief services analyst with IDC Australia. He believes Web services are a critical enabler of distributed computing but cautions it could take 18 months or more before most companies will seriously consider them.
"Vendors continue to put a disproportionate emphasis on the technology sale, as opposed to articulating the value proposition in business terms. Web services is the best philosophical link to allow businesses to maintain the best of what they've got whilst using the best of what is new. But customers tell us time and again they're in no way keen to give up an established line of business applications. If people don't buy technology for its own sake, they certainly don't buy technology architectures for their own sake."
Therein lies the biggest problem with Web services: for now, they lack enough clear business benefit to justify their risk, or the investment in time and development resources that they necessarily involve.
These days, moving to Web services is like taking the first swig of a new batch of moonshine from the still: It might give you a great buzz, but it might also blind you. If you aren't keen to hand a bottle of this potentially lethal brew to your managers, avoid the temptation to let the Web services hype make you promise too much. Articulate a clear business case, and maintain a healthy sense of skepticism that puts considerable weight on the fact that Web services are still very much a work in progress.
Go into it with your eyes open, avoid over committing too early, and regularly reassess your technology platform to make sure Web services are delivering according to your expectations.
"People are seeing Web services as a technology answer, but they haven't asked the business question it's supposed to be answering," says Compuware's Pritchard.
"Realize that if you're doing this now, you're either an innovator or an early adopter. There's got to be a plan for it, including risk analysis and management and fallback positions. These are all business decisions, and not pure IT decisions. You really have to ask yourself why you're doing it."
Plan your move into Web services
SOAP and other Web services standards may get better every day, but diving into them too soon could be as much fun as jumping into a cold shower on a chilly winter morning. Although it's probably too early for most companies to go full bore into Web services, there are a few things you can do to ease your own efforts.
Don't SOAP yourself up unnecessarily
Don't listen to the vendors. Web services are still evolving; you have not lost out to competitors by not implementing them yet, and you don't have to rush into them early just to say you got there first.
Hold off for a while, watch the experience of others, and don't be afraid to tell vendors to go away until they can offer more than technological novelty.
Start very, very small
No matter how enthusiastic the vendors are, for now Web services are going to come into most organizations at a trickle rather than flood. Pilot testing in a small part of the business is always a good idea, but it's doubly so when the potential pitfalls and benefits of Web services are still becoming apparent even to those that build them.
Choose your partners carefully
Web services vendors may have worked to make the transition as seamless as possible, but most companies are going to need outside help to ensure a smooth implementation. Given the technology's novelty, it will be some time before internal developers have climbed high enough on the learning curve to make it work effectively.
Look to traditional consulting and implementation partners, many of whom have close ties with vendors and have been busy building internal Web services expertise, to help with strategic planning and provision of the skills to get you where you want to go.
Working with .NET experts at Avanade, says Taronga Zoo IT and planning manager Jenny Vasseleu, "has made the world of difference to us; we're quite comfortable going ahead with these projects knowing we've got that support."
Building your own skills
Once your pilot program is successful, the right amount of trumpeting should gradually win more interest from both technical and business interests. Choose carefully, but work to rapidly expand your internal skills base so that Web services can grow organically throughout the organization.
Involve business leaders
One of the biggest potential mistakes when implementing Web services is to treat them as a technology upgrade alone. The best Web services implementations will use technology to provide easily accessible hooks into complex business functions; scoping, redefining, and delivering those business functions requires the intimate involvement of business leaders who can step away from the technology to identify areas of potential improvement.
Aim for a quick ROI
Web services are new technologies, and management will be rightfully skeptical of vendors' guarantees. Try scoping any implementation to cover just one or two specific business processes whose improvement will deliver measurable benefits within around six months. Larger, more ambitious projects—particularly those that integrate commodity services from outside providers—will flow from there.
You can implement Web services in a small way or go whole hog. Either way, it's critical to manage the project with stringent configuration management, change management, project management, version management, and other facets of quality control.
"You need to have a lot of internal rigor around the way you manage the architecture," advises Avanade's Chrimes. "Web services don't give you the right not to do standard IT maintenance and methodology, and they don't give you the right to skip business risk management either."
Don't spread your wings too fast
One of Web services' mooted promises is the ability to seamlessly link application modules over the Web. The challenge of finding and assessing various modules—the realm of still nascent UDDI and WSDL—is big enough. But even when that's possible, you can still suffer business interruption unless you've adequately considered strategies for managing unexpected downtime, Internet congestion, and other issues that aren't anywhere near as problematic over private networks. Such issues will seriously hamper broader Web services deployment until the Net's underlying technology improves.
Expect to write code
New development tools can hide a lot of the pain of developing Web services, but even they cannot eliminate the need for you to write and test code—lots of it. Vendors will tell you Web services can do OO development one better by allowing reuse of previously built modules, but who really wants to rely on code that's months or years old? Strive for continuous improvement in your software, and don't get lazy just because the tools make it possible.
Web services will not cure cancer
They won't automatically fix your application infrastructure, either. For the near future, Web services will remain complex, ill defined, ambiguously implemented, and subject to disproportionate amounts of vendor politicking and hype. Don't expect too much and you won't be disappointed.
"There is no silver bullet," cautions Rational Software chief scientist Grady Booch. "Bullets aren't even the right ammunition to be using."