Web services: Messiah or mirage?

Software vendors keep telling us that Web services are the answer. But what is the question? ZDNet Australia explores the state of Web services today.

Software vendors keep telling us that Web services are the answer. But what is the question? Are there business drivers for using Web services, or are software vendors just trying to drive their sales?

Business and IT managers like to think in terms of distinct projects with clearly defined deliverables and a relatively certain return on investment. Vendors, on the other hand, are keen to preserve their revenue growth and are therefore somewhat loath to concede that things work well enough pretty much as they are.

Thus is the eternal struggle of innovation versus pragmatism, and the two are rapidly coming to the fore as software companies follow a now well-worn path by hyping Web services into the stratosphere.

This year, anybody who's anybody in the software world is building a Web services strategy that will dictate the composition of their next major version.

Customers are being exhorted to buy into the technology; new Web services standards, and improvements to existing ones, are rolling off the treadmill; observers are jumping on the bandwagon with heady market share and take-up projections that will no doubt prove over-ambitious over time, but nonetheless add further fuel to the fire that the concept of Web services has become.

Analysts have jumped on the bandwagon, spewing out grandiose projections about the potential of the Web services market that belie the torpor plaguing the rest of the IT industry.

In January Gartner, for one, predicted that Web services "will capture substantial attention" this year, and will "dominate" new application deployment by 2004.

The firm has also projected a US$21 billion market for Web services by 2005, with local analysts expecting Australian businesses to contribute nearly AU$1 billion by that time.

That's not bad for a technology that is still far from concrete or specific, and offers still uncertain benefits.

While other highly hyped technologies like CRM and ERP promised specific outcomes, Web services is little more than a collection of standards without a specific product to encapsulate them. It is a means to no particular end, rather like buying a box of Lego building blocks for a child and waiting to see what they come up with.

Taking sides

Because everybody recognises their importance, the Web services industry has become the latest battleground in the IT industry's perennial battle between dark and light, although this time the players are different.

Microsoft, which by now has become used to being universally distrusted because of its chronic desire to differentiate its products, has been working closely with IBM to rapidly develop consistent and workable Web services standards under the auspices of the 106-member Web Services Interoperability Organisation (WS-I).

"We're trying to take the Internet and make it into an enterprise-class platform, says Charles Stevens, vice president of Microsoft's Enterprise Partners Group.

"Customers like the building-block approach because it lets them pull these systems together. But we can't just talk the talk: we've got to prove that we can be a world-class integrator using XML across business-class platforms."

Microsoft's vision of Web services is based around its .NET strategy, which has been rapidly gaining substance with the recent release of its Visual Studio.NET development environment and the anticipated launch of its Web services-enabled Windows.NET operating system next year.

Then there's IBM, which has been heavily promoting both Linux and XML-based Web services, but favours the use of J2EE (Java 2 Enterprise Edition) as an environment for building and integrating Web services.

Sun, the creator of J2EE, has consistently refused to join the WS-I, although in May it finally conceded that doing so might be beneficial in its efforts to position itself as a Web services leader.

However, its offer to join was contingent on the creation of two new board positions that would put it on equal footing with IBM and Microsoft in determining the course of Web services.

IBM thinks Sun's Java expertise would be beneficial; Microsoft, typically sceptical of its ideological nemesis, was against Sun's membership.

As of publication time, the decision as to whether Sun would be admitted--much less whether it could secure itself two board seats, which are coveted by dozens of other WS-I members--was still up in the air.

With vendor sniping rampant and the final pantheon of Web services still being formulated, customers can be forgiven for remaining sceptical of the whole concept.

Political machinations aside, however, Web services will soon be a part of your life whether you want them to be or not. They will soon be ubiquitous within major operating systems, and within a few years all major enterprise applications will use Web services interfaces to interoperate.

If nothing else, Web services have the potential to end the headaches of application integration that continue to plague all manner of business.

Because vendors recognised the need for backwards compatibility early on, they've positioned Web services as both a way of developing completely new functionality, whilst integrating existing code developed previously in all manner of environments.

"The justification has to be to unlock the value that people have already got," says Mark Chrimes, managing director of Avanade Australia, which has already engaged several customers in .NET discussions and has helped drive early Web services implementations with customers such as the NSW Department of Employment and Workplace Relations and Sydney's Taronga Zoo.

"There's a lot of dissatisfaction from current implementations, but the cost of replacing legacy systems is phenomenally and prohibitively high. There are many more moving parts in the IT world now than before, and the real value of Web services is going to be in reducing the cost of development of those applications."

While openly praising the concept of Web services, however, analyst firms are universally recommending caution for customers considering investing in them just yet. That's because everybody agrees Web services are a Good Thing, but there is still no consensus on exactly what they are.

Add to this a lack of available products and a business world with far bigger concerns, and you've got a Web services market that even vendors concede is set to go through a period of disillusionment as customers cotton onto the fact that two simple words and a good idea won't be enough to improve their bottom line overnight.

Alphabet SOAP

Specifics of each vendor's view of Web services vary based on each company's ideological perspective, but it's safe to define Web services as online modular applications that use consistent standards to provide seamless interfaces between multiple and distributed business processes.

There are, necessarily, common elements to each vendor's vision. Because it has already been recognised as the most desirable standard for interoperable data representation, XML is accepted as the most desirable method of data representation.

TCP/IP is the common transport mechanism, allowing seamless connectivity between internal networks and the internet. HTTP is used for data flow control.

SOAP (Simple Object Access Protocol) has dominated efforts to provide a consistent control interface between Web services. SOAP includes a number of specific HTTP headers designed to ease the flow of information through firewalls and proxies.

It also includes an XML schema that forces Web services components to comply with the tightly defined parameter and exception values that facilitate communication between software modules.

In many ways, SOAP is an improvement over the COM and CORBA modular application architectures that have become clumsy facilitators of integration between applications within local environments.

While COM and CORBA saddled developers with the not insignificant burden of carefully specifying inter-application interfaces, SOAP-based Web services obviate much of this requirement by using a significantly easier vocabulary based on standard Internet protocols and common methods for data representation.

If that sounds a lot like an API for tying an application into a standard environment, you're on the right philosophical track. Instead of providing an interface into a specific program, however, Web services encapsulate discrete business processes-hiding the specific back-end functionality and making it accessible to other applications through a raft of standardised connectors.

Web services are effectively the next step in the evolution of object-oriented development, which allowed developers to standardise module interfaces within an environment, but failed to provide a viable method for linking disparate environments.

"Object-oriented techniques like CORBA focused very much on procedure interoperability and nothing whatsoever on data interoperability," says Hal Stern, chief technology officer with iPlanet, the onetime Sun Microsystems-AOL joint venture that in March set out on its own to develop Sun's Open Net Environment (SunONE) Web services architecture.

"If I have complete gibberish, I can have all the objects I want but I won't be able to exchange data," Stern continues. Using existing OO models, he says, developers "ended up building an awful lot of context; there's not enough structure out of the box to make those things useful right away. By leveraging the Internet as a way of doing delivery, and by leveraging the Java environment as a way of providing a robust development and deployment platform, we overcame some of those. We've reduced the tare weight of [OO development]."

By taking over many of the arcane aspects ofdata exchange and presentation, Web services sugar-coat the challenge of truly distributed computing that has been so difficult to achieve in widespread practice.

Hooks into standards-based, server-side logic should ease many once unfathomably difficult integration tasks.

Business leaders, therefore, can use Web services to integrate applications based on well understood business functions rather than having to get bogged down with the vagaries of distributed application mechanisms. Typical Web services components might include modules for adjusting inventory, paying contractors, calculating and lodging tax, and so on.

Wading into mUDDIer water

In early Web services implementations, modules will be hosted on a company's intranet and used to assemble and interact with local enterprise applications.

This will be possible through use of WSDL (Web Services Description Language) and UDDI (Universal Description, Discovery, and Integration), two other evolving Web services standards that will eventually provide a mechanism for describing Web services and documenting the way in which applications interact with them.

WSDL describes the protocols and formats that each service uses, while the proposed WSFL (Web Services Flow Language) describes workflow relationships between those services. UDDI, on the other hand, provides for searchable directories of WSDL information, which will allow Web services enabled applications to find and use the components they need.

While XML and SOAP define the inner workings of a Web service, UDDI is the real enabler for interoperable Web services. It will provide a means for software to search out and find the services it needs, weighing up competing offerings to identify one that's best suited for the business's requirements.

As in your normal phone book, Web services are organised within a UDDI directory as white pages (by addresses and contacts), yellow pages (by industry classification), and green pages (where services are listed based on descriptions that include XML version, encryption type and an XML Document Type Definition).

Use of UDDI within an enterprise environment may promote re-use of individual components by allowing disparate development groups to leverage the work and experience of other business units.

Ultimately, this could enable far faster application development if, for example, an internal logistics organisation published hooks into its system for other business units to use.

In this way, Web services could ultimately prove to be an enabler for component reuse in ways that simple object-oriented development has never managed to achieve.

That's because object reuse within OO environments requires a working knowledge of each module's interfaces and design; WSDL, on the other hand, effectively provides a consistent way of documenting those modules and publishing that documentation in a readily accessible format.

Consistent with the hype that has accompanied Web services in general, starry-eyed vendors believe widespread public implementation of UDDI will eventually lead to the development of online marketplaces where access to Web services is bought and sold with impunity.

The theory goes something like this: vendors and business partners, having developed what they feel to be best-of-breed application components offering some novel take on a specific purpose, will decide to offer them to the global market by publishing them as Web services.

For example, an inventory management system needing to pass data to a standard shipping module could look up Australia Post's available Web services and pick the one corresponding to the type of delivery required.

A payroll service provider, on the other hand, could expose its systems to the Net for trusted (and paying) customers to link to directly.

If the application can't decide which is best, a UDDI browser will allow humans to manually search, compare and specify services to be used based on listings of the services contained therein.

Because they'll be hosted by the Web service provider, use of the modules will involve all manmanner of extra due diligence to address quality of service, disaster recovery planning, and even contingency plans should the Web services supplier go out of business. It's like the application service provider (ASP) delivery model married equally unsuccessful B2B exchanges and produced Web services as their offspring.

A good example of the applicability of Web services can be found in the ATO's recently announced Australian Business Register (ABR) project, which builds on Microsoft .NET infrastructure to provide a centrally managed platform for business authentication that will be made available as a Web service to other government departments and relevant organisations.

Two years in the making, ABR is not completely finished and has not been publicly discussed; an ATO spokesperson, somewhat irritated that Microsoft and Accenture had spoken publicly on the project, declined to comment on it for this article.

Maintaining a central register of business information, accessible to trusted outside parties' applications on an as-needed basis, is an excellent example of the way that Web services can hide the complexity of an organisation's underlying infrastructure whilst allowing it to be polled for relevant information.

Of course, the ATO is a known and trusted entity; buying access to modules from smaller or unknown organisations introduces a whole new range of issues.

"More and more we rely on modules to do stuff," says Peter Pritchard, Asia-Pacific regional manager with development and testing tools giant Compuware, who believes necessity will drive Web services' earliest adopters.

"Vendors are going to get a reputation for providing good, solid modules that work. You do a lot of things that have no competitive advantage, and I think SMEs are going to do this first because they often can't afford to build entire applications themselves."

While it's reasonably well understood in theory, UDDI is far from completion. A number of vendors-including Antarcti.ca Systems, Cape Clear Software's ClearConnect, Mind Electric GLUE, Systinet WASP, and the open-source jUDDI-now offer UDDI development tools. For managing UDDI registries, look to Peregrine Systems, RealNames, ResolveNet, and Riskebiz Internet Services. Sun will release its first UDDI product soon.

Such products may provide early frameworks for implementing UDDI directories, but do little to address concerns about Web services security.

That's because the standards for enabling Web services security are still largely unwritten. In April, Microsoft, IBM, and VeriSign finally debuted the first: WS-Security, which enables encryption of data being passed between Web services and integrates SOAP with the W3C's existing XML Security and XML Encryption standards.

Microsoft and IBM plan to release six more Web services security standards in the next 18 months. WS-Policy, WS-Trust, and WS-Privacy will allow data to be certified based on the sender's identity or internally defined business policies whilst meeting privacy obligations.

WS-Secure Conversation, WS-Federation, and WS-Authorization will provide a high-level security interface designed to promote seamless communication between secure environments.

No matter how good the security standards may be, potential Web services customers mustwork particularly hard to make sure their data remains safe at every point within the loosely affiliated universe of Web services that are envisioned to make up next-generation applications.

Although this naturally requires considerable technical skill, it's also going to demand involvement of business experts with the knowledge to build robust security. And this, as is always the way, will take considerable time and effort.

"Web services absolutely will create new security weaknesses. These services are not being designed by bankers," James Molini, chief executive of security firm Brink's Internet Security, recently told CNET.

"Many services we see, especially those built by smaller firms, are not actually built using real financial security people. As a result, sometimes they don't really know how to even comply with federal regulation regarding the security of their system."

Reconcilable differences

The extent of crystal ball-gazing about UDDI's future, and the fact that vendors are promoting Web services even though they're still very much works in progress, suggest the current hype over the technology has been driven more by vendors' desperate need to escape the current downturn than by genuine customer need.

"We're very close to being there," concedes Doug Tidwell, IBM's Web services evangelist. "One of the main ideas behind Web services is that they give you a fairly lightweight way to move structured data from one place to another. It's the next logical step for XML, and we're working very hard to make sure there are standards that people can work to."

"There are still some issues," he adds optimistically, "but it's much better today than it was six months ago. The next step for us and the rest of the industry is the [UDDI] registry piece, where there's a lot of work to be done. Right now, the main thing we're doing is encouraging people to just start working with this."

Although this conceded level of immaturity might introduce enough uncertainty to have risk managers shaking in their boots, developers have rapidly warmed to the ideological potential of Web services.

Indeed, many have already spent long days building ad hoc Web services around XML and SOAP, which are far more mature than the more fantastic aspects of Web services such as UDDI.

Just which vision of Web services wins out in the end is anybody's guess. Although J2EE proponents are quick to evangelise that environment's benefits, Microsoft's heft in development circles--and its commitment to support a variety of development environments--has won considerable interest among those investigating Web services.

A survey of 813 North American developers by Evans Data Corporation, released in April, showed that .NET had 28 percent of the market while J2EE was neck-and-neck at 27 percent.

"Custom code" ranked third, at 11 percent, while proprietary EAI products were named by just 4.2 percent of respondents. Fully 82.5 percent of respondents planned to use XML interfaces in at least some of their applications, and 19 percent will use XML in all their applications.

A survey by Merrill Lynch, also conducted earlier this year, confirmed Evans Data's results, with two-thirds of 100 surveyed European CIOs investing in Web services, but allegiance evenly split between .NET and J2EE.

Sun, true to form, has been openly critical of Microsoft's strategy for enabling Web services development. Sun's Stern, for one, criticises C# for what he sees as a relatively lax security model that lets developers circumvent built-in protections if they know what they're doing: "these are programming conveniences, but a disaster for people who manage deployments. We're trying to drive some developer discipline here."

In May, a separate Evans Data Corporation survey found Microsoft's C# language was already being used by 12 percent of North American software developers, up from 7 percent of developers six months earlier.

The language's penetration is expected to pass 24 percent within the next year, portending massiveacceptance of .NET as a concept and C# as the facilitator of that concept.

Just what effect the introduction of a major new software infrastructure will have on businesses remains to be seen. Certainly, the facilitation of full distributed development--and the potential integration of external code as necessary--will force companies to become even more stringent with their development and testing discipline.

Development industry guru Grady Booch, long serving as chief scientist within XML and tools developer Rational Software, believes Web services are pointing developers in the right direction despite the attendant complexity they introduce.

"Software development is fundamentally, wickedly, pervasively hard, and no one technology is going to change that," Booch says.

"Reuse at the small scale has always been oversold as a benefit of any coding technique; at best, we can whack away at some inefficiencies, which means doing more model driven developments. There's sound technology behind [Web services], which I see as a natural progression in raising the level of abstraction of systems. Business experts are very well versed in arcane rules of [business], but you'll never train them to be programmers. Instead, Web services let you bring the code up to them."

Reality rains out the Web services party

Interoperability, standardisation, modularisation, and distributed computing are all noble causes, and analysts have universally lauded the efforts of Web services vendors in bringing the cause forward so far. At the same time, all agree that it will be some time before companies actually begin seriously using the stuff.

That's because real applications simply aren't built like this. Application interfaces must be coded, data interoperability assessed, scalability tested, and configurations optimised.

Source code must be examined, optimised, and upgraded. And, while it may be no small technical accomplishment, depending on external and unknown Web services providers for any important business component is a risk that would set off alarm bells during any due diligence process.

The alternative--establishing vendor trust before integrating modules--doesn't make sense since it would obviate the "discovery" phase of UDDI and could be better addressed by simply installing the software internally.

Businesses want stable, predictable and well understood applications that will consistently perform a specific function as they expect it to. So while Web services may have caught the attention of developers--Evans Data Corporation found that more than half of respondents planned to develop XML Web services this year--business managers have been far less enthusiastic.

"Customers are more preoccupied with running existing IT and business alignment than they are concerned about investigating every new bit of technology," says Merv Langby, chief services analyst with IDC Australia, who believes Web services are a critical enabler of distributed computing but believes it could take 18 months or more before they're seriously considered by most companies.

"Vendors continue to put a disproportionate emphasis on the technology sale, as opposed to articulating the value proposition in business terms. Web services is the best philosophical link to allow businesses to maintain the best of what they've got whilst using the best of what is new. But customers tell us time and again they're in no way keen to give up an established line of business applications. If people don't buy technology for its own sake, they certainly don't buy technology architectures for their own sake."

Therein lies the biggest problem with Web services: for now, they lack enough clear business benefit to justify the risk they entail, or the investment in time and development resources that they necessarily involve.

These days, moving to Web services is like taking the first swig of a new batch of moonshine from the still: it might give you a great buzz-but it might also blind you. If you aren't keen to hand a bottle of this potentially lethal brew to your managers, avoid the temptation to let Web services hype make you promise too much.

Articulate a clear business case, and maintain a healthy sense of scepticism that puts considerable weight on the fact that Web services are still very much a work in progress.

Go into it with your eyes open, avoid overcommitting too early, and regularly reassess your technology platform to make sure Web services are delivering according to your expectations.

"People are seeing Web services as a technology answer, but they haven't asked the business question it's supposed to be answering," says Compuware's Pritchard.

"Realise that if you're doing this now, you're either an innovator or an early adopter. There's got to be a plan for it, including risk analysis and management and fallback positions. These are all business decisions, and not pure IT decisions. You really have to ask yourself why you're doing it."

Plan your move into Web services

SOAP and other Web services standards may be better every day, but diving into them too soon may be as fun as jumping into a cold shower on a chilly winter morning. Although it's probably too early for most companies to go full bore into Web services, there are a few things you can do to ease your own efforts.

  • Don't SOAP yourself up unnecessarily: Don't listen to the vendors. Web services are still evolving, you have not lost out to competitors by not implementing them yet, and you don't have to rush into them early just to say you got there first.

    Hold off for a while, watch the experience of others, and don't be afraid to tell vendors to go away until they can offer more than technological novelty.

  • Start very, very small: No matter how enthusiastic the vendors are, for now Web services are going to come into most organisations at a trickle rather than flood. Pilot testing in a small part of the business is always a good idea, but it's doubly so when the potential pitfalls and benefits of Web services are still becoming apparent even to those that build them.

  • Choose your partners carefully: Web services vendors may have worked to make the transition as seamless as possible, but most companies are going to need outside help to ensure a smooth implementation.

    Given the technologies' novelty, it will be some time before internal developers have climbed the learning curve enough to make it work effectively.

    Look to traditional consulting and implementation partners, who have close ties with vendors and have been busy building internal Web services expertise, to help with strategic planning and provision of the skills to get you where you want to go.

    Working with .NET experts at Avanade, says Taronga Zoo IT and planning manager Jenny Vasseleu (see sidebar), "has made the world of difference to us; we're quite comfortable going ahead with these projects knowing we've got that support."

  • Building your own skills: Once your pilot program is successful, the right amount of trumpeting should gradually win more interest from both technical and business interests.

    Choose carefully, but work to rapidly expand your internal skills base so that Web services can grow organically throughout the organisation.

  • Involve business leaders: One of the biggest potential mistakes when implementing Web services is to treat them as a technology upgrade alone.

    The best Web services implementations will use technology to provide easily accessible hooks into complex business functions; scoping, redefining, and delivering those business functions requires the intimate involvement of business leaders who can step away from the technology to identify areas of potential improvement.

  • Aim for a quick ROI: Web services are new technologies, and as such management will be rightfully sceptical of vendors' guarantees. Try scoping any implementation to cover just one or two specific business processes whose improvement will deliver measurable benefits within around six months.

    Larger, more ambitious projects--particularly those that integrate commodity services from outside providers--will flow from there.

  • Stay disciplined: You can implement Web services in a small way or go the full hog. Either way, it's critical to manage the project with stringent configuration management, change management, project management, version management and other facets of quality control.

    "You need to have a lot of internal rigour around the way you manage the architecture," advises Avanade's Chrimes. "Web services don't give you the right not to do standard IT maintenance and methodology, and they don't give you the right to skip business risk management either."

  • Don't spread your wings too fast: One of Web services' mooted promises is the ability to seamlessly link application modules over the Web.

    The challenge of finding and assessing various modules--the realm of still nascent UDDI and WSDL--is a big enough problem. But even when that's possible, you can still suffer business interruption unless you've adequately considered strategies for managing unexpected downtime, Internet congestion, and other issues that aren't anywhere near as problematic over private networks. Such issues will seriously hamper broader Web services deployment until the Net's underlying technology improves.

  • Expect to write code: New development tools may hide a lot of the pain of developing Web services, but even they cannot eliminate the need for you to write and test code-lots of it.

    Vendors will tell you Web services can do OO development one better by allowing reuse of previously built modules, but who really wants to rely on code that's months or years old? Strive for continuous improvement in your software, and don't get lazy just because the tools make it possible.

  • Web services will not cure cancer: They won't automatically fix your application infrastructure, either. For the near future, Web services will remain complex, ill-defined, ambiguously implemented, and subject to disproportionate amounts of vendor politicking and hype. Don't expect too much and you won't be disappointed.

    "There is no silver bullet," cautions Rational Software chief scientist Grady Booch. "Bullets aren't even the right ammunition to be using."