When Java developers need a blueprint for the “best” way to design and build a J2EE application, they turn to Sun’s  Pet Store demo application. Pet Store serves as a reference model showing the best practices for architecting and building the ultimate J2EE application. In November of 2001, Microsoft decided to open its own pet shop, only this time in C# and .NET. After benchmarking the .NET Pet Shop against Oracle’s published performance results for its own J2EE Pet Store implementation, Microsoft pronounced the .NET implementation to be superior in terms of performance, implementation costs, and developer efficiency. In doing so, quite predictably, Microsoft has created a firestorm of controversy.

What’s the big deal?
Since the initial announcement, Oracle and Microsoft have both issued rebuttals. Even IBM has weighed in with comments on its WebSphere application server, which was mentioned in the .NET Pet Shop white paper. Most of the contention seems to revolve around these key issues:

  • Microsoft’s gauging developer efficiency by counting lines of code is an attempt to objectively measure a subjective quality of the design.
  • The .NET implementation was performance-optimized for SQL Server through the use of stored procedures, while Sun actively discourages database tuning in the J2EE implementation in the interest of portability.
  • Both sides have called the other’s cited implementation costs into question.
  • Naturally, the underlying platforms differ: Oracle’s benchmarking was performed using a three-tiered Solaris system, while the .NET Pet Shop was benchmarked on a three-tiered Windows 2000 system.

Going straight to the source
In an effort to get some perspective on the controversy, I decided to go straight to the source. I tracked down Scott Stanfield, founder and CEO of Vertigo Software, the development house Microsoft asked to build the .NET Pet Shop, and asked him some questions about his experiences developing the project.

Scott Stanfield, CEO of Vertigo Software

Builder.com: How did you get involved with the .NET Pet Shop project? Did you set out to prove anything?
Scott: Given our extensive experience in benchmarking, having worked on the FMStocks 2000 and the Nile Bookstore projects, Microsoft asked us to design and to implement a “best practices” .NET application that contained the same business requirements as the Java Pet Store blueprint. While we didn’t set out to prove anything, we had two key goals during the project: Implement the Java Pet Store in .NET with an eye toward reducing the amount of code as a whole, and run the Oracle “challenge” to see how fast this thing in .NET performs.

Builder.com: What do you think of all the controversy your project stirred up? Did you anticipate any of it?
Scott: I’m happy, because without it, you wouldn’t be writing this article. We figured there might be controversy over the line count program; that’s why we published the source code. You’d think it would be an easy thing, to count lines, but what really constitutes a line of code? We came up with guidelines and heuristics that we thought were fair and would stand up to scrutiny, then wrote the app. It’s fully documented.

Builder.com: About counting the lines of code—do you think that is a fair measurement of developer productivity, considering that most developers use some kind of code-generating wizard or IDE?
Scott: You have to think of it in terms of total cost of ownership. Any line of code you don’t write is a line of code you don’t have to maintain, document, debug, and test. Considering that, is the Java code, being four times as long, four times as expensive? It all depends, but it’s a big enough point to consider carefully.

Builder.com: How long did the project take?
Scott: We did the whole thing with two developers in a total of five weeks, which includes about three to four days of load testing and benchmarking at the end.

Performance versus portability
Builder.com: The .NET implementation tested 28 times faster than the J2EE implementation. Were you at all surprised by how much of a performance difference you discovered?
Scott: Very surprised, especially since we didn’t do any tuning to the code, with the exception of using the single-line OutputCache directive on some of the pages. I was expecting about a five times performance difference.

Builder.com: What about the stored procedures and SQL Server optimizations?
Scott: Here’s the deal: Sun held up Pet Store as its blueprint for implementing best practices to build Web applications. The .NET version should be held to the same standards: It should serve as a template for people building Web applications in .NET, and the preferred way of building non-ad-hoc queries for SQL Server is to use stored procedures.
Stored procedures are a great way to encapsulate the relational table design behind a procedural-oriented, friendly function call. They’re easy to debug and don’t require the middle-tier developers to understand a more complicated join syntax. We could readily rewrite the app so it doesn’t use stored procedures, and my guess is that it would perform just as well. By doing so, though, we’d lose the DB encapsulation and spread table knowledge to the middle-tier.

Builder.com: So you aren’t buying the argument that using stored procedures in the .NET implementation makes the comparison unfair?
Scott: I’ve been using stored procedures with SQL Server since it was Sybase/Microsoft SQL Server 4.x running on OS/2. The best way to build apps for SQL Server is to use stored procedures. Why not take advantage of the best a platform has to offer?
The detractors say, “But then you’re tied to one database.” I say, “So what?” There are myriad other enterprise features you are probably using that would tie you to one platform anyway. The whole cross-platform Java issue is a red herring: If you’re doing J2EE, you’re either a BEA WebLogic developer or an IBM WebSphere developer. Rarely does code transfer between these platforms.
I’ve heard this argument before. We used to have to build lowest common denominator apps with a single source that would work both on Windows 3 and the Mac. And you know what? Those apps sucked.

Builder.com: Sun encourages J2EE developers to use the Model-View-Controller pattern to separate data and business logic from presentation, yet you didn’t use the MVC pattern in the .NET Pet Shop. Why?
Scott: We were able to achieve the same separation through the intrinsic nature of ASP.NET. The ASP.NET model supports the idea of “code-behind,” where the logic for the individual ASP.NET pages are actually contained in separate C# files.

Builder.com: Was there anything else in .NET’s toolset that you feel gave it an advantage over the Java toolset? Or did you find yourself reinventing something in .NET that Java already had?
Scott: No, we didn’t find ourselves reinventing anything in .NET, and we were able to leverage .NET in several areas to reduce the number of lines of code [over the J2EE implementation]. For example, we used Data Binding to avoid writing loops that iterate through the records in a result set.

Builder.com: Are there any plans you are aware of to possibly port Pet Shop onto Mono?
Scott: As far as I know, there are no plans to reproduce the test with the Mono project at this point.

J2EE versus .NET

Is code portability really all that important to you? Do the benchmark results affect how you think of these products? Send us an e-mail with your thoughts and opinions or post a comment below.