Efficient delivery of dynamic Web pages remains a challenge for Internet developers, especially when moving static HTML pages to CGI. I will review some performance numbers I obtained from testing three CGI strategies during the generation phase of page delivery.
Candidate CGI technologies
My testing assessed three common Perl techniques for page generation performance:
- HERE documents
- CGI.pm module
Here's a brief overview of what each technique entails.
HERE documents are described in the perldata man page under the heading Scalar Value Constructors. They allow plain text to be embedded in a file that otherwise contains only Perl code. Since HTML is represented as plain text, HERE documents offer a direct solution. They also support variable interpolation, meaning that the plain text can contain Perl variables. Here's a quick example:
my $color = "red;
This car is $color.
This script prints, "This car is red." The most common criticism of HERE documents is that they make scripts harder to read. Having large HERE documents scattered throughout Perl code interrupts the normal flow of statements.
A somewhat cleaner solution is to templatize the original HTML. The HTML stays in a separate file but contains extra instructions, usually in the form of custom XML tags. The HTML file is read by the CGI program, and when those custom tags are detected, they are replaced with dynamic content. This is the approach taken by the HTML::Template module, available from the CPAN Perl archives. The following code snippet shows HTML templatized:
This car is <TMPL_VAR NAME="COLOR">.
And here's the Perl code to process it:
my $template = HTML::Template->new(filename=>"temp.html");
The value of the COLOR template variable, "red," replaces the <TMPL_VAR> tag. Templates present a cleaner solution than HERE documents because the code stays in one file, data in another. Read the HTML::Template man page for more information on this module. Many server-side technologies, from SSI to ASP, use a template strategy.
One of the earliest CGI solutions to rear its head was the eponymous CGI.pm. This module treats HTML tags as code, and for every HTML tag there is a corresponding Perl subroutine. You can specify the HTML without straying from normal Perl syntax. CGI.pm is the opposite of HERE documents because everything is Perl code. The sample page implemented using CGI.pm might look like this:
use CGI ":standard";
my $q = new CGI;
print $q->start_html, "This car is red", $q->end_html;
Of course, there's a lot more to these modules; we're just staking out the territory for testing.
What this test excludes
In my testing, I planned to find out what styles of CGI program are efficient generators of Web page content. To do that, I needed fair and clearly specified performance metrics, which meant excluding all factors that distract from the performance measurements I sought. One factor I excluded was page load time. Several conditions influence page load time, such as client computer speed, caching, network latency, and Web server configuration. Here are some other issues that influence performance.
Database performance and network latency
Database access is a major constraint of dynamic page generation and shouldn't be tossed aside lightly. However, for this test, any database access had to be the same for each of the three technologies. Removing database access took away an irrelevant performance penalty and made the test scripts simpler to create. So in this case, database access was stubbed out (not done). If page generation time turned out to be trivial compared to database access time, at least it would be known for sure, and later performance analysis could shift back to the database side.
Common sense dictates that network latency and database access are real performance constraints on CGI-generated Web pages, but you can test only one thing at a time. For a Web server that's heavily CPU-loaded, the page generation phase can still be the area that needs performance tweaking first.
Perl scripting environment
One aspect that is relevant to page generation is the Perl scripting environment. For a Web server, CGI Perl scripts can run as standalone processes or inside Apache's mod_perl module. With mod_perl, scripts are reused without being reinitialized. Mod_perl presents a faster execution environment than standalone Perl. Since performance is the issue at hand, this is an opportunity to see the benefit of mod_perl. Cases with and without mod_perl need testing.
Disk access and caches
There is also the old bugbear of disk access and caches. Disk drives are so slow compared with RAM access and CPU speeds that the smallest interference from the disk can wildly distort any performance information. For a busy Web server, disk access is not an issue—all the relevant information should be cached in memory. If database access or cycle time (start-stop time) were issues, disk access might be a consideration. It shouldn't be for the normal so-called steady state system.
And finally, this isn't a full system-wide application load test. We're just trying to find a better technology choice at this point, and you have to start somewhere.
Running the test
The test equipment involved an IBM-compatible PC running Linux. The hardware baseline included:
- An AMD Duron 900-MHz CPU.
- One 20-GB IDE disk drive (10+ GB free).
- 512-MB PC133 SDRAM (256+ MB free).
- 100-Mb Ethernet card on a two-computer LAN.
The software baseline was:
- RedHat 7.2 Linux (kernel 2.4.9).
- GNOME 1.4.
- Perl 5.6.0.
The Linux kernel had a buffer cache of default size. Perl was installed with the standard @INC set of directories, and module HTML::Template was also installed. Vnc was used remotely as a virtual X-Windows console and was running at all times. The only X-Windows clients were xterm sessions.
During testing, I used vmstat to ensure that at no time did the computer page, swap, or flood the Linux disk buffer cache. Before each test run, all interpreters, scripts, and files were cached to /dev/null. These steps were enough to ensure that everything needed was cached and that no perfidious disk access occurred during testing.
You can download the zipped archive of files attached to this article for the scripts (named gen1, gen2, and gen3.pl). I used a standard HTML page for each one. This HTML page was about 5 KB and required dynamic addition of two headers and a set of table rows. The file Test.html contains an example of a generated page. Since the page was not displayed in a browser, it mattered little whether the HTML was perfect or not.
I was after two statistics (performance metrics): total memory used and total CPU time spent. Calculating total memory used gets tricky fast for UNIX. For these simple tests, I used top(1)'s RSS figure for memory; for time tests, I used output from time(1).
All page generation runs were repeated 1,000 times to produce readable multisecond statistics. For standalone tests, the scripts were restarted each time. For mod_perl tests, the scripts were started only once, but all their logic was enclosed in a 1,000-fold loop. The "b" versions of the downloadable scripts match the mod_perl test runs. Try them yourself.
I conducted six tests: three for standalone CGI and three for mod_perl CGI. After these runs, I added two runs for a total of eight. Table A outlines the results.
Clearly, HERE documents are the fastest, smallest solution. Use of CGI.pm is both slow and memory intensive, which accounts for its poor reputation. The figures for HTML::Template lie between the others and bear some explaining.
Normal use of HTML::Template involves a "prepare" and an "output" phase. You can proceed with these phases via the not-so-good or the good way. The not-so-good way is to reprepare each time the script is called—that's the (b) case in Table A. The good way is to prepare only the first time—the (c) case. Choosing (b) or (c) makes no time difference running CGI standalone, but in the case of mod_perl, the time difference is dramatic. An efficient algorithm is obviously as important as an efficient technology.
From these results, it appears that anything beyond a HERE document means a substantial processing overhead for CGI. Perhaps the time overhead of HTML::Template can be further trimmed by following the caching advice in its man pages. Then, a good engineering compromise could be struck between readable code and performance. HERE documents are seriously lacking in the readability department.
Web page generation may not be as expensive as network latency or database access, but it still consumes a great deal of resources in building a CGI page. The right technology choice can make a dramatic difference to the resources consumed. In these tests, you've seen that memory requirements can be doubled or halved by the technology choice. Choosing mod_perl over standalone Perl can mean a 10x to 20x processing improvement. Even with mod_perl in place, processing time can vary by a factor possibly as large as 20.