Today's Web development managers are more challenged than ever to find the appropriate tools and technologies for their mission-critical Oracle databases. With the explosive interest in e-commerce, end users have high expectations. They demand systems on the Web that can support thousands of concurrent transactions per second while at the same time providing subsecond response time.
Because Oracle dominates the market for Web-enabled databases, it is the task of the Oracle manager to create a Web architecture that satisfies these end-user requirements while at the same time creating a flexible and scalable Web architecture.
Many of the world’s top e-commerce systems use Oracle databases. Behemoths such as Amazon and eBay use Oracle databases and support mindboggling transaction rates. And they make it look easy. But the reality is that a tremendous amount of manual effort goes into creating a Web architecture that provides a high-performance and scalable solution for Oracle databases on the Web.
This article explores several of the most important issues involved in choosing a scalable Oracle Web architecture and describes some of the choices made by successful e-commerce companies when implementing Oracle databases over the Web.
Basic Oracle tools for an e-commerce system
Oracle has a wealth of tools that can be used to facilitate the development of Web-based systems. These products include:
- Oracle Internet Application Server (iAS)
Oracle9i iAS is the latest incarnation of the Oracle Web Server product. Oracle iAS claims to provide a complete framework for the interception, routing, and management of incoming data requests from the Internet. Oracle9i iAS competes with freeware Web server alternatives such as Apache.
- Oracle Portal
This is the successor to the popular Web database tool that Oracle used to facilitate the easy creation of Web pages that access Oracle data. Oracle Portal competes with mainstream portal products such as Plumtree and Digital Dashboard.
- Oracle XML
Oracle now offers a suite of XML tools to allow for the parsing of incoming XML messages and the reformatting in sending of XML messages. These XML interfaces facilitate communication between portals and include utilities to parse, validate, and generate XML.
In sum, Oracle has provided a complete suite of tools that can be used for all phases of e-commerce development.
The Oracle product suite or a best-of-breed solution?
The first part of creating a Web infrastructure is deciding whether to use the Oracle suite of tools or to develop a custom solution with third-party tools. Proponents of the integrated suite approach note that Oracle will fully support all of your Web architecture. Proponents of the best-of-breed approach argue that many of the non-Oracle products are faster and easier.
Have you faced this choice between using all Oracle products or selecting other products to use with Oracle? How did you decide? Tell us about your experiences.
The Oracle Web architecture
Regardless of product choices, the entire e-commerce environment must be flexible, maintainable, and scalable. Many of the top e-commerce companies are choosing a four-tiered architecture to support their Oracle databases. The four-tiered architecture isolates processing requirements at each level and provides independence for each layer (see Figure A). Each layer provides data caching and performs load-balancing to ensure that no individual component is overwhelmed.
This architecture is almost infinitely scalable because additional Web servers and application servers can be added as volume grows. The Oracle database is also expandable, and additional Oracle resources can be added as well. For some Oracle databases, real application clusters can be used so that many Oracle regions can access a single database. Here are the central components of a four-tiered Oracle e-commerce architecture.
An e-commerce site must have enough listener processes so that a single port is not overwhelmed with incoming requests. Web listeners are simple processes that listen on a specific port and forward all incoming requests to the least-loaded Web server. It is imperative that the Web listeners have load-balancing intelligence so that a single Web server is not overloaded with work.
It is the job of the Web servers to manage the flow of the HTML and XML. On the incoming end, the Web server validates and parses incoming XML strings. For outbound transactions, the Web server takes data from the application server and creates the outbound HTML pages or XML strings. When an incoming transaction requests services, the Web servers forward the transaction to the appropriate application server. The Web servers also manage load and forward requests to the least-loaded application server. Many of today’s e-commerce Web servers are custom written, borrowing heavily from pieces of the Apache freeware code.
The Web server layer is the most challenging layer of any Oracle Web architecture. In addition to facilitating the routing of incoming Internet requests, the Web servers are also charged with performing all translation formatting operations. As such, the Web server must be able to cache as much information as possible and interface with other ancillary tools, such as XML parsers and HTML translation tools.
Application servers manage all processing related to a specific transaction. All business logic is contained at this layer as well. Whenever a request is made to Oracle for data, the application server funnels the request to a series of prespawned shadow processes that hold a persistent connection to Oracle. By preestablishing the connections between the Web server and the Oracle database, the overhead of reestablishing thousands of connections per second is bypassed. Most of these systems also utilize Oracle's multithreaded server, or MTS, which allows for the sharing of Internet memory for such operations as sorting.
At the Oracle database layer, the Oracle relational database management system (RDBMS) must be able to support all incoming requests from the application servers. The Oracle database is designed to scale to thousands of transactions per second, but there is a significant amount of manual tuning required to support large data volumes. This tuning involves the resolution of disk bottlenecks, RAM memory overloads, and CPU shortages on the database server.
Scaling the Oracle database for e-commerce
Now that we’ve discussed the functions of each layer in the four-tiered architecture, let’s take a closer look at managing Oracle in a Web environment.
Savvy database managers do a number of things to ensure that their Oracle database performs at optimal speeds to support high volumes of Web traffic. To support large e-commerce loads, many Oracle administrators implement the following Oracle scalability features:
- Install a multithreaded server
By using a multithreaded server, the Oracle database can utilize internal memory within the Oracle region called a large pool. The multithreaded server also allows for many thousands of end users to connect through response dispatchers. Each dispatcher within the Oracle database can spawn many subtasks to handle high volumes of incoming connections.
- Use very large RAM data buffers
Most Oracle8i and Oracle9i databases utilize very large buffer pool storage and cache as much of the database information as possible. The goal is to minimize disk input and output (I/O) by having as much of the salient information stored in RAM as possible. With Oracle's new scalability features, it's not uncommon to see data buffers grow anywhere from 10 GB up to 50 GB or more, effectively caching the most important and most frequently referenced information from the Web server. Using this approach, the Oracle database does not need to do any unnecessary I/O, and incoming Internet requests for information simply perform in memory transfers from the Oracle data buffers, then go to the Web servers, and finally back across the Internet to the originating site.
- Presummarize aggregate information
Any Oracle e-commerce systems that are required to summarize your aggregate information use the Oracle facility called materialized views. With materialized views, it is possible to precalculate aggregate information and store it in intermediate tables, which are transparent to the Oracle SQL. Whenever an incoming request desires an aggregation calculation, Oracle rewrites to query the precalculated aggregates instead of recomputing the information. This provides the commerce system with the illusion of extremely fast aggregation capabilities.
- Store HTML pages within the Oracle database
Many high-volume Oracle e-commerce systems choose to store preformatted HTML text in one of Oracle's numerous data types that support large objects. These data types include character large objects (CLOBS). Oracle takes these large objects and stores them either offline within the Oracle data files or offline using file linkages. By using this technique, the Web server is relieved of the tedious chore of having to redefine all the outgoing HTML upon a request from the end user. The preformatted HTML makes its way from the Oracle data buffers to the Web server cache, where all variables are parameterized. When incoming data requests a specific HTML page, symbolic substitution takes place within the cached version of the HTML page on the Web server, and a complete HTML page (with the Oracle data embedded into the HTML) is shipped back across the Internet.
Of course, it’s impossible for one article to fully cover all of the area of Oracle management for Web systems. But this overview has introduced several important architectural considerations for constructing a robust Oracle system over the Web. It is only with proper management and planning that your Oracle database will be able to scale to large data volumes while remaining flexible.