How to deploy scalable Web caching solutions

Whatever the size of your business, Web caching can help optimize performance and save bandwidth. And if you choose the right caching solution, it can grow as your network does without the need for expensive and time-consuming re-structuring.

Web caching provides for a means to store frequently accessed Web pages and objects on a system that's closer to the user than the Internet Web server hosting those objects. That speeds up performance when those objects are accessed again.

A few years back, the theory was that super-high bandwidth Internet connections would make Web caching unnecessary—but it didn't work out that way. Even the fastest fiber optic Internet connections, at 30-45 Mbps, are slow in comparison to typical LAN speeds of 100 Mbps to 1 Gbps, so performance is still an issue. In addition, caching increases availability since the cached copy of the object can still be accessed even if the hosting Web server is down or unreachable due to network problems. Caching can also reduce the cost of Internet connectivity if your company pays by the megabyte.

So it looks like Web caching is here to stay and gaining in popularity. Caching can be used to your advantage in even the smallest business, and good caching solutions will scale to the enterprise level.

Tips in your inbox
TechRepublic's free Strategies that Scale newsletter, delivered each Tuesday, covers topics such as how to structure purchasing, when to outsource, negotiating software licensing or SLAs, and budgeting for growth.
Automatically sign up today!

Caching concepts

There are two broad categories of Web caching:

  • Forward caching, where copies of Web objects from Internet servers that are frequently accessed by your users are stored on your local network.
  • Reverse caching, where copies of the Web objects on your own internal Web servers are stored on a proxy server at the edge of your network to increase performance for outsiders who visit your sites.

Web caches can be deployed at different levels:

  • Local cache: the cached copies of Web objects are stored on a local computer. Most popular Web browsers keep a cache of previously accessed objects by default. For example, Internet Explorer calls them "temporary Internet files." Copies cached locally are only helpful when the user accesses the pages frequently from the same machine.
  • Proxy cache: Proxy servers are separate machines that cache Web objects for multiple users/client computers in the organization. They are computers that sit between the client and the hosting Web server, and they are more efficient than local caches because when a Web object is accessed by any user or computer on your local network, the cached copy is then available to any other user/computer that wants to access that object, without going out to the Internet server to retrieve it. A proxy cache can be integrated with a firewall at the network's edge.

Microsoft's ISA Server includes both firewall and caching proxy server, as do BlueCoat's appliances. Caching servers can also be separate machines, running free caching software or commercial products such as:

Caching proxies are often marketed in appliance form factor, such as Cisco's Content Engines and Stratacache's Axcela series.

Scalable caching solutions

As your business grows, a single Web caching server may not be able to handle all the traffic or store enough Web objects. In that case, you can expand your caching solution to create a caching array—a group of caching proxy servers that work together to distribute the caching load across the group. This also provides fault tolerance in case a caching server goes down.

To operate in an array, your caching servers must be able to use protocols for communicating with one another, such as:

  • WCCP (Web Cache Coordination Protocol), which is used by Cisco caching products and open source proxies such as Squid.
  • ICP (Internet Cache Protocol), also supported by Squid and BlueCoat.
  • CARP (Cache Array Routing Protocol), which is used by ISA Server Enterprise Edition to manage failover and load balancing for arrays of caching servers.

CARP can support almost unlimited linear scale-out to meet the needs of fast-growing companies. When a server is added to or removed from an array, CARP adjusts dynamically and reassigns URLs to effectively distribute the load.

Caching arrays can be arranged in either a hierarchical or distributed architecture. With distributed caching, all of the proxy servers in the array are on an "equal footing" and the load is distributed across them. With hierarchical caching, proxies are configured in a chain where they are at different levels so that the servers or arrays connect to other servers or arrays that are closer to the Internet (those closest to the Internet are considered "upstream" and those farther away are "downstream"). This way, the cached content is kept as close to the users who will need it as possible.

Arrays are highly scalable because you can add servers to an array, or add array levels to a hierarchical architecture, without having to upset your current caching solution.

Another scalability issue is the ability to use caching to reduce bandwidth for branch office networks. The branch office proxy might not have a direct connection to the Internet, but can use a dialup connection or office-to-office WAN connection to request Web objects from the upstream proxy server at the main office.

Another option is to use a service such as that offered by Akamai for Web-based applications that need to be made available to customers on a global scale. Their Web Application Accelerator service optimizes performance by:

  • Dynamically mapping requests to their edge server, and monitoring Internet routes to send transactions over those that are fastest and most reliable.
  • Using compression technologies and pre-fetching to minimize bandwidth usage.
  • Securing Web transactions with Secure Sockets Layer (SSL).
  • Storing Web content based on customized rules specified by your company.

For specialized applications such as e-commerce, you might consider a solution such as the OracleAS Web Cache, which is deployed between the Internet and your application Web servers, with an Oracle database on the back end.

About Deb Shinder

Debra Littlejohn Shinder, MCSE, MVP is a technology consultant, trainer, and writer who has authored a number of books on computer operating systems, networking, and security. Deb is a tech editor, developmental editor, and contributor to over 20 add...

Editor's Picks

Free Newsletters, In your Inbox