Web Server Load Balancers

As the e-commerce industry continues to grow, more businesses rely on their Web sites to communicate with customers. A high-performance Web site that quickly and reliably delivers content gains and retains customers and is crucial to a successful and competitive e-business. Few potential customers will return to a frustratingly slow Web site if the customer experiences significant delays or failure. Thus, as part of your organization's Web infrastructure planning and implementation, you need to seriously consider how to improve your Web site's performance.

You can use several methods to improve Web performance: expand Internet bandwidth, use fast network equipment, design efficient Web applications, optimize and upgrade Web server software and hardware, and use Web-caching technology. (For information about Web caching, see "Surfing Web-Caching Technology, Part 1," September 1999 and "Surfing Web-Caching Technology, Part 2," October 1999.) In addition to these options, you can improve your Web site's performance by adding Web servers and sites and mirroring content across all servers and sites. This method lets you share the overall load among servers and sites and reduce the information turnaround time involved in a server's internal processing of client requests. In addition, you can preserve your existing servers rather than retire them to make way for new servers.

Load sharing, or balancing, on multiple servers ensures that Web traffic doesn't overload one server while other servers sit idle. To load balance Web servers, traditionally you use the DNS round-robin feature to evenly distribute Web server IP addresses to clients; thus, your Web servers are equally accessible. However, this mechanism doesn't provide load balancing in an environment in which the Web servers have different hardware and software capacities. For example, a Windows 2000 Server (Win2K Server) system with two 450MHz Pentium III processors and 1GB of memory should handle more load in a load-balanced environment than a Windows NT Server system with one 300MHz Pentium II processor and 256MB of memory. However, DNS's round-robin feature treats these two systems equally; it doesn't know a Web server's availability because the round-robin feature doesn't detect whether the server is up or down. (For more information about the DNS round-robin feature's limitations, see Douglas Toombs "Load Sharing for Your NT Web Server," April 1998.)

Recently, vendors developed load balancers, which are products that balance load evenly across multiple servers. In addition, a load balancer ensures Web servers' fault tolerance by redirecting traffic and clients to another server or site in case of failure. Therefore, clients experience fewer delays and no failures. You can use load balancers in single Web site and multiple Web site scenarios. Knowing what a load balancer is and how it works will help you identify important features to consider and evaluate when choosing a load balancer.

What Is a Load Balancer?
A Web server load balancer is a tool that directs a client to the least busy or most appropriate Web server among several servers that contain mirrored contents. The client transparently accesses this set of servers as one virtual server. For example, suppose you have two Web servers in a single Web site scenario: web1.acme.com with the IP address 193.168.36.1 and web2.acme.com with the IP address 193.168.36.2, as Figure 1 shows. The load balancer uses a virtual host name (e.g., www.acme.com) and virtual IP (VIP) address (e.g., 193.168.35.10) to represent the Web site. You associate the virtual host name and the corresponding VIP address with the two Web servers by publishing the virtual host name and its VIP address in your DNS server. The load balancer constantly monitors the load and availability of each Web server. When a client accesses www.acme.com, the request goes to the load balancer instead of a Web server. Based on the load of each monitored server and conditions and policies you've defined, the load balancer decides which server should receive the request. The load balancer then redirects the request from the client to the server and usually redirects the reply from the server to the client depending on the implementation.

Load balancers can also support load balancing across multiple Web sites. Implementing multiple sites places mirrored Web servers close to customers and reduces delay between your Web site and customers. In addition, multiple Web sites provide load balancing, high availability, and fault tolerance in case of a complete site failure (e.g., a power or Internet connection outage at a data center). In a multisite scenario, which Figure 2 shows, every load balancer at every site has the same virtual host name but a different VIP address. For example, load balancer 1 at site 1 in New York has the virtual host name www.acme.com and VIP address 193.168.35.10. Load balancer 2 at site 2 in Los Angeles has the same virtual host name but a different VIP address—193.200.1.10. You associate each load balancer with its local Web servers using the same method you use in a single-site scenario. In addition to monitoring the load of local servers, the load balancers exchange site configuration and load information and check site availability with load balancers at other sites. Thus, each load balancer has the global load and availability information locally. Load balancers in multisite scenarios also often work as the DNS server for the virtual host name. When a load balancer receives a DNS lookup from a client for the virtual host name, the load balancer returns to the client the VIP address of the best site according to the current site load, client proximity, and other conditions. The client then transparently accesses that site.

Three types of load balancers exist: hardware appliances, network switches, and software. A hardware appliance-based load balancer is a closed box that is usually an Intel-based machine running a vendor's load-balancing software and a UNIX or proprietary OS. Hardware appliance-based load balancers provide a Plug and Play (PnP) solution for Web administrators. A network switch-based load balancer uses a Layer2 or Layer3 switch to integrate the load-balancing service. This device doesn't require an add-on box between the switch and the Web servers, but the appliance-based load balancer requires an add-on box. A software-based load balancer doesn't require you to modify network connectivity or equipment when you introduce the load-balancing service to your Web server farm. You can install the software on existing Web servers or dedicated load-balancing servers. (For a list of Web server load balancer vendors and products, see "Web Server Load Balancer Resources," page 70. The sidebar "Microsoft's Load-Balancing Services," explores Microsoft's load-balancing solutions.)

Regardless of which product category a load balancer belongs to, it fulfills the following three functions: monitoring server load and health, selecting the right server for a client, and redirecting traffic between the client and server. Let's look at each of these functions and learn how load balancers implement them.

Server Monitoring
A load balancer constantly monitors the load and health of managed Web servers so that it can use the load and health information to select the best available server to respond to a client request. Load balancers use two methods to monitor servers: external monitoring and internal monitoring.

To externally monitor a server, the load balancer calculates a server's response time by inputting a request to the server and waiting for a response. Using an Internet Control Message Protocol (ICMP) ping is the simplest way for a load balancer to externally monitor a server. An ICMP ping tests a server's availability and the round-trip time between the server and load balancer. If the load balancer doesn't receive a response from the server after several consecutive pings, the load balancer assumes that the server isn't available. Administrators usually connect Web servers directly to the load balancer, so if the round-trip response time is long, the load balancer knows that the server is very busy.

However, the ICMP ping tests only a server's IP stack but can't monitor the health of the TCP stack that HTTP uses. To verify that a server's TCP stack works, the load balancer attempts to establish a TCP connection, which requires a three-way handshake, with the server. In a three-way handshake, the load balancer sends the server a TCP packet with the SYN bit set to 1. If the load balancer receives back from the server a TCP packet with the SYN bit set to 1 and the ACK bit set to 1, the load balancer sends another TCP packet with the SYN bit set to 0 and the ACK bit set to 1. A completed handshake means that the server's TCP stack is healthy. After completing the handshake, the load balancer immediately drops the connection to save server resources. The load balancer can estimate a server's TCP connection performance based on the time a completed three-way handshake takes to complete.

In addition to testing the protocol stacks, a sophisticated load balancer can monitor the response time and availability of a Web server and its applications by making an HTTP request for content or a URL. For example, suppose web1.acme.com's home page is index.htm. The load balancer in Figure 1 can initiate an HTTP Get command asking for the content of index.htm on web1.acme.com. If the load balancer receives from the Web server a return code of 200, the home page on web1.acme.com is available. The load balancer measures the response time by measuring the time between sending the content request and receiving the return code.

Although external monitoring lets you ascertain useful information, it provides limited or no information about several important aspects of a server's status, including CPU, memory, system bus, I/O bus, NIC, and other system and application resources. Only internal monitoring can provide such detailed server load information. To internally monitor a server, the load balancer uses an internal-monitoring agent, which physically resides in each server, monitors a server's status, and reports the status to the load balancer. Some vendors provide scripting tools that let you write internal monitoring utilities for your Web applications. Internal monitoring is common for software-based load balancers, but few appliance- and switch-based load balancers use internal monitoring.

Server Selection
A load balancer can use the information from externally and internally monitoring a server to select which server is best for handling a client request. If all servers have the same hardware and software capacity, you can configure the load balancer to use a round-robin system to select a server based on the servers' status. However, if a load balancer manages a server with a Pentium III processor and a server with a Pentium Pro processor, you can configure the load balancer to redirect more traffic to the more powerful server. This setup is a weighted round-robin configuration.

A sophisticated load balancer lets you specify a custom policy of server selection. For example, you can configure the policy to include CPU utilization, memory utilization, number of open TCP connections, and number of packets transferred on a server's NIC. Your load balancer's load formula might look like (10 ¥ CPU utilization) + (3 ¥ memory utilization) + (6 ¥ the number of open TCP connections) + (3 ¥ the number of transferred packets) = a server's load. When it receives a client request, the load balancer calculates the load for each server according to the formula and redirects the request to the server with the lightest load.

In some cases, after the load balancer assigns a server to a client and the server and client make an initial connection, an application requires the load balancer to persistently send the client's traffic to that server. This connection is a persistent or sticky connection. For example, a user is shopping in an online bookstore and puts three books in the shopping cart. If the server that processes that client's request caches the shopping cart information locally, the load balancer can't switch the client's new traffic to another server even if the load becomes unbalanced across a site's servers. Otherwise, the three books in the client's shopping cart will be lost because the new server doesn't have the client's shopping cart information. Therefore, the load balancer must remember which client is accessing which server for a certain amount of time that you define based on your customers' behavior and applications. If you enable a load balancer's persistent feature, this feature will always override other load-balancing policies.

The key to maintaining a persistent connection is to find out a client's identity and bind this identity to a destination server. The load balancer usually uses the client's source IP address as the client's identity. However, the client's source address might not be the client's real IP address. Many companies and ISPs use proxy servers to control Web traffic and hide their users' IP addresses. Thus, if 500 clients access your Web site from America Online (AOL) and 10 clients access your Web site from another company, the server load will be unbalanced because the load balancer will bind all 500 AOL clients that have the same source address to one server and the other 10 clients to another server. To overcome this disadvantage, a load balancer that supports source IP address and TCP port number binding can distinguish clients even if the clients are using the same proxy server. The load balancer can make this distinction because each TCP connection has a unique source IP address and TCP port number. Another way to identify a client if the client is using a secure HTTP session is to monitor a Secure Sockets Layer (SSL) session ID. The SSL protocol assigns an ID to an established SSL session, and online shopping applications often use SSL. The most recent innovation to support a persistent connection is the Web cookie, which contains a client's identity and other information, such as which server the client last accessed. By examining Web cookies' content, a load balancer can better identify clients and select the appropriate server for them. Cookie-aware load balancer vendors include Alteon WebSystems, ArrowPoint Communications, F5 Networks, and Resonate.

In another server-selection method, immediate binding, load balancers can choose a server for a client and send the client to the server as soon as the load balancer receives the client's TCP SYN packet. A load balancer bases the server selection on server load-balancing policies and the IP address and TCP port numbers in the client's TCP SYN packet. Although this method is fast, a load balancer doesn't have time to ascertain other information, such as the SSL session ID, cookie, URL, or application data. To learn more about the client and make a better decision, the load balancer needs time to peek into application-layer information. In the delayed-binding method of server selection, the load balancer waits to make a server selection until the TCP three-way handshake is complete and the load balancer and client establish a connection. The load balancer becomes content-aware by examining the application-layer information before selecting a server.

   Previous  [1]  2  Next