Exchange 2000 Performance Planning

Microsoft Exchange Server 5.5's performance characteristics are well known. In 1996, Exchange Server 4.0 laid down the basic principles of achieving optimum performance through file distribution, and not much has changed since. True, Microsoft expanded the capacity of the Information Store (IS) to a theoretical limit of 16TB, but the messaging server's essential characteristics remain. The hot spots—the files that generate the heaviest I/O load—are the IS and Directory Store databases, their transaction logs, the Windows NT swap file, and the Message Transfer Agent (MTA) work directory.

Exchange 2000 Server is a different beast. The new messaging server boasts the following improvements:

• The IS architecture has evolved from the simple partitioning of the private and public databases to a point at which, theoretically, the architecture lets you run as many as 90 databases on one server.
• Microsoft Internet Information Server (IIS) handles all protocol access for SMTP, Internet Message Access Protocol 4 (IMAP4), HTTP, Network News Transfer Protocol (NNTP), and POP3, so IIS is more important to Exchange 2000 than it was to earlier versions of Exchange Server.
• A new streaming database can hold native Internet content.
• Windows 2000's (Win2K's) Active Directory (AD) replaces the Directory Store.
• A new SMTP-based Routing and Queuing engine replaces the older X.400-based MTA.

These improvements come in a customizable package that third-party solutions will likely extend to provide Exchange 2000 with antivirus, fax, workflow, document management, and other capabilities that aren't part of the base server. Exchange 2000 introduces important architectural changes that have a profound effect on performance. The question that system designers now face is how to best optimize these new features in terms of system and hardware configurations. To answer that question, let's start by investigating Exchange 2000's partitioning of the IS.

Partitioning the IS
Exchange Server 5.5 uses one storage group composed of the private and public stores. Exchange 2000 extends this functionality into storage groups. A storage group is an instance of the Extensible Storage Engine (ESE) database engine, which runs in the store.exe process and manages a set of databases. Exchange Server 5.5 uses a variant called ESE 97; Exchange 2000 uses the updated ESE 98.

Each Exchange 2000 storage group has a separate set of transaction log files that as many as six message databases share. A message database consists of two files—the .edb file (i.e., the property database) and the .stm file (i.e., the streaming database). The .edb file holds message properties (e.g., author, recipients, subject, priority), which Exchange Server typically indexes for use in search operations. The .edb file also stores message and attachment content that Messaging API (MAPI) clients such as Microsoft Outlook 2000 generate. The .stm file holds native Internet content (e.g., MIME). The ESE manages the seamless join between the .edb and .stm files. The new IS architecture permits as many as 16 storage groups on a server. Exchange 2000 devotes 15 of these storage groups to regular operation and 1 to restoring or recovering databases. Each active group consumes system resources such as virtual memory. Microsoft is working to identify the maximum number of storage groups and databases that can be active on a 32-bit platform. That number will likely be well under the maximum that the architecture allows—possibly between four and six storage groups. As Windows and Exchange Server move toward a 64-bit platform, memory management will become less important and servers will be able to use as many storage groups as the architecture allows.

In response to criticism about the original 16GB database limit, Microsoft lifted some internal Exchange Server restrictions to let the database grow as large as available disk space permits. (The limit still exists for the standard version of Exchange Server 5.5.) A larger database lets you allocate greater mailbox quotas to users and lets a server support more mailboxes. However, when a database grows past 50GB, you need to pay special attention to backup and restore procedures, as well as the performance characteristics of the I/O subsystem. Although databases of any size require backups, the larger a database grows, the more challenging it becomes to manage. The ability to store massive amounts of data is useless if poor operational discipline compromises that data or if the data can't get to the CPU for processing because of I/O bottlenecks. In this respect, 50GB is an artificial limit.

Despite of the larger store, the practical limit for user mailboxes on one Exchange server—even when you involve Microsoft Cluster Server (MSCS)—remains at about 3000. Hardware vendors have published performance data that suggests the possibility of supporting 30,000 or more simulated users on one 8-way Xeon server. Regardless of that data, if one large database experiences a problem, thousands of users will be unhappy. Large databases are potential single points of failure. Therefore, you won't find many Exchange servers that support more than 3000 mailboxes. The largest database in production today is approaching 200GB, so functioning with very large Exchange Server databases is possible—but only when you devote great care to day-to-day operations and system performance.

Partitioning the store is interesting from several perspectives. First, by removing a potential single point of failure (i.e., dividing user mailboxes across multiple databases), you can minimize the impact of database failure. Second, you can let users have larger mailbox quotas. Third, you can avoid potential I/O bottlenecks by dividing the I/O load that large user populations across multiple spindles generate. Finally, the advent in Win2K of active-active 2-way and 4-way clustering (which Exchange 2000 supports) increases overall system resilience through improved failovers.

On an operational level, Microsoft has gone to great lengths to ensure that multiple databases are easier to manage. As Screen 1 shows, Win2K's Backup utility can back up and restore individual storage groups and databases rather than process the entire IS. Third-party backup utilities (e.g., VERITAS Software's Backup Exec, Legato Systems' NetWorker, Computer Associates'—CA's—ARCserveIT) will probably support this feature by the time Microsoft releases Exchange 2000. Using the Exchange System Manager Microsoft Management Console (MMC) snap-in, you can dismount and mount an individual database for maintenance without halting all store operations, as Exchange Server 5.5 does. For example, suppose that the Applied Microsoft Technologies database in the First Storage Group is dismounted, as Screen 2 shows. Right-clicking the database brings up a context-sensitive menu in which you can choose the All Tasks, Mount Store option to bring the store online. Generally, you'll find that Exchange 2000 database operations are easier than Exchange Server 5.5 operations because the databases are smaller, and because you can process operations against multiple databases in parallel.

However, Exchange 2000 uses a single storage group by default. Out of the box, or following an upgrade from Exchange Server 5.5, Exchange 2000 operations proceed exactly as they do in Exchange Server 5.5. To take advantage of the new features and gain extra resilience, you need to partition the store, and you can't partition the store until you carefully consider database placement, file protection, and I/O patterns.

In terms of I/O, the Exchange Server 5.5 store is a set of hot files. All mailbox operations flow through priv.edb, whereas all public folder operations channel through pub.edb. If you partition the store and create multiple databases, you need to consider how to separate I/O across a system's available disks in such a way that you increase performance and protect your data. I don't mean to suggest that you ought to rush out and buy a set of large disks. Increased information density means that 72GB disks are now available, and the price per megabyte is constantly dropping. However, you won't attain good I/O performance by merely increasing disk capacity. The number of disks, as well as the intelligent placement of files across the disks, is much more important. CPU power increases every 6 months, and 8-way processors are now reasonably common. However, even the most powerful system can't surpass a humble single processor if its CPUs can't get data. Therefore, storage configuration is crucial to overall system performance.

Storage Performance Basics
Figure 1 illustrates a typical disk response time. As the number of requests to the disk increase, the response time also increases along an exponential curve. Disk queuing causes this behavior, and you can't do anything about it. Any disk can service only a limited number of I/Os, and I/O queues accumulate after a disk reaches that limit. Also, the larger the disk, the slower it typically is. For example, don't expect a 50GB disk to process more than 70 I/O requests per second. Over time, disks might spin faster, get denser, and hold more data, but they can still serve I/O at only a set rate, and that rate isn't increasing.

Transactions that the ESE applies to the Exchange Server databases use a two-phase commit (2PC) process, which ensures that all database changes that are part of a transaction occur. A transaction modifies database pages as it proceeds, and the transaction log buffer stores the changes. To ensure the integrity of the database, a special memory area called the Version Store holds the original page content. When the transaction commits, the database engine writes the page changes from the transaction log buffers to the transaction log files, then removes the pages from the Version Store. If the ESE must abort the transaction, any changes related to the transaction will roll back.

Writing to the transaction log is the performance-critical part of the process. The IS orders the pages in memory and commits them in an efficient, multithreaded manner, but the writes to the transaction log are sequential. If the disk holding the logs is unresponsive, delay will occur and Exchange Server won't log transactions quickly. Therefore, you need to ensure that the I/O path to the disk holding the transaction logs is as efficient as possible. Note that the same performance characteristic is evident in AD, which uses a modified version of the ESE.

   Previous  [1]  2  Next