CHAPTER 20

                                MICROSOFT EXCHANGE 2000 SERVER   

                            MAINTENANCE AND TROUBLESHOOTING

 

 

The ESE Extensible Storage Engine is very reliable, tried, and proven transaction-based database

technology, which forms the foundation for Exchange 2000 Server’s incredible scalability.

 

Lesson 1:  System Maintenance and Monitoring

 

Deleted Items Retention

 

Exchange 2000 server allows you to configure your mailbox and public stores to retain deleted items

for a specified period of time.  Within that time frame, accidentally deleted items can be recovered

quickly without the need for backup.  This advantageous feature is known as deleted retention.

 

By default, deleted mailboxes are retained for 30 days. Within this interval, you can reconnect

mailboxes to the same or different user accounts without difficulty using Exchange System manager.

 

 

 

Exercise Summary:

 

  •   By default, the Cleanup Agent purges delete mailboxes after 30 days. 
  •   For recovery, they are retained as defined by the Keep Deleted Mailboxes for (Days)

parameter.

 

 

System Monitors as Maintenance Tools

 

The LSI Link State Information of the affected server is updated, the routing group master is

informed, and the new system state is eventually propagated to all servers in the organization.

 

 

Configuring Monitored Services and Resources

 

By default, the following services are checked:  Microsoft Exchange Information Store, Microsoft

MTA Stacks, Microsoft Exchange routing Engine, Microsoft Exchange System Attendant, SMTP

and WWWPublishing Service.    If any of these services is not running the server enters the critical state.

 

 

========================================================================

 

winexc20.html                                                 PAGE 2                                                           2002/06/30

 

 

 

 

Exchange 2000 Server shuts down important services if less then 10MB are available on the drive, but

you should not wait until that happens before taking action.

           

Configuring Notifications

 

If the monitoring server resides in a remote routing group, a broken routing group connector may prevent

the propagation of LSI and changes of warning, or critical states may not be detected in a timely manner.

 

 

Link and System States

 

Select the Status container in the console tree, which you can find in the Monitoring and Status container

within Tools, to display the LSI of connectors and servers in the details pane.

 

 

Limitations of State Information and System Monitoring

 

Launch WINROUTE.EXE from the \Support\Utils\I386 directory on the Exchange 2000 Server CD.

 

  • Trouble-free server has a value beginning with 0701
  • Warning state server has a value beginning with 0702.
  • Critical State is indicated by 0703.

 

 

Exercise Summary:

 

  •   System monitoring is a maintenance utility that should be used extensively.
  •   It is a good idea to use multiple machines in each routing group as monitoring servers for

warning and critical state notifications.

 

 

Using the Message Tracking Center

 

All messages routed through the transport engine of a server that is enabled with message tracking are

added to the tracking logs.  Message tracking is disabled by default.

 

 

Message tracking enables you to do the following:

 

  • Locate specific messages in message queues.
  • Verify that a message has been delivered successfully.
  • Determine delays on each segment of a route for performance tuning.

 

 

 

========================================================================

 

winexc20.html                                                 PAGE 3                                                           2002/06/30

 

 

 

 

Message Tracking Log Files

 

They are stored in \Program Files\Exchsrvr\Manchester .log directory.

 

 

Exercise Summary:

 

  • You can save message histories to text files for later reference or to preserve this information.

 

 

Using Message Queues for Troubleshooting

 

It is advisable to check the message queues of an Exchange 2000 server frequently to verify that the

system is functioning properly.  Too many backlogged messages can indicate a configuration or

performance problem.

 

 

 

Checking SMTP-Based Message Queues Manually

 

Every SMTP virtual server provides a subcontainer called Queues that allows access to system and

connector queues.  To test a new Routing Group Connector, for instance, send test messages to a

remote recipient.

 

 

Deleting and Freezing Messages

 

If you suspect that a displayed message is blocking a queue, you can delete it by right-clicking on it

and selecting Delete (Send NDR) or delete (No NDR), depending on whether you want to inform

the originator about the deletion or not.

 

 

Checking X.400-Based Messages Queues

 

It is also possible to work with X.400-based queues.  This is required if you have deployed X.400

 connectors to remote routing groups or foreign messaging systems.  The MTA of Exchange 2000

Server is also responsible for message routing to gateway connectors, such as the Connector to

Lotus cc:Mail or Novell GroupWISE.

 

X.400-based Queues container can be found under the X.400 Protocol Container.  The Microsoft

Exchange MTA Stacks service must  be running; otherwise, the queue contents are not displayed. 

 

 

 

========================================================================

 

winexc20.html                                                 PAGE 4                                                           2002/06/30

 

 

 

 

The MTA Check Utility

 

The MTA is an important component responsible for communication over X.400 Connectors and gateways

to foreign messaging systems.  This component maintains its message queues in .dat files, which can be found

in the \Program Files\Exchsrvr|metadata directory.    Several of these .dat files are installed during setup and

represent important MTA message queues.  Others are created during message conversion.  Temporary

.dat files represent the actual contents of messages that are currently located in an MTA message queue. 

A few moments later, when the MTA has processed the messages, the related .dat files disappear again.

 

 

Fixing MTA Startup Problems

 

Unfortunately, .dat files can become corrupted just like any other file on a computer’s hard disk, during

a system crash, for instance.

 

Use the command line Mtacheck /? Within the \Program Files\Exchsrvr\Bin directory to display a short

help file about available parameters.

 

 

Lesson 2:  Database Operation and Maintenance

 

Both Active Directory and Exchange 2000 Server utilize the ESE database engine (ESE.DLL), which

in turn manages a variety of database files to ensure data integrity and consistency even in the event of

a system crash.

 

 

Considerations About Information Store

 

Several system improvements allow you to consolidate your resources on fewer but larger server. 

Benchmark tests prove that Exchange 2000 Server is very scalable.

 

 

Information Store Database Sizes

 

One important consideration in determining the maximum number of users per server is the resulting

size of the information store database files.  High-performance backup solutions may be able to save

between 25FB and 50GB per hour.  If 10,000 users cannot work for 10 hours, altogether 100,000

work hours are lost, that’s 12,500 workdays or approximately 62 work years (using 200 workdays/year).

 

 

Defining the Databases

 

Exchange 2000 server databases can be divided into two groups:  Core databases and additional

databases.

 

========================================================================

 

winexc20.html                                                 PAGE 5                                                           2002/06/30

 

 

 

 

Core Databases.  The core databases belong to the Information Store service.  The databases of the

default mailbox store are called PRIV1.EDB and PRIV1.STM, and PUB1.EDB and PUB1.STM

for the default public store.  These files can be found in the \Program Files\Exchasrvr\Mdbdata directory.

 

Additional Databases:   ? can’t locate information.

 

 

Optional Databases

 

The KMS database called KMSMDB.EDB resides in the \Program Files\Exchsrvr\Kmsdata

directory, provided that you have installed KMS on the local computer.

 

The Dirsync database, called XDIR.EDB is located in the \Program Files\Exchsrvr\Dxadata directory. 

It keeps track of MS Mail directory synchronization (DIRsync) transactions. 

 

 

Mail Database Components

 

A number of .log files exist, along with a .chk file and TMP.EDB. TMP.EDB is not important

because it contains temporary information that is deleted when all stores in the storage group are

dismounted or the Information Store service is stopped.  TMP.EDB is not included in backups.

 

 

Transaction Logs

 

Transaction log files give ESE the ability to manage data storage efficiently with high speed.  ESE

stores new transactions, such as the delivery of a message, in a memory cache and in the transaction

log concurrently.  The E00.LOG is used for all mailbox and public stores in this storage group. 

If you create additional storage groups, the prefix number is incremented to E01, E02, and E03.

 

 

Checkpoint Files

 

The ESE memory cache greatly improves system performance, but this cache may be lost due to a

sudden power outage or system failure before the transactions are written to the database.

 

 

Previous Logs

 

Transaction log files are always exactly 5.242.880 bytes (5MB) in size.

 

 

 

========================================================================

 

winexc20.html                                                 PAGE 6                                                           2002/06/30

 

 

 

Reserved Logs

 

Reserved logs are an “emergency repository” for transactions.  They provide enough disk space

to write them from memory to the hard disk even if a server’s disk is filled to a point where no

new transactions can be added to a log file.  Reserved logs are called RES1.LOG and RES2.LOG

and can be found in the transaction log directories.

 

 

Patch Files

 

PRIV1.PAT and PUB1.PAT are patch files.

 

 

Log Files Maintenance and Circular Logging

 

Exchange 2000 Server holds each transaction in two file-based repositories:  the transaction log and

 the databases.

 

 

Manual Deletion.  Manually deleting transaction log files is not advisable because you run the risk of

corrupting the database.

 

Database Backups.  Transaction log files are deleted when you complete a full or an incremental backup.

 

Circular Logging.  Automatically deleting transaction log files and their entries.  Circular logging causes

the server to discard transactions as soon as they have been committed to the database.  Circular logging

prevents duplicate consumption of disk space, but it is not compatible with sophisticated fault-tolerant

configurations and several online backup types, which rely on the existence of transaction logs.

 

 

Database Partitioning

 

Each storage group requires a directory for its transaction logs, but this location does not need to be

the directory for the database.

 

Transaction Log Location.  Places the logs of each storage group on a separate disk, if possible. 

You can use RAID or mirror the.

 

Database Location.  A stripe set with parity, or any other advanced RAID configuration may provide

 the required level of protection against hardware failures.  If one disk breaks, it can be replaced without data loss.

 

 

========================================================================

 

winexc20.html                                                 PAGE 7                                                           2002/06/30

 

 

 

 

Maintaining Databases

 

Databases become fragmented over time in a normal process that you can’t prevent, just as you

can’t prevent the fragmentation of a computer’s hard disk.  It will slow down the server when

retrieving data.

 

Database Defragmentation.  The Information Store defragments its databases automatically during

scheduled maintenance cycles.  For that reason you don’t need to worry about defragmentation

that much.

 

Database Compaction.  Deleted objects are reordered and free space in the database files is

marked as available.  This is usually sufficient, but it may be desirable to shrink the database size

 to recover free disk space, for instance.

 

Troubleshooting Database Problems.  Use ESEUTIL.EXE with the /g switch to verify database

integrity.  The /p switch will fix the corruption.  By default this utility does not correct any

corruption; it checks only for table damage, incorrect reference counters, and nonreferenced

items.

 

 

Copying Databases Using the esefile Utility

 

It is important to note that low-level database tools, such as ESEUTIL.EXE work with temporary

database files.  You need to make sure that you have sufficient free disk space on the machine

where you want to perform the operation.

 

 

Lesson 3:  Backup, Restore, and Disaster Recovery

 

Exchange 2000 Server provides a dedicated API for backing up and restoring databases.  This

API, implemented in ESEBCL12.DLL, gives backup applications, such as Microsoft Windows

2000 Backup, the ability to perform backup and restore operations online, without the need to

stop database-related services.

 

 

Backing up the Databases

 

You should only use the Exchange 2000 Server-enabled backup program to perform backup

and restore operations.  The Windows 2000 Backup utility, is made Exchange 2000 Server-aware

when you install the Microsoft Exchange System Management Tools using the Exchange 2000

Server Setup program.

 

Offline Backups.  An offline backup is a regular file-based backup of the \Program Files\Exchsrv

directory and its subdirectories.  It can be performed only when the server services are stopped (offline).

 

 

 

========================================================================

 

winexc20.html                                                 PAGE 8                                                           2002/06/30

 

 

 

 

Online Backups.  Performed when the server services are running (online).

Full Backup.  Covers entire information store.  It saves databases as well as transaction log entries that

have not yet been committed to the databases.

 

Incremental backup.  Saves only new transaction log files, which are purged once they have been

backed up, setting the context for the next incremental or differential backup.  Incremental backups

are not supported on storage groups where circular logging is enabled.

 

Differential Backup.  Works similar to the incremental backup, but does not purge transaction

log files.  This backup type is not supported on servers or storage groups where circular logging is

enabled.

 

Copy Backup.  Save databases and transaction logs, but does not purge any files from the system. 

 

 

Verifying Database Backups

 

Professional backup solutions, including Windows 2000 Backup, are able to verify the successful

completion of backup jobs.  During the data verification process, Backup compares the data that was

written to tape with the data in the databases and detects possible damage.

 

 

Designing a Disaster Recovery Plan

 

It is vital to document your backup and restore procedures, label backup media properly, and store

the media in a secure location.  Many organizations keep full backups offsite in a separate

geographical place for maximum security.

 

 

  •   A copy and documentation of any other software, such as service packs or configuration changes

applied to the system.

  •   Copy of the original Windows 2000 Server and Exchange 2000 Server Product CDs.

An emergency repair disk.

 

 

 

Restoring to the Same Server

 

It is straightforward to restore Exchange 2000 Server databases to their original location.

 

Restoring a Full Backup with Incremental or Differential Backups.  Make sure you restore

the full backup first.  During the restore, you need to specify a temporary location where Backup

will place transaction log and patch files.

 

 

 

 

========================================================================

 

winexc20.html                                                 PAGE 9                                                           2002/06/30

 

 

 

 

 

Performing a Manual Hard Recovery.  Nobody is perfect and you may forget to select the Last

Backup Set check box.  In this situation, Exchange System Manager will not mount restored databases

and will report an internal processing error.  The error message will suggest that you try restarting an

 internal processing error.

 

 

Complete Disaster Recovery

 

In a complete disaster recovery, you will restore Exchange Server to a different physical computer. 

This is required if the original server has been fully destroyed. 

 

  1.   Reinstall the operating system.  Use the same version of Windows 2000 Server that was

previously installed, such as Microsoft Windows 2000 Advanced Server (or the security

version).  Apply current service packs.

  1.   When you reboot, log on as Administrator, you can restore any file-based backups to the

drives plus the original system state information.

  1.   When you reboot, error message, that services did not start.
  2.   As soon as Setup completes the disaster recovery, you can restore the databases.
  3.   Reboot the server, and test whether you can log on to your mailbox successfully.

 

 

 

Forklifting  Users by Means of a Complete Disaster Recovery

 

Many organizations use the complete disaster recovery methodology to replace existing server

hardware.  Although this is promising, you should not use it without careful preparation.  For

 instance, you should not dismantle the old server until you have restored the system to the new

machine successfully.  The larger your database (some might be larger than 100GB), the longer

it takes to complete this process.

 

 

Restoring to different Server (p715)

 

If the old machine is out of order and a new system is not immediately available, you will be

forced to recover the databases onto a different existing production Exchange 2000 server. 

Restoring to a different server introduces a number of critical issues.  See page 716.

 

 

========================================================================

 

winexc20.html                                                 PAGE 10                                                         2002/06/30

 

 

 

 

High-Speed Recovery with Delayed Restore

 

Restoring databases directly to the production server implies that users cannot work with their

mailboxes during the restoration.  Left off 716, too detailed, review.

 

 

 

Chapter Summary:

 

  •   Exchange 2000 Server utilizes the features of the ESE for its database repositories.  Transaction

logging and other database features turn Exchange 2000 Server into an extremely reliable platform.

ESEUTIL.EXE is a utility. It is used for database compaction.