LIS 525 - Estimating Server Needs

Dedicated Servers

To set up a server to work effectively, you need to know how many requests it will need to handle at once (say within 7 seconds) and how many bytes it will have to deliver per second at peak times.

How much variation there will be in demand from one time to another will differ depending on the type of site. For a relatively smooth demand curve, estimating peak demand at about twice the average might be relatively safe; but a site with time-sensitive information might have much higher peaks.

Once you have an estimate of the peak rate of requests, you can estimate the peak rate of delivery required by multiplying the rate of requests by the average file size.

To determine the speed of network connection required in bits per second, multiply the peak bytes to be delivered per second by about 11.

For example, if the average file size is 10 Kbytes and the peak request rate is 0.5 per second (30 per minute), you need to be able to deliver 5 Kbytes per second and so require a network connection of about 56 Kbps.

Shared Servers

If you are sharing a server with other information providers, you still need to be able to estimate total bandwidth usage over longer periods of time and you may need to know how many concurrent requests you want to be able to accommodate. Many Web hosting plans specify a maximum amount of data transfer per day or per month (e.g., 193 MBytes per day or 6 MBytes per month), after which additional charges apply (e.g. $0.05 per KByte). Others may make a general provision that your traffic should not compromise site performance, or they may restrict the kinds of files that you can mount (e.g., no MP3, no software). Some plans may set a maximum number of requests (e.g., 10,000 per month), after which there are additional charges (e.g., $5 per 10,000).

To determine an acceptable daily traffic limit, first estimate the peak rate of requests per day and then multiply that by the average file size. Add another 10% or more to the byte count to allow for requests and upload traffic.

Points to Note

Remember that a request for a single Web page may actually involve many requests for inline graphics files. On the other hand, if pages are revisited or the same graphics are used in multiple pages, caching at the client side may reduce the number of bytes that actually have to be transmitted. Byte counts may also be reduced somewhat if HTTP compression is enabled.

Many requests will not be from your intended users. Quite apart from denial of service (DoS) attacks, there may be a substantial portion of hits on your site that do not involve actual use of your information services. For example, a study by Cooper of about 2½ million sessions with a Web-based library catalog found about 27% involving less than 10 seconds of connection (all these visits were presumed to be from "spiders"). About another 11% of visits took longer than 10 seconds but involved no searching.

To estimate storage needs, one could just add up expected file lengths. Practically speaking, this approach will actually provide an underestimate, especially when files are small, because disk space is assigned to files in clusters (the smallest units that the operating system allows to be assigned to files). For example, the smallest file on this course's Web site, ssiincl.txt, though only 45 bytes long, still takes up 4096 bytes on a Windows XP hard drive, because that is the cluster size.

For More Information


Last updated October 17, 2007.
This page maintained by Prof. Tim Craven
E-mail (text/plain only):
Faculty of Information and Media Studies
University of Western Ontario,
London, Ontario
Canada, N6A 5B7