The recently emerging class of middleware services must take into consideration the performance of conventional content-providing Internet services as well as the characteristics of the client population. Middleware services retrieve and transform content on behalf of clients, and as such interact directly with content-providing services, relying in part on the services' performance to determine their own.
In figure 11, we present a breakdown of the time elapsed during the servicing of clients' requests. Figure 11a shows the distribution of the elapsed time between the first byte of the client request and the first byte of the server's response observed by the trace gatherer, shown using both a linear and a logarithmic y-axis. This initial server reaction time distribution is approximately exponentially decreasing, with the bulk of reaction times being far less than a second. Internet services are thus for the most part quite reactive, but there is a significant number of very high latency services.
Figure 11b shows the distribution of the elapsed time
between the first observed server response byte and the last observed server
response byte (as measured by when the TCP connection to the server is shut
down).
From these graphs, we see that complete server
responses are usually delivered to the clients in less than ten seconds,
although a great number of responses take many tens of seconds to deliver.
(Bear in mind that the response data is being delivered over a slow modem
link, so this is not too surprising.)
A number of anomalies can be seen in this graph, for instance the pronounced spikes at 0, 4, 30, and roughly 45 seconds. The spike at 0 seconds corresponds to HTTP requests that failed or returned no data. The spike at 4 seconds remains a bit of a mystery - however, note that the 4 second delivery time corresponds to 14 KB worth of data sent over a 28.8 KB modem, which is almost exactly the size of the ``home_igloo.jpg'' picture served from Netscape's home page, one of the most frequently served pages on the Internet. We believe that the spikes at 30 and 45 seconds most likely correspond to clients or servers timing out requests. Finally, figure 11b shows the distribution of total elapsed time until a client request is fully satisfied. This distribution is dominated by the time to deliver data over the clients' slow modem connections.
Figure 11: Response time distributions (a) elapsed time
between the first observed byte from the client and the first
observed byte from the server, (b) elapsed time between
the first observed byte from the server and the last
observed byte from the server, and (c) total elapsed time
(between the first observed byte from the client and the
last observed byte from the server). All distributions are shown with both
a linear and a logarithmic Y-axis.
From these measurements, we can deduce that Internet servers and middleware services must be able to handle very large amounts of simultaneous, outstanding client requests. If a busy service expects to handle many hundreds of requests per second and requests take tens of seconds to satisfy, there will be many thousands of outstanding requests at any given time. Services must be careful to minimize the amount of state dedicated to each individual request the overhead incurred when switching between the live requests.
