The growth of the Internet, and particularly of web-oriented middleware services ([15], [3], [6]) within the Internet, has seen a recent explosion [31]. These middleware services, particularly the more popular services that experience extremely high load, must overcome a number of challenging system design issues in order to maintain fast response time, constant availability, and capacity. Services must be able to accommodate an increasingly varied client population (in terms of hardware, software, and network connectivity). They must be able to handle offered loads of hundreds of requests per second, and because of the often slow connectivity to clients and the implied lengthy delivery times, they must be able to handle hundreds of simultaneously outstanding tasks.
Previous work has explored the performance of operating system primitives and the relationship between OS performance and architecture ([29], [2]), and operating system design issues for busy Internet services ([19], [27]). In contrast, this paper raises a number of system design issues specifically for Internet middleware services. These issues were encountered during two separate but related efforts: the analysis of a set of extensive client-side HTTP [5] traces that we gathered from the University of California at Berkeley's dial-in modem banks during October and November of 1996, and the implementation and deployment experience we gained from the TranSend Internet middleware service [15].
Since nearly 70% of all Internet clients use dial-in modems of speeds of 28.8 Kb/s or less [18], we use the traces to make a number of observations about the Internet user population and the services with which they communicate. Section 2 discusses the gathering of the traces, including the tools used and the information gathered, and section 3 performs a detailed analysis of these traces, both in terms of observations made about the client population and the services themselves. In section 4, we discuss the middleware system design issues drawn from our experience with the TranSend transformation proxy service, and in section 5 we present related work. Finally, in section 6 we conclude.