Backbone Proxy Server

For the backbone cache manager, we use squid [4]. However, any proxy server that understands standard HTTP/1.0 should work, since all communication with the backbone cache manager occurs via HTTP/1.0 requests.

The HTTP response filter is implemented in the process filterd. filterd reads a configuration file upon being started by a running copy of bcs. The configuration file indicates what scripts to run on what kinds of responses. For example, JPEG files might require the script jpegfilter. filterd listens to a well known port for incoming HTTP requests. It then forwards all requests to squid [4]. After it receives the response from squid [4], it checks to see if the response is filterable. In the above example, if the response file is called picture.jpg, filterd will issue the command:

jpegfilter picture.jpg tempfile.nam QoS

The script is expected to produce the filtered response in tempfile.nam based on the QoS parameter. filterd then makes necessary changes to the HTTP response headers and sends the new response back. Basically, filterd maintains the state of pending HTTP requests, responses, and the connection to the local cache manager and squid [4].

The QoS parameter measures the quality of service of the network connections between the local proxy server and the backbone proxy server along two somewhat orthogonal dimensions: rate and delay. Periodically, the local proxy server measures the HTTP request-response round trip time (RTT) for a request and signals the backbone proxy server to do the same for the same request. (This is done via piggy-backed HTTP headers.) The difference in the measured RTTs at the local and backbone proxy servers gives an RTT estimate for the HTTP request. By fitting a curve over a sequence of such measurements, the local proxy server can estimate the rate and delay QoS parameter. These values are then transmitted to filterd and fed back into the local HTTP request filter.

The backbone pre-fetch engine pfetcher keeps state of all current Web sessions. It is able to do this because every copy of bcs connects to it. It also maintains a queue of URLs that it feeds (as HTTP requests) to squid [4]. Each running copy of bcs will send their own pre-fetch list. pfetcher will amortize these pre-fetch requests to squid [4]. As pre-fetch responses arrive, pfetcher will determine the dependents and issue more pre-fetch requests if necessary.

The backbone group profile manager gpm keeps track of all the groups which have their group profiles stored on that particular backbone server. It also maintains the list of members of each group. It acts as a query-response server to the group profiles database. When a running copy of bcs informs gpm where its user is currently located, gpm will update the group profiles of all the groups that the user is subscribed to. It then returns a list of recommendations based on the updated group profiles to bcs.

The backbone communication surrogate bcs is a forking process that forks itself for every new Web session by a different user. It maintains the per-user states. That is, it maintains the user profile, the groups a user belongs to, and the user's current QoS. It updates gpm about the movements of the user and obtains a pre-fetch list from it when it wants to recommend pages to the user. It connects to pfetcher and sends it a list of items to pre-fetch based on the user's individual profile. This is for the aggressive backbone pre-fetch. It also spawns the user's HTTP response filter, i.e. filterd.