Performance

We have studied the performance of the intrusion detection wrappers. We measured the overhead caused by the intrusion detection wrappers on the running time of programs using a Kernel Build test, in which the time taken to compile a Generic version of the FreeBSD kernel was measured. Also, we measured the overhead caused by ID wrappers from a user's perspective, in terms of latency and throughput, for a Web server and a FTP server.

 

Table 2: FreeBSD Prototype Performance for Kernel Build and Web Server Benchmarks

	Average Kernel Build Time			Average HTTP Latency			Average HTTP Throughput
	time	$\sigma$	penalty	time	$\sigma$	penalty	t-put	$\sigma$	penalty
	(s)	(s)		(s)	(s)		(Mbits/s)	(Mbits/s)
no WSS	583.43	0.53	0%	0.657	0.025	0%	7.455	0.21	0%
WSS only	604.38	0.46	3.47%	0.652	0.0023	-0.081%	7.456	0.08	0.01%
Seq_id	624.62	1.23	6.59%	0.687	0.017	4.52%	7.038	0.164	5.61%
Http_id	-	-	-	0.705	0.0247	7.38%	6.928	0.157	7.08%
Http_id & Seq_id	-	-	-	0.744	0.018	13.26%	6.607	0.127	11.39%

 

 

Table 3: FreeBSD Prototype Performance for FTP Server Benchmarks

	Average FTP Latency			Average FTP Throughput
	time	$\sigma$	penalty	t-put	$\sigma$	penalty
	(s)	(s)		(Mbits/s)	(Mbits/s)
no WSS	28.2418	0.9019	0%	8.776	0.093	0%
WSS only	28.3332	1.0773	0.32%	8.768	0.069	0.09%
Seq_id	28.30125	1.0835	0.21%	8.743	0.076	0.38%
Ftpd_id	28.3592	0.7954	0.42%	8.756	0.085	0.23%
Ftpd_id & Seq_id	27.9224	1.2007	-1.13%	8.573	0.012	2.31%

 

Table 2 summarizes the results of the performance tests for Kernel Build and for a Web server. The first column shows the average time taken to compile the FreeBSD kernel 1) under normal conditions, 2) with the WSS loaded into the kernel, and 3) with the sequence-based intrusion detection wrapper Seq_id wrapping the compilation process. The second and third columns of the table contain results for a custom-made Web server benchmark. The Average HTTP Latency column describes the delay a Web client experiences between the moment it makes a request and the moment it receives the Web server reply. The Average HTTP Throughput describes the rate at which the Web server returns data to the Web clients. We measured the latency and throughput of the Web server when the Web server is wrapped by Seq_id, Http_id, and both Seq_id and Http_id. The results were produced by a custom-made Web server benchmark executed with an Apache 1.3.0 Web server and the WebStone 2.0.1 benchmarking software. The Apache Web server ran on a 166MHz Intel Pentium-based microcomputer with 32MB RAM running a Generic FreeBSD 2.2.2 kernel. Two Pentium 400MHz machines were used to run 32 WebStone 2.0.1 Web clients through a series of 10 15-minute trials using the standard WebStone 2.0.1 file set for each row in table 2. Table 3 shows the results of the performance tests using a custom-made FTP server benchmark. The Average FTP Latency column describes the delay a FTP client experiences between the moment it makes a anonymous request and the moment it receives all the data from the server. The Average FTP Throughput describes the rate at which the FTP server returns data to the FTP clients. The table denotes the latency and throughput of the FTP server under controls to Seq_id, Ftp_id, and both Seq_id and Ftp_id. Ftp_id is a specification-based wrapper that restricts the operations that can be performed by the FTP server. The FTP server (ftpd) in the FreeBSD 2.2.2 distribution was used in the tests. The Average FTP Latency and Average FTP Throughput results were obtained in a smiliar manner to the HTTP results using a modified WebStone software that performs anonymous FTP fetches instead of HTTP fetches. The performance results show that WSS alone imposes 3-4% penalty on the compilation time of the FreeBSD kernel. Seq_id adds another 3-4% to the compilation time of the FreeBSD kernel. Impact caused by WSS on the latency and throughput of a Web/FTP server is minimal, possibly because WSS only intercept the fork, execve, and exit system calls, which are used infrequently in a Web/FTP server. The sequence-based wrapper and the specification-based wrappers impose approximately 5-7 % overhead on the Web/FTP server, and their impacts add up when they are used together. While we have designed the wrapper toolkit and ID wrappers with consideration for performance, we have not optimized the prototype; therefore, performance can possibly be improved.