Check out the new USENIX Web site. next up previous
Next: Other Benchmarks Up: Macro-benchmarking Experiments Previous: PostMark Results


TPC-C and TPC-H Results

TPC-C is an On-Line Transaction Processing (OLTP) benchmark that leads to small 4 KB random I/Os. Two-thirds of the I/Os are reads. We set up TPC-C with 300 warehouses and 30 clients. We use IBM's DB2 database for Linux (version 8.1 Enterprise Edition). The metric for evaluating TPC-C performance is the number of transactions completed per minute (tpmC).

Table 6 shows the TPC-C performance and the network message overhead for NFS and iSCSI. Since these are results from an unaudited run, we withhold the actual results and instead report normalized throughput for the two systems.[*]As shown in the table, there is a marginal difference between NFS v3 and iSCSI. This is not surprising since TPC-C is primarily data-intensive and as shown in earlier experiments, iSCSI and NFS are comparable for data-intensive workloads. An analysis of the message count shows that the vast majority of the NFS v3 protocol traffic (99%) is either a data read or a data write. The two systems are comparable for read operations. Since data writes are 4KB each and less-intensive than in other benchmarks, NFS is able to benefit from asynchronous write support and is comparable to iSCSI.


Table 6: TPC-C Results. Reported throughput (tpmC) is normalized by a factor $ \alpha $ equivalent to the throughput obtained with NFS v3.
Peak Throughput (TpmC) Messages
NFS v3 iSCSI NFS v3 iSCSI
$ \alpha $ 1.08 $ *\alpha$ 517,219 530,745

The TPC-H benchmark emulates a decision support systems that examines large volumes of data, executes queries with a high degree of complexity, and gives answers to critical business questions. Our TPC-H experiments use a database scale factor of 1 (implying a 1 GB database). The page size and the extent size for the database were chosen to be 4 KB and 32 KB, respectively. We run the benchmark for iSCSI and NFS and report the observed throughout and network message overheads in Table 7. Again, we report normalized throughputs since our results are unaudited. The reported throughput for TPC-H is the number of queries per hour for a given database size (QphH@1GB in our case).

We find the performance of NFS and iSCSI is comparable for TPC-H. Since the benchmark is dominated by large read requests--an analysis of the traffic shows that the vast majority of the messages are data reads--this result is consistent with prior experiments where iSCSI and NFS were shown to have comparable performance for read-intensive workloads.

Workloads dominated by large sequential reads can also signify the maximum application throughput that be sustained by a protocol. The experiments indicate no perceptible difference in this particular edge-condition case.


Table 7: TPC-H Results. Reported throughput (QphH@1GB) is normalized by a factor $ \beta $ equivalent to the throughput obtained in NFS v3.
Throughput (QphH@1GB) Messages
NFS v3 iSCSI NFS v3 iSCSI
$ \beta $ 1.07 $ *\beta$ 261,769 62,686


next up previous
Next: Other Benchmarks Up: Macro-benchmarking Experiments Previous: PostMark Results
2004-02-10