Supporting Dynamic GPU Computing Result Reuse in the Cloud

Graphics processing units (GPUs) have been adopted by major cloud vendors, as GPUs provide ordersof- magnitude speedup for computation-intensive dataparallel applications. In the cloud, efficiently sharing GPU resources among multiple virtual machines (VMs) is not so straightforward. Recent research has been conducted to develop GPU virtualization technologies, making it feasible for VMs to share GPU resources in a reliable manner. This paper seeks to improve the efficiency of sharing GPU resources in the cloud for accelerating general-purpose workloads. Our key observation is that redundant GPU computation requests are being seen in many GPU-accelerated workloads in the cloud, such as cloud gaming where multiple clients playing the same game call GPUs to perform physics simulation. We have measured this redundancy using a gaming case study, and found that more than 24% (47%) of the GPU computation requests called by the same VM (multiple VMs) are identical. To exploit this redundancy, we present GRU (GPU Result re-Use), a GPU sharing, result memoization and reuse ecosystem in a cloud environment. GRU transparently enables VMs in the cloud to share a single GPU efficiently, and memoizes GPU computation results for reuse. It leverages the GPU full-virtualization technology, which enables GPU result memoization and reuse without modification of existing device drivers and operating systems. We have implemented GRU on top of the Xen hypervisor. Preliminary experiments show that GRU is able to achieve a significant speedup of up to 18 times compared to the state-of-the-art GPU virtualization framework, while adding a rather small amount of runtime overheads.