Afternoon Tutorial 1: Designing Modern Software Caches

Caching is one of the most basic and most effective mechanisms for boosting computing storage systems' performance.

In this tutorial we will survey recent developments in designing software cache libraries, while using Guava, Caffeine, and Ristretto as running examples. We will start by explaining some basic concepts in caching terminology and discuss why software caches are different than hardware designs and the challenges of designing an effective software cache library.

In particular, we will address issues like concurrency, memory management, timers handling, and cache management (admission and eviction). We will also address open research directions.

Specifically, the tutorial will include 4 sections:

1. An introduction to software caching: basic concepts, differences from hardware caches, principle of locality, significance of workload, other challenges—a total of 30 minutes + 5 minutes for questions.
2. General design concerns: effective concurrency, watermarks for evacuation, timers handling—60 minutes + 10 minute break at the end.
3. Cache management: A survey of modern cache management schemes (replacement and evacuations) including ARC, CAR, Hyperbolic, W-TinyLFU, FRD, and Mini-Sim—60 minutes + 10 minutes break at the end.
4. Open research directions: Including, e.g., entry cost (weight, latency), capacity allocations for multiple tenants, prediction of workload changes—30 minutes + 5 for questions.