Design Evolution of the EROS Single-Level Store

File systems have (at least) two undesirable characteristics: both the addressing model and the consistency semantics differ from those of memory, leading to a change in programming model at the storage boundary. Main memory is a single flat space of pages with a simple durability (persistence) model: all or nothing. File content durability is a complex function of implementation, caching, and timing. Memory is globally consistent. File systems offer no global consistency model. Following a crash recovery, individual files may be lost or damaged, or may be collectively inconsistent even though they are individually sound.

Single level stores offer an alternative approach in which the memory system is extended all the way down to the disk level. This extension is accompanied by a transacted update mechanism that ensures globally consistent durability. While single level stores are both simpler and potentially more efficient than a file system based design, relatively little has appeared about them in the public literature. This paper describes the evolution of the EROS single level store design across three generations. Two of these have been used; the third is partially implemented. We identify the critical design requirements for a successful single level store, the complications that have arisen in each design, and the resolution of these complications.

As the performance of the EROS system has been discussed elsewhere, this paper focuses exclusively on design. Our objective is to both clearly express how a single level store works and to expose some non-obvious details in these designs that have proven to be important in practice.