From disk arrays through clouds to archival systems, storage systems must tolerate failures and prevent data loss. Erasure coding provides the fundamental technology for storage systems to add redundancy and tolerate failures. This tutorial will cover the fundamentals of erasure coding, the mechanics of many erasure codes that apply to today's storage systems, and the properties of various erasure codes designed for a variety of storage scenarios.
Topics include:
General matrix-based codes, starting with classic Reed-Solomon codes
Galois Field arithmetic for erasure-coding, and how to implement it efficiently
More general codes implemented with only the XOR operation: Generalized RDP/EVENODD
Cauchy Reed-Solomon codes
Open source library support for erasure codes
The reconstruction problem and techniques to reduce bandwidth and I/O
Regenerating codes
Practical MDS codes with efficient reconstruction: Rotated Reed-Solomon
Practical non-MDS codes with efficient reconstruction and their application in cloud storage: Pyramid codes, LRC and its deployment in Windows Azure Storage, PMDS
connect with us