Rolling Your Own Storage: Software Defined Storage and Commodity Hardware for Cloud and Large Scale Workloads

The monolithic enterprise storage appliance has been falling out of vogue lately. Storage architects are increasingly looking for "software defined" storage solutions that run on top of commodity hardware as well as "hyper-converged" systems that combine storage and compute resources in the same platform. As an IT architect or buyer, it is is very empowering to buy storage software decoupled from hardware, but. . . . uh. . . someone still has to engineer the hardware!

Yes, you can turn pretty much any computer into a storage device, but there are huge advantages to doing it right. This lecture reveals the major issues to consider when selecting hardware for software defined storage systems. We look at I/O subsystems, backplanes, and the nuances of solid state storage devices. Using CEPH as a case study, we describe various ways to get the most performance and reliability out of your storage hardware.

Lecture Outline

• Defining "Software Defined" Storage
  • What does it mean? What should it mean?
• Deconstructing Hyper-convergence
  • Storage and compute on the same server, yes or no?
  • Pros and cons of scale-out storage architectures
  • Virtualizing the storage controller
  • Local affinity v. low Latency and parallelism
  • NVMe over Fabric
• SSDs - Getting Your Money's Worth
  • Leveraging lower cost SSDs
  • Eliminating hidden bottlenecks in the storage I/O path
  • Averting write-amplification
• Commodity Hardware Concepts
  • COTS
  • Backplanes and PCIe lanes
  • Power, cooling, and vibration
  • Controlling the supply chain
• CEPH Case Study -- Maximizing CEPH Performance