The Evolution of Storage Networking: Blending Object Storage with the Conventional Storage Stack

Object storage is the latest and greatest trend in storage networking. . . or is it?  The reality is that object stores have been around forever, but people are talking about objects as if they were something new, largely because Amazon commercialized an object-based storage service (S3) and lots of other companies are now marketing products based on a similar paradigm. Object stores are, in fact, ubiquitous. They are quietly embedded in the inner working of EVERY system that stores data.

So really, this lecture is not so much about object storage as it is about the ever-evolving storage "stack" that originated decades ago with lowly hard drives and simple file systems, then morphed into modern SAN and NAS solutions, and that will continue to evolve as long there are new challenges to overcome.  We begin with a simple definition of object and object store, outline key concepts, and then illustrate different ways to leverage object storage models to achieve new levels of performance, scale, consistency, and resiliency. The concepts are illustrated with real-world examples from commercial and community products.

Lecture Outline

• The evolution of the traditional storage stack - block/SAN and POSIX/NAS
• Storage virtualization
  • Object storage concepts
  • Object storage for resiliency and scale
  • Object storage for performance and scale
  • Object storage for locality, consistency, and coherency
  • Content addressing and deduplication
  • Recipe-based storage
• Big Metadata
• Fine Grained v. Coarse Grained Objects
• Integrating object stores into the traditional storage stack
  • Object-based file systems
  • File system gateways
• Nesting of object stores
  • Jelly beans and Jars
  • Nested containers
  • Protocol shims
• Federated storage and global namespaces
  • Making files behave like objects
  • Making objects behave like files