Dan Lambright, Facebook; Nakul Vankadari Ramesh, Akriti Bhat, Xing Du, and Anand Kumar, Northeastern University
Lustre is a distributed file system popular with high performance computing (HPC) workloads. In this talk, we will describe how we run Lustre within Kubernetes. This makes Lustre portable across different cloud platforms and helps automate deployment, elasticity and management. We’ll show how we used Kubevirt to get Lustre’s kernel modules into containers and show our performance measurements using that infrastructure. Our objective has been to integrate Lustre as closely as possible within Kubernetes, and we will demonstrate how we provision Lustre infrastructure using standard YAML configuration files.
The second part of our talk will discuss our work-in-progress to leverage the Rook framework. We wish to use it to help implement features such as autoscaling and node failure recovery. We are developing a Kubernetes operator that interacts with the KubeVirt operator for VM provisioning and other tasks. We hope to collaborate with the Rook community on this.
We are a team of four graduate students from Northeastern University, mentored by an industry specialist. Our work was done on the Massachusetts Open Cloud (MOC).
Dan Lambright, Facebook
Dan Lambright has given talks at Vault in the past. He is currently at Facebook working on social graph consistency, and enjoying working with the team at Northeastern.
Nakul Vankadari Ramesh, Northeastern University
Nakul Vankadari Ramesh previously worked at Intel, Schneider Electric and Philips. He is keen on learning cloud technologies for large scale systems and enjoys exploring various efforts in open-source.
Akriti Bhat, Northeastern University
Akriti Bhat previously worked with J.P. Morgan and Amazon Web Services. She is passionate about developing software to support large scale applications.
Anand Kumar, Northeastern University
Anand Kumar is pursuing concurrent Bachelors and Masters degrees in Computer Science. Throughout academics and his internships at Google and Circle, he has enjoyed creating software to build comprehensive user experiences.
Open Access Media
USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.
author = {Dan Lambright and Nakul Vankadari Ramesh and Akriti Bhat and Anand Kumar},
title = {Lustre in Kubernetes},
year = {2020},
address = {Santa Clara, CA},
publisher = {USENIX Association},
month = feb
}