Conveyor: One-Tool-Fits-All Continuous Software Deployment at Meta

Authors: 

Boris Grubic, Meta; Yang Wang, Meta and the Ohio State University; Tyler Petrochko, Ran Yaniv, Brad Jones, David Callies, Matt Clarke-Lauer, and Dan Kelley, Meta; Soteris Demetriou, Meta and Imperial College London; Kenny Yu and Chunqiang Tang, Meta

Abstract: 

We present Conveyor, Meta's software deployment tool, along with the valuable data obtained from managing over 30,000 deployment pipelines that deploy all kinds of services at Meta across millions of machines. We describe a wide range of deployment scenarios that Conveyor supports to achieve universal coverage. At Meta, out of all the deployment pipelines for services deployed via containers, 97% of them employ fully automated deployments without manual intervention: 55% utilize continuous deployment, instantly deploying every code change to production after passing automated tests, and the remaining 42% are automatically deployed on a fixed schedule (mostly daily or weekly) without manual validation. We highlight several distinguishing features of Conveyor, including safe in-place updates to reduce hardware costs, analysis of code dependencies to prevent faulty releases, and the capability to deploy complex ML models at scale.

OSDI '23 Open Access Sponsored by
King Abdullah University of Science and Technology (KAUST)

Open Access Media

USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.

BibTeX
@inproceedings {288560,
author = {Boris Grubic and Yang Wang and Tyler Petrochko and Ran Yaniv and Brad Jones and David Callies and Matt Clarke-Lauer and Dan Kelley and Soteris Demetriou and Kenny Yu and Chunqiang Tang},
title = {Conveyor: {One-Tool-Fits-All} Continuous Software Deployment at Meta},
booktitle = {17th USENIX Symposium on Operating Systems Design and Implementation (OSDI 23)},
year = {2023},
isbn = {978-1-939133-34-2},
address = {Boston, MA},
pages = {325--342},
url = {https://www.usenix.org/conference/osdi23/presentation/grubic},
publisher = {USENIX Association},
month = jul
}

Presentation Video