More Data Science, Less Engineering: A Netflix Original

Data Science usage at Netflix goes much beyond our eponymous recommendation systems. It touches almost all aspects of our business - from optimizing content delivery to making our infrastructure more resilient to failures and beyond. Also, our unique culture affords our data scientists extraordinary freedom of choice in ML tools and libraries, all of which results in an ever-expanding set of interesting problem statements and a diverse set of ML approaches to tackle them. Our data scientists, at the same time, are expected to build, deploy, and operate complex ML workflows autonomously without the need to be significantly experienced with systems or data engineering.

In this talk, we discuss the infrastructure available to our data scientists focused on providing an improved development and deployment experience for ML workflows. We focus on Metaflow (now open source at metaflow.org), our ML framework, which offers delightful abstractions to manage the model’s lifecycle end-to-end and how our culture and focus on human-centric design affects our data scientist’s velocity.