Monitoring without Infrastructure @ Airbnb

Millions of requests flow through Airbnb’s systems in a given day. Interruptions in traffic can be extremely costly for us, and leave our users stranded in foreign countries with no recourse. We needed a comprehensive suite of introspection tools which would allow us to prevent or quickly identify and remediate any issues.

However, we didn’t have the engineering resources to build our own in-house monitoring system. Instead, we levered a combination of open-source software, third-party vendors, and just enough glue code to make it all stick together.

All all of these tools are available to you, too! Come hear how we manage tens of thousands of metrics and billions of log lines per day from thousands of machines, all operated by less then a full-time engineer. We will cover logstash, statsd, NewRelic, Datadog, and our own open-sourced configuration-as-code alerting framework.

Igor Serebryany is an engineer on Airbnb’s Developer Happiness team. He is the author of SmartStack, an open-source distributed service discovery framework. Prior to building the backend infrastructure at Airbnb, he has worked on running Hadoop clusters, automating datacenters, and running scientific computing simulations in biology and astrophysics.