Ingrid Epure, Intercom
Shipping code is not enough. You also need logs, tests and static analyzers to have the necessary confidence in the change you just deployed. Especially when you’re shipping to production every 10 minutes, at peak time.
A common philosophy for feeling that things are in control is simply adding more data. And when something bad happens, first reaction is even more instrumentation to cover that specific scenario. You need more information in the logs to troubleshoot, more inputs for your tests and more linting rules. And then you’ll never run into that problem again, right!?
Well…maybe, but you’ve just hit a bigger one:
When your app is small, you can easily get away with test duplication, log noisiness and introducing new tools.
As you grow, the noise becomes impactful. You’re being slowed down by the shortcuts and poor decisions you took earlier, until it becomes a non trivial problem to solve.
In this talk we will explore the approach we took in Intercom for introducing information sanity, by focusing on:
- logging myths, and dangerous fallacies
- being deliberate about operational and engineering needs
- better troubleshooting using structured and canonical logs
- getting more benefits and performance out of notoriously slow static analyzers
- writing tests with performance in mind
Ingrid Epure, Intercom
Ingrid is an engineer currently working for Intercom in Dublin, Ireland. She is passionate about distributed systems, automation and simplifying things. She is a conference speaker, an active member in the Python community and loves mentoring and helping with community-driven events.
Open Access Media
USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.
author = {Ingrid Epure},
title = {The Dangers of Being {Overly-Paranoid}},
year = {2017},
address = {Dublin},
publisher = {USENIX Association},
month = aug
}