A Scalable and Resilient Microservice Environment with Apache Mesos and Apache Aurora

"Treat your data center as a single machine." This idea has been getting more and more traction over the last years. Beside a couple of other projects, there's Apache Mesos which is providing a solution to easily create this one big single pool of your resources. There are companies which are running it on 10,000s of machines, but for an architecture that relies on easy scalability and good resilience, it completely makes sense to run it on a small cluster as well. A framework that runs upon Mesos is the scheduler Aurora, which takes care about how many instances of a job should run on which machines, and reschedules running jobs if machines in your cluster die.

After the introduction of these projects, I will show you how and why Gutefrage.net has glued together those technologies with Jenkins to implement a continuous deployment workflow, that takes care of our 100+ daily deployments on a relatively small Mesos cluster and has the goals of providing a fault tolerant and low latency user experience to our customers.

Florian is Head of Data and Infrastructure at gutefrage.net. Before that he learned the ropes at Yahoo! Together with his team he accepts the challenges that running Germany's biggest Q&A site brings. As an agile company, this not only means the usual scaling and high availability topics, but also multiple daily releases and branchless development with feature switches.

In February 2014 he introduced Mesos as the basic building block for the next generation platform for gutefrage.net