Journey from Fluent Bit, Fluentd and Prometheus to OpenTelemetry Collector - Lessons Learned

Thursday, 12 October, 2023 - 10:1010:30

Marcin "Perk" Stożek, Canonical

Abstract: 

Telemetry collection seems very straightforward at first glance: just use Fluent Bit or Fluentd for logs, Prometheus for metrics and call it a day. But problems arise when we want the best performance or simplicity of the solution. It’s hard to maintain three different agents, with three different configurations and documentation in three different places. In the talk, Perk discusses why his team replaced Fluent Bit, Fluentd and Prometheus with OpenTelemetry Collector and the challenges they faced along the way. Most importantly - is there a happy ending?

View the animated slides. (PDF slides available for download below.)

Marcin "Perk" Stożek, Canonical

Product manager by day, developer by night. Remote work practitioner.

My dev journey started with a copy-book and a pencil for C64 picture bytes calculations. Since then I have been involved in many different projects - ranging from security in message queues to Hadoop; from monolith to microservices; from bare metal to Kubernetes; from software tester, through dev and ops to lead and manager.

I love automation and believe that Linux is the best thing since sliced bread. Well, maybe after Vim.

BibTeX
@conference {292076,
author = {Marcin "Perk" Sto{\.z}ek},
title = {Journey from Fluent Bit, Fluentd and Prometheus to {OpenTelemetry} Collector - Lessons Learned},
year = {2023},
address = {Dublin},
publisher = {USENIX Association},
month = oct
}

Presentation Video