Indrajeet Kumar, LinkedIn
Kafka at Linkedin processes over 3 Trillion messages a day with over 2000 kafka brokers. At such a scale, maintaining balanced workload on kafka clusters as they go through irregular traffic patterns and hardware failures is a daunting task. SREs at Linkedin expend significant time and effort in handling these curveballs and making sure the hardware resources are utilized evenly, which made it quite evident that intelligent automation was crucial to scale any further. This talk outlines Linkedin’s approach towards solving this problem with the help of Kafka Cruise Control.
Indrajeet Kumar, LinkedIn
Indrajeet is part of the Kafka SRE team at Linkedin. He builds tools and automation to help manage the Kafka ecosystem at Linkedin.
Open Access Media
USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.
author = {Indrajeet Kumar},
title = {Autonomous Workload Rebalancing in Kafka},
year = {2018},
publisher = {USENIX Association},
month = jun
}