Linux Performance Analysis Using Perf and BPF

This tutorial will give you experience with two powerful Linux performance analysis tools: perf and BPF. Learn how to profile CPU usage, create flame graphs, trace TCP connections, investigate file system latency, explore software internals, and more.

perf_events, aka "perf" after its front-end, is a Linux mainline tool for profiling and tracing. We will summarize some of its most useful one-liners, and discuss real world challenges and solutions for using it with JIT runtimes (Java, Node.js), and in cloud environments.

Enhanced BPF (Berkeley Packet Filter) is a new in-kernel programmable runtime with a variety of uses, including extending Linux static and dynamic tracing capabilities. We'll primarily focus on the BPF Compiler Collection (bcc) front-end for BPF, which provides a toolkit of many ready-to-run analysis tools, including DTrace classics like execsnoop, opensnoop, and biolatency, and new tools including memleak, trace, and argdist. bcc also provides Python and C interfaces for writing your own powerful dynamic tracing-based tools, and we'll show how that can be done.

We will spend more time exploring the new world of BPF and its features that were made available in the Linux 4.4 release. Enhanced BPF has become a recent hotspot for systems innovation, helping create other new technologies including bcc, kernel connection multiplexer (KCM), and eXpress Data Path (XDP), and is being developed by engineers from many companies, including Facebook, PLUMGrid, Netflix, Cisco, Huawei, Github, SELA, and Intel. Join this workshop to get up to speed with BPF for tracing, try some hands-on labs, and gain real experience with the technology from contributor and performance expert Brendan Gregg.