Matt Fleming, Cloudflare
Modern systems are inherently nondeterministic and that leads to noisy benchmark results. Change Point Detection has emerged as a helpful technique for detecting significant changes in performance results even when those results are noisy and unstable. This talk will explain how Change Point Detection works and the open source projects available for developers to use CPD with noisy benchmark results.
Matt Fleming, Cloudflare
Matt is co-founder of Nyrkiö and a Systems Engineer at Cloudflare. He has spent over 15 years working on low-level, high-performance systems and was previously the maintainer for the Linux kernel EFI subsystem. He has co-authored papers on performance change detection and distributed systems testing and served on the ACM/SPEC ICPE program committee. Matt can often be found on Twitter, discussing topics such as software performance, benchmarking and statistics.
author = {Matt Fleming},
title = {Taming Noisy Benchmark Results Using Change Point Detection},
year = {2024},
address = {Dublin},
publisher = {USENIX Association},
month = oct
}