LISA21 Conference Program

Have you ever tried convincing your high school friends that they should use a password manager?

This talk is about how to change people's minds and successfully introduce new concepts to groups and individuals without damaging relationships.

Each of us travels across cultures—work, home, multiple friendship groups. We even travel across micro-cultures—adjacent work teams, different arms of the family. We navigate groups with distinctive communication norms, consent norms, political awareness, technical knowledge, and assumptions of morality—and we do it several times a day.

How do we do that successfully? More importantly, how do we bring new concepts to existing groups without damaging relationships? We do it by accepting that normal is different everywhere, and it doesn't have to remain the same over time.

We will cover change management theory and practice, organisational transformation techniques, complexity communication, and theories of mind. The audience will leave with concrete tools for enabling organisational and group behaviour change, and adaptive communication strategies for translating contexts and concepts across domains.

Control groups (or cgroups for short) are one of the most fundamental technologies underpinning our modern love of containerisation and resource control. Back in 2016, we released a complete overhaul of how cgroups work internally: cgroup v2, released with Linux 4.5. This brought many new and exciting possibilities to increase system stability and throughput, but with those possibilities have also come challenges of a type which we have largely not faced in Linux before.

This talk will go into some of the challenges faced in overhauling Linux's resource isolation and control capabilities, and how we've gone about fixing them. This will include some of the most complex and counter-intuitive practical effects we've seen in production, with details of how our expectations and knowledge have developed over the last 5 years using this on over a million machines in production, with insights that are immediately applicable to anyone who runs Linux at scale.

We will also go over the state-of-the-art of resource control in the "real world" outside of companies like Facebook and Google, looking at how cgroup v2 is changing the technical landscape for distributions and containerisation technologies for the better.

When a company critically relies on the ongoing functioning of a complex and highly interconnected technical stack, support of that stack implies that appropriate personnel be reliably available to troubleshoot and correct issues that occur. These personnel will be referred to as responders. When the scope of a technical stack grows beyond one person's capacity to understand and maintain state, we split up the technical stack such that multiple responders can each provide coverage on a single component of the whole stack. Such a highly interconnected system-of-systems (SoS) allows production issues to cascade throughout wide swaths of the SoS, or sneak in between system-to-system (StS) boundaries. We will here explore one private industry implementation of a responder group designed to respond to emergent distributed computing SoS failures. In contrasting the functions of component responders and SoS responders, we demonstrate that the component ownership skillset is distinguishable from the core skill set of an SoS responder. Technical organizations can benefit from setting up SoS response to enable expedient distributed system outage mitigation.

Every software and firmware component running on a system can be the vector for delivering an attack to the host itself and the wider infrastructure around it. We often focus on protecting the system from what runs in user space or kernel space, and we don't always include in our threat model the integrity of the lower layers in the stack. In this talk, we want to show what could be the impact of compromising a host through a persistent implant in its system firmware. We will focus specifically on UEFI, the industry-wide standard that defines how system firmware should operate. We will demonstrate a "hello-world" system firmware malware from its development to its injection on the host. We will then introduce the concept of Trusted Platform Module, a secure cryptoprocessor that has become an industry standard on consumer and enterprise systems, and explain how the TPM can help protect the platform from our demonstrative malware. We will assume that our system requires secrets to be able to interface with the infrastructure around and we will leverage the TPM to give the host access to those secrets only if we can guarantee that all layers of the stack have not been compromised.

We're quite familiar with managing resources like CPU, memory, bandwidth, storage, I/O—but hidden underneath all that is the power, space, and cooling systems. Pushing the limits of these underlying resources means building a connection between the datacenter facilities & your infrastructure management services. Optimizing requires understanding the very unique industrial controls environments and working around the many issues they have. Come listen to a completely rational tale on why we should go on such adventures, and why we'll be so unhappy with them we want to burn the datacenter to the ground.

No one emerges from the womb an Infrastructure Engineer (except maybe John AllSpaw), and the idea of which skills gaps constitute acceptable hiring risk are highly contested. After a year of struggles, triumphs, and self-reflection, I humbly present my context as a case study in the Sisyphean task of remedying skills gaps in a field where lifelong-learning is a given.

A service mesh is an abstraction layer that binds together microservices-based distributed systems by providing observability, security, and reliability.

This talk will take that description, remove the jargon, and use a demo to show you the power of a service mesh using Linkerd.