ALS 2000 Abstract
Dynamic Probes
and Generalised Kernel Hooks Interface for Linux
Richard J. Moore, IBM Corporation, Linux Technology Centre
Abstract
Dynamic Probes (Dprobes)
is a generic and pervasive system debugging facility that will operate
under the most extreme software conditions such as debugging a deep rooted
operating system problems in a live environment. For example, page-manager
bugs in the kernel or perhaps user or system problems that will not re-create
easily in either a lab or production environment. For such inaccessible
problem scenarios Dprobes not only offers a technique for gathering diagnostic
information but has a high probability of successful outcome without the
need to build custom modules for debugging purposes.
Dprobes allows the
insertion of fully automated breakpoints or probepoints, anywhere in the
system and user space. Probepoints are global by definition, that is they
are defined relative to a module and not to a storage address. Each probepoint
has an associated set of probe instructions that are interpreted when the
probe fires. These instructions allow memory and CPU registers to be examined
and altered using conditional logic. When the probe program terminates
an external debugging facility may be optionally triggered - should it
register for this purpose. For example:
-
A trace facility may augment
its capability with a dynamic trace capability by using the Dprobes facility
as a means of inserting tracepoints - dynamically, without any prior code
modification.
-
A crash dump facility
may use Dprobes as a means of invoking dumps conditionally when a specific
set of circumstances occurs in a particular code path.
-
A debugger may use Dprobes
as high-speed complex conditional breakpoint service.
This paper describes the
architecture of Dynamic Probes and briefly discusses a couple of examples
of its successful application.
In creating Dynamic
Probes, we were challenged with the conflicts between:
-
Size of the kernel modification
-
Co-existence with other
kernel enhancements, particularly debugging and instrumentation enhancements.
-
Maintaining concurrency
with the latest kernel version.
-
Ease of development and
continued enhancement of Dynamic Probes.
We alleviated these
problems by developing a generalised interface for kernel modifications
to use allowing them to exist as dynamically loadable kernel modules.
This interface: The
Generalised
Kernel Hooks Interface (GKHI) is described in the second part of this
paper.
|