lttng-calibrate — Quantify LTTng overhead
lttng [GENERAL OPTIONS] calibrate
lttng calibrate commands quantifies the overhead of LTTng tracers.
lttng calibrate command can be used to find out the combined
average overhead of the LTTng tracers and the instrumentation mechanisms
used. This overhead can be calibrated in terms of time or using any of
the PMU performance counter available on the system.
For now, the only implemented calibration is the Linux kernel function instrumentation (kretprobes).
As an example, we use an i7 processor with 4 general-purpose PMU
registers. This information is available by issuing
The following sequence of commands gathers a trace executing a kretprobe
hooked on an empty function, gathering PMU counters LLC
(Last Level Cache) misses information (use
lttng add-context --list to
get the list of available PMU counters).
lttng create calibrate-function lttng enable-event calibrate --kernel \ --function=lttng_calibrate_kretprobe lttng add-context --kernel --type=perf:cpu:LLC-load-misses \ --type=perf:cpu:LLC-store-misses \ --type=perf:cpu:LLC-prefetch-misses lttng start for a in $(seq 1 10); do lttng calibrate --kernel --function done lttng destroy babeltrace $(ls -1drt ~/lttng-traces/calibrate-function-* | tail -n 1)
The output from babeltrace(1) can be saved to a text file and
opened in a spreadsheet (for example, in LibreOffice) to focus on the
per-PMU counter delta between consecutive
calibrate_return events. Note that these counters are per-CPU, so
scheduling events would need to be present to account for migration
between CPUs. Therefore, for calibration purposes, only events staying
on the same CPU must be considered.
Here’s an example of the average result, for the i7, on 10 samples:
|PMU counter||Average||Standard deviation|
As we can notice, the load and store misses are relatively stable across runs (their standard deviation is relatively low) compared to the prefetch misses. We could conclude from this information that LLC load and store misses can be accounted for quite precisely, but prefetches within a function seems to behave too erratically (not much causality link between the code executed and the CPU prefetch activity) to be accounted for.
Set to 1 to abort the process after the first error is encountered.
$HOME environment variable. Useful when the user
running the commands has a non-writable home directory.
Absolute path to the man pager to use for viewing help information
about LTTng commands (using lttng-help(1) or
lttng COMMAND --help).
Path in which the
session.xsd session configuration XML
schema may be found.
Full session daemon binary path.
--sessiond-path option has precedence over this
Note that the lttng-create(1) command can spawn an LTTng session daemon automatically if none is running. See lttng-sessiond(8) for the environment variables influencing the execution of the session daemon.
User LTTng runtime configuration.
This is where the per-user current tracing session is stored between executions of lttng(1). The current tracing session can be set with lttng-set-session(1). See lttng-create(1) for more information about tracing sessions.
Default output directory of LTTng traces. This can be overridden
--output option of the lttng-create(1)
User LTTng runtime and configuration directory.
$LTTNG_HOME defaults to
$HOME when not explicitly set.
Command warning (something went wrong during the command)
If you encounter any issue or usability problem, please report it on the LTTng bug tracker.
Special thanks to Michel Dagenais and the DORSAL laboratory at École Polytechnique de Montréal for the LTTng journey.
Also thanks to the Ericsson teams working on tracing which helped us greatly with detailed bug reports and unusual test cases.