Comparison with SystemTap and DTrace
Feature Comparison
| LTTng | Systemtap | DTrace | |
| Project | |||
| license | Kernel tracer: GPLv2 Userspace tracer: LGPLv2.1 (library) and GPLv2 (tools) |
GPL | CDDL |
| operating system support | Linux | Linux | Solaris, Mac OS X, BSD, QNX |
| processor support | Kernel tracer: x86-32, x86-64, SPARC, SPARC64, ppc, ppc64, sh, sh64, ia64, s390, MIPS 32/64, ARM, (arch-agnostic core) Userspace tracer: x86-32, x86-64 |
as per kprobes: x86-32, x86-64, ppc64, ia64, s390, arm, sparc? | x86-32, x86-64, SPARC, ppc, ppc64 |
| overhead | low | high | high |
| core developers | open community | open community | open community |
| development began | January 2005 | January 2005 | October 2001 |
| development status | ongoing | ongoing | stable with continuing development |
| target audience | application (userspace) developers, kernel developers, users, sysadmins | developers, users, sysadmins | developers, users, sysadmins |
| target usage | debugging, tracing, profiling, monitoring | debugging, tracing, profiling | debugging, tracing, profiling |
| Language | |||
| style | C | scripting | scripting |
| speculative tracing | no | yes (from first principles via auxiliary data and control structures) | yes |
| binary tracing | yes | yes | ? |
| early boot tracing | not "as early as it could" yet | not yet [[http://sourceware.org/PR2035] | yes |
| probe execution | optimized native code | optimized native code | interpreted bytecodes |
| Probing Capability | |||
| number of available symbolic probe points in the kernel | thousands (functions, tracepoints, markers) | millions (statements, markers) | thousands (functions, markers) |
| type of instrumentation available in user-space | Markers inserted by the developer. GDB tracepoints are used for statement and function instrumentation. | statements, functions | functions, developer-inserted markers |
| high-speed breakpoint-less userspace data extraction | yes | no | no |
| statically inserted probe points, user side | yes, fast, breakpoint-less and system call-less | in progress http://sourceware.org/PR1918 (slower, breakpoint based) | yes (USDT) (slower, breakpoint based) |
| probe arbitrary statements in code symbolically (function entry, exit, interior, source code co-ordinates) | Kernel tracer: within kprobes capabilities Userspace tracer: with gdb tracepoints |
yes (using debugging information) | limited to ABI boundaries (function entry/exit, markers) |
| statically identify any arbritrary data to extract | yes (with tracepoints and markers) | yes (with help of markers) | yes (with markers) |
| non-symbolic (manually addressed) probe points / data | Kernel tracer: yes, via kprobes Userspace tracer: with gdb tracepoints |
yes | yes |
| probe dynamically loaded kernel objects | yes | yes | yes |
| concurrent probes on multiprocessors | yes | yes | yes |
| context pointer type punning/casting | yes | not yet http://sourceware.org/PR5634 | yes |
| statically inserted probe points, kernel side | yes (tracepoints, markers) | yes (markers) | yes (SDT) |
| end-user extendable probe library | yes | yes (script based tapsets) | no |
| trace user-space stack backtraces | had this feature previously, not currently | soon http://sourceware.org/PR5635 | yes |
| trace Java programs | yes, via debugfs special file (highly efficient java tracing in progress), or via userspace tracer | soon (special case of user-space) | yes |
| trace Java stack backtraces | no | no | yes |
| statically inserted probe points, Java | in progress | no | yes |
| trace script language programs | yes, if the script language can call C | not yet | yes: Ruby, JavaScript, Perl, Python, PHP, APL, Bourne shell, ksh, zsh, Tcl |
| timer-based probing | no | yes (profiling interrupt, software timers) | yes |
| hardware performance counter based probing | soon | soon http://sourceware.org/PR909 | soon |
| Safety | |||
| non-blocking, atomic probe handlers | yes | yes | yes |
| space-limited execution | yes (bounded to buffer size + probe size + memory allocated by probe) | yes (static allocation of all data) | yes |
| Analysis performed | offline (post-mortem) | online | online |
| Trace streaming support | yes, traces sent efficiently through TCF (http://git.dorsal.polymtl.ca/?p=lttng-agent.git) | ? | ? |
| Built-into the system | yes (mainlining in progress) | no | yes |
| Lockless tracing | yes | no | yes |
| Efficiency | |||
| Zero-copy trace data extraction | yes | ? | no (designed to allow for concise traces) |
| Provides analysis tools to navigate in large multi-GB traces | yes (lttv), and upcoming Eclipse Linux Tools Tracing and Monitoring Framework (link) | no | no (designed to allow for concise traces) |
This table is based upon the one on the Systemtap site.
Userspace Tracing Performance Comparison of LTTng / DTrace
See the paper Combined Tracing of the Kernel and Applications with LTTng presented at the Linux Symposium 2009.