eBPF lets you run small, verified programs inside the Linux kernel, safely, without writing kernel modules or rebooting. It powers modern observability (seeing exactly what the system is doing), networking (Cilium, load balancers), and security tools (Falco, Tetragon). For server admins, the most immediately useful part is a set of ready-made tracing tools that answer questions like “what’s opening this file?” or “why is disk I/O slow?”.

Why it’s useful

Traditional tools show summaries: top shows CPU, iostat disk utilisation. eBPF tools show events and latencies as they happen:

  • every new process that starts (even short-lived ones that top never shows);
  • every file opened, and by whom;
  • disk I/O latency distributions;
  • TCP connections, retransmits and their latency;
  • slow filesystem operations.

All with low overhead, safe enough for production servers.

Requirements

  • A reasonably modern kernel (4.9+ for basics; 5.x+ recommended). Check with uname -r.
  • Root privileges (or specific capabilities).
  • A KVM VPS or dedicated server: eBPF tracing needs your own kernel, so it isn’t available in OpenVZ/LXC containers. See what is KVM virtualization.

Install the tools

# Debian/Ubuntu
sudo apt install bpfcc-tools bpftrace linux-headers-$(uname -r)
# RHEL/AlmaLinux
sudo dnf install bcc-tools bpftrace

On Debian/Ubuntu, BCC tools are named with a -bpfcc suffix (execsnoop-bpfcc). On RHEL, they live in /usr/share/bcc/tools/.

Ready-made BCC tools

Tool Shows Use when
execsnoop Every new process, with arguments Mystery CPU spikes, cron jobs, suspicious commands
opensnoop Files being opened “Which config file is it reading?”, missing files
biolatency Disk I/O latency histogram Slow disk suspected
biosnoop Each disk I/O, with process and latency Which process hits the disk
tcpconnect / tcpaccept Outgoing / incoming TCP connections Unexpected connections, debugging services
tcpretrans TCP retransmits Network quality problems
ext4slower / xfsslower Slow filesystem operations Latency investigations
runqlat CPU run-queue latency CPU contention
profile Stack sampling Where CPU time goes

Examples:

sudo execsnoop-bpfcc                 # watch new processes (Ctrl+C to stop)
sudo opensnoop-bpfcc -n nginx        # files opened by nginx
sudo biolatency-bpfcc 5 1            # 5-second disk latency histogram
sudo tcpconnect-bpfcc                # outgoing connections

execsnoop is especially good at revealing short-lived processes (scripts spawned thousands of times by cron or an app), which never stay long enough to appear in top. See Linux CPU usage high.

bpftrace one-liners

bpftrace is a small scripting language for custom questions:

# count system calls by process
sudo bpftrace -e 'tracepoint:raw_syscalls:sys_enter { @[comm] = count(); }'

# files opened, with the process name
sudo bpftrace -e 'tracepoint:syscalls:sys_enter_openat { printf("%s %s\n", comm, str(args->filename)); }'

# distribution of read() sizes for mysqld
sudo bpftrace -e 'tracepoint:syscalls:sys_exit_read /comm == "mysqld"/ { @bytes = hist(args->ret); }'

Press Ctrl+C to print the results.

eBPF in other tools

  • Networking: Cilium (Kubernetes networking), Katran (load balancing), XDP-based DDoS filtering.
  • Security: Falco and Tetragon detect suspicious behaviour (a shell spawned in a container, unexpected outbound connections).
  • Observability: Pixie, Parca, and Grafana Beyla collect metrics and profiles without code changes.

Safety

eBPF programs are checked by the kernel’s verifier before running, so they can’t crash the kernel or loop forever. Tracing still adds some overhead. Use targeted filters (a process name, a duration) on busy production servers, and don’t leave heavy traces running.

Quick reference

Question Tool
What processes are starting? execsnoop
What files are opened? opensnoop
Is the disk slow? biolatency, biosnoop
What connections are made? tcpconnect, tcpaccept
Network retransmits? tcpretrans
Custom question bpftrace -e '…'

Related: perf profiling and Linux I/O wait.

  • Linux
  • Performance
  • Observability