/proc isn’t a real directory on disk. It’s a virtual filesystem the kernel generates on the fly, exposing live information about the system and every running process as readable files. Tools like top, ps, free and uptime mostly read from here. Reading it directly gives answers those tools don’t show.

System-wide information

cat /proc/cpuinfo          # CPU model, cores, flags (one block per logical CPU)
cat /proc/meminfo          # detailed memory statistics
cat /proc/loadavg          # load averages, running/total processes, last PID
cat /proc/uptime           # seconds since boot, idle seconds
cat /proc/version          # kernel version and build
cat /proc/cmdline          # kernel boot parameters
cat /proc/filesystems      # filesystems the kernel supports
cat /proc/mounts           # mounted filesystems (same as /etc/mtab)
cat /proc/partitions       # block devices and sizes
cat /proc/swaps            # active swap devices
cat /proc/net/dev          # bytes and packets per network interface

Handy one-liners:

grep -c ^processor /proc/cpuinfo             # number of logical CPUs
grep 'model name' /proc/cpuinfo | head -1    # CPU model
grep -E 'MemTotal|MemAvailable' /proc/meminfo
grep -E 'vmx|svm' /proc/cpuinfo | head -1    # hardware virtualization exposed? (nested virt)

MemAvailable is the best single number for “how much memory can apps still get”. See free command and CPU info.

Per-process information: /proc/PID

Each process has a directory named after its PID:

PID=$(pgrep -o nginx)
ls /proc/$PID

The most useful files:

File Shows
cmdline Full command line (NUL-separated)
environ Environment variables at start (NUL-separated, root or owner only)
status State, memory (VmRSS, VmSwap), threads, UID/GID, capabilities
limits Resource limits: open files, processes…
fd/ Open file descriptors as symlinks
cwd Current working directory (symlink)
exe The executable (symlink). Shows (deleted) if replaced on disk
io Bytes read and written
oom_score / oom_score_adj OOM killer score and adjustment
cgroup Which cgroup / systemd unit the process belongs to

Examples:

tr '\0' ' ' < /proc/$PID/cmdline; echo                 # how exactly was it started?
sudo tr '\0' '\n' < /proc/$PID/environ | grep -i db    # which DB settings does the running app have?
grep -E 'State|VmRSS|VmSwap|Threads' /proc/$PID/status
grep 'open files' /proc/$PID/limits
sudo ls -l /proc/$PID/fd | wc -l
ls -l /proc/$PID/cwd /proc/$PID/exe
cat /proc/$PID/cgroup

environ is especially useful when an app behaves as if a variable is missing. It shows the environment the process actually got, which may differ from your shell’s. See environment variables.

/proc/self always points to the process reading it.

Security investigations

/proc can reveal suspicious processes:

sudo ls -l /proc/*/exe 2>/dev/null | grep deleted        # binaries deleted after starting (malware often does this)
sudo ls -l /proc/*/cwd 2>/dev/null | grep -E '/tmp|/dev/shm'   # processes running from temp dirs

A process running from /tmp or /dev/shm, or whose binary is deleted, deserves a closer look. Common examples are crypto miners started through a vulnerable web app. See Linux server hardening.

/proc/sys: kernel tunables

/proc/sys holds kernel parameters that you can read and (as root) change:

cat /proc/sys/vm/swappiness
cat /proc/sys/net/ipv4/ip_forward
echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward      # temporary change

sysctl is the friendlier interface to the same values, and /etc/sysctl.d/ makes changes permanent. See sysctl tuning.

Other interesting files

cat /proc/pressure/cpu /proc/pressure/memory /proc/pressure/io   # PSI: how much time tasks stall on each resource
cat /proc/sys/fs/file-nr                                          # open file handles system-wide
cat /proc/net/tcp | wc -l                                         # rough count of TCP sockets
cat /proc/interrupts                                              # interrupt counts per CPU
cat /proc/vmstat | grep -E 'pswpin|pswpout|oom_kill'              # swap and OOM counters

Pressure Stall Information (/proc/pressure/*, kernel 4.20+) is a modern, precise measure of resource contention. some avg10=12.5 means some tasks were stalled 12.5% of the last 10 seconds. It’s often more meaningful than load average. See load average.

Things to know

  • Files show 0 bytes in ls -l but have content. They’re generated when read.
  • Values are a snapshot at read time.
  • Some files need root, especially other users’ environ, fd and io.
  • In containers, /proc may show the host’s CPU and memory totals rather than the container’s limits. Check cgroup files (/sys/fs/cgroup/memory.max) for real limits.

Quick reference

Question File
How many CPUs? /proc/cpuinfo
Available memory? /proc/meminfo (MemAvailable)
How was this process started? /proc/PID/cmdline
What env does it have? /proc/PID/environ
How many files open? /proc/PID/fd/
Its limits? /proc/PID/limits
Kernel parameter? /proc/sys/…
Resource stalls? /proc/pressure/*

Related: ps and file descriptors.

  • Linux
  • Troubleshooting
  • Basics