node_exporter is Prometheus’s standard agent for Linux hosts. It exposes hundreds of system metrics (CPU, memory, disk, filesystem, network, load and more) on an HTTP endpoint, which Prometheus scrapes every few seconds. It’s the foundation of most Prometheus and Grafana server monitoring. This guide installs it, connects it securely, and gives you the queries and alerts that matter.

For the Prometheus and Grafana side, see Prometheus and Grafana monitoring.

Install node_exporter

On each server you want to monitor:

VER=1.8.2     # check the latest release on GitHub
cd /tmp
curl -LO https://github.com/prometheus/node_exporter/releases/download/v$VER/node_exporter-$VER.linux-amd64.tar.gz
tar xzf node_exporter-$VER.linux-amd64.tar.gz
sudo install -m 755 node_exporter-$VER.linux-amd64/node_exporter /usr/local/bin/
sudo useradd -r -s /usr/sbin/nologin node_exporter

(Distribution packages, such as prometheus-node-exporter on Debian and Ubuntu, also work, but can lag behind.)

systemd unit:

# /etc/systemd/system/node_exporter.service
[Unit]
Description=Prometheus node_exporter
After=network-online.target

[Service]
User=node_exporter
ExecStart=/usr/local/bin/node_exporter --web.listen-address=10.0.0.11:9100 --collector.systemd
Restart=on-failure
NoNewPrivileges=true
ProtectSystem=strict
ProtectHome=true

[Install]
WantedBy=multi-user.target
sudo systemctl daemon-reload && sudo systemctl enable --now node_exporter
curl -s http://10.0.0.11:9100/metrics | head

--collector.systemd adds systemd unit states (great for “is nginx running?” alerts). See systemd unit files.

Don’t expose metrics publicly

Metrics reveal a lot about your server. Options:

  • Listen on a private IP or a VPN interface (WireGuard, Tailscale), as above, and firewall port 9100 to the Prometheus server only. See WireGuard.
sudo ufw allow from 10.0.0.5 to any port 9100 proto tcp
  • Or use node_exporter’s TLS and basic auth (--web.config.file) when scraping over the internet.

Scrape it from Prometheus

# prometheus.yml
scrape_configs:
  - job_name: node
    scrape_interval: 15s
    static_configs:
      - targets: ['10.0.0.11:9100', '10.0.0.12:9100']
        labels:
          env: production

Reload Prometheus, and check Status → Targets. Every target should be UP.

Useful PromQL queries

CPU usage (%) per instance:

100 - (avg by (instance) (rate(node_cpu_seconds_total{mode="idle"}[5m])) * 100)

CPU steal (%):

avg by (instance) (rate(node_cpu_seconds_total{mode="steal"}[5m])) * 100

See CPU steal time.

Memory available (%):

node_memory_MemAvailable_bytes / node_memory_MemTotal_bytes * 100

Disk usage (%) per filesystem:

100 - (node_filesystem_avail_bytes{fstype!~"tmpfs|overlay"} / node_filesystem_size_bytes{fstype!~"tmpfs|overlay"} * 100)

Disk will fill within 24 hours (linear prediction):

predict_linear(node_filesystem_avail_bytes{fstype!~"tmpfs|overlay"}[6h], 24*3600) < 0

Network throughput (bits/s):

rate(node_network_transmit_bytes_total{device!="lo"}[5m]) * 8

Load per core:

node_load5 / count without (cpu, mode) (node_cpu_seconds_total{mode="idle"})

See load average.

Alert rules

# alerts.yml
groups:
  - name: node
    rules:
      - alert: InstanceDown
        expr: up{job="node"} == 0
        for: 2m
      - alert: DiskAlmostFull
        expr: (node_filesystem_avail_bytes{fstype!~"tmpfs|overlay"} / node_filesystem_size_bytes{fstype!~"tmpfs|overlay"}) < 0.10
        for: 10m
      - alert: DiskWillFillIn24h
        expr: predict_linear(node_filesystem_avail_bytes{fstype!~"tmpfs|overlay"}[6h], 24*3600) < 0
        for: 30m
      - alert: LowMemory
        expr: node_memory_MemAvailable_bytes / node_memory_MemTotal_bytes < 0.10
        for: 10m
      - alert: HighCPU
        expr: 100 - (avg by (instance) (rate(node_cpu_seconds_total{mode="idle"}[5m])) * 100) > 90
        for: 15m
      - alert: SystemdUnitFailed
        expr: node_systemd_unit_state{state="failed"} == 1
        for: 5m

Route alerts with Alertmanager to email, Slack or Telegram. Use for: durations, so brief spikes don’t page you. See server security monitoring.

Grafana dashboard

Import the community dashboard “Node Exporter Full” (ID 1860) in Grafana. It gives you complete host dashboards instantly. Build simpler overview panels for daily use.

Beyond node_exporter

Add exporters for your services: mysqld_exporter, postgres_exporter, redis_exporter, nginx-prometheus-exporter, blackbox_exporter (HTTP/TLS checks from outside), and cAdvisor for containers. See PostgreSQL monitoring.

Quick reference

Task Command / query
Metrics endpoint http://host:9100/metrics
Service systemctl status node_exporter
CPU % 100 - avg(rate(node_cpu_seconds_total{mode="idle"}[5m]))*100
Memory available node_memory_MemAvailable_bytes
Disk prediction predict_linear(node_filesystem_avail_bytes[6h], 86400)
Dashboard Grafana ID 1860

Related: Netdata and Zabbix.

  • Monitoring
  • Prometheus
  • Linux