fio (Flexible I/O tester) is the standard tool for measuring storage performance. It can simulate almost any workload: database-style random reads and writes, large sequential transfers, different queue depths. Unlike dd, it gives meaningful numbers for IOPS, throughput and latency.

Install

sudo apt install fio      # Debian/Ubuntu
sudo dnf install fio      # RHEL family

Ground rules

  • Test on the filesystem you’ll use, with a test file. Never point fio at a raw device that holds data. Writing to /dev/vda destroys it.
  • Use --direct=1 to bypass the page cache. Otherwise you’re measuring RAM.
  • Use a test file larger than any cache layer, and a runtime of at least 30–60 seconds.
  • Delete the test file afterwards.
  • Keep runs short on shared infrastructure.

The four essential tests

Run them in a directory on the disk you’re testing:

cd /root          # or the mount point of the disk under test

1. Random read IOPS (4k)

This is how databases read indexes and rows:

fio --name=randread --filename=fio.test --size=2G --rw=randread --bs=4k \
    --iodepth=32 --numjobs=1 --ioengine=libaio --direct=1 \
    --runtime=30 --time_based --group_reporting

2. Random write IOPS (4k)

fio --name=randwrite --filename=fio.test --size=2G --rw=randwrite --bs=4k \
    --iodepth=32 --numjobs=1 --ioengine=libaio --direct=1 \
    --runtime=30 --time_based --group_reporting

3. Sequential read and write throughput (1M)

Backups, large files, video:

fio --name=seqread --filename=fio.test --size=2G --rw=read --bs=1M \
    --iodepth=8 --ioengine=libaio --direct=1 --runtime=30 --time_based --group_reporting
fio --name=seqwrite --filename=fio.test --size=2G --rw=write --bs=1M \
    --iodepth=8 --ioengine=libaio --direct=1 --runtime=30 --time_based --group_reporting

4. Mixed, database-like (70% read)

fio --name=mixed --filename=fio.test --size=2G --rw=randrw --rwmixread=70 --bs=4k \
    --iodepth=16 --numjobs=4 --ioengine=libaio --direct=1 \
    --runtime=60 --time_based --group_reporting

Clean up with rm fio.test.

Reading the output

randread: (groupid=0, jobs=1): err= 0
  read: IOPS=48.2k, BW=188MiB/s (197MB/s)(5650MiB/30001msec)
    clat percentiles (usec):
     |  1.00th=[  330],  5.00th=[  400], 10.00th=[  445], 50.00th=[  603],
     | 90.00th=[  857], 95.00th=[  988], 99.00th=[ 1418], 99.50th=[ 1729],
     | 99.90th=[ 3687], 99.95th=[ 5211], 99.99th=[ 9634]
  • IOPS — operations per second. The key number for random 4k tests.
  • BW — bandwidth. The key number for sequential 1M tests.
  • clat percentiles — completion latency. p99 and p99.9 show consistency. A disk with high IOPS but occasional 50 ms spikes makes a database stutter.

Rough expectations:

Storage Random 4k IOPS (QD32) Sequential
HDD (7200 rpm) 100–200 150–250 MB/s
SATA SSD 20k–90k ~500 MB/s
NVMe SSD 100k–1M+ 2–7 GB/s
Cloud/VPS virtual disk Depends on platform and plan limits Varies

Virtual disks on a VPS share physical storage and may be rate-limited. Compare plans and providers with the same fio commands.

Queue depth and jobs

  • --iodepth is how many I/Os are in flight per job. Real single-threaded apps often run at QD1–4, and databases with many connections run higher.
  • --numjobs is parallel workers.

For latency-sensitive workloads, test QD1. It shows the raw latency of one operation:

fio --name=qd1 --filename=fio.test --size=1G --rw=randread --bs=4k --iodepth=1 \
    --ioengine=libaio --direct=1 --runtime=30 --time_based

Job files

Save repeatable tests in a file:

# disk.fio
[global]
ioengine=libaio
direct=1
size=2G
runtime=30
time_based
filename=fio.test
group_reporting

[randread]
rw=randread
bs=4k
iodepth=32

[randwrite]
stonewall
rw=randwrite
bs=4k
iodepth=32

Run it with fio disk.fio. stonewall makes jobs run one after another instead of in parallel. Add --output-format=json for scripted comparisons.

Common mistakes

  • dd as a benchmark: it writes sequentially from /dev/zero, and compression or caching inflates results.
  • Forgetting --direct=1: you’ll see impossible numbers from RAM.
  • Tiny test files that fit in the controller or host cache.
  • Testing once: results on shared storage vary. Run several times.
  • Testing /tmp on tmpfs: that’s RAM, not disk.

Quick reference

Test Key options
Random read IOPS --rw=randread --bs=4k --iodepth=32
Random write IOPS --rw=randwrite --bs=4k --iodepth=32
Sequential --rw=read|write --bs=1M
Database-like --rw=randrw --rwmixread=70
Latency --iodepth=1
Always --direct=1 --ioengine=libaio --runtime=30 --time_based

Related: iostat, benchmark a VPS and NVMe vs SSD.

  • Performance
  • Storage
  • Linux