Every Linux command has three standard streams, and the shell lets you connect them to files or other commands. Pipes and redirection are what make the command line so powerful: you chain small tools into one-liners that search logs, save output, or silence noisy cron jobs.

The three standard streams

Stream Number Default Purpose
stdin 0 Keyboard Input
stdout 1 Terminal Normal output
stderr 2 Terminal Errors and diagnostics

Keeping stdout and stderr separate means you can save results while still seeing errors (or the other way around).

Output redirection

command > file.txt        # stdout to file (overwrite)
command >> file.txt       # stdout appended
command 2> errors.txt     # stderr to file
command > out.txt 2>&1    # stdout AND stderr to the same file
command &> out.txt        # same, Bash shorthand
command 2>&1 | less       # merge stderr into the pipe

Order matters: > file 2>&1 sends both to the file. 2>&1 > file sends stderr to the terminal (where stdout pointed at that moment) and stdout to the file.

Protect files from accidental overwrite with set -o noclobber (then >| forces an overwrite).

Discarding output: /dev/null

command > /dev/null            # hide normal output
command 2> /dev/null           # hide errors
command > /dev/null 2>&1       # silence everything

Common in cron jobs, to avoid email for every run. But silencing errors hides problems. Prefer logging them:

*/5 * * * * /usr/local/bin/sync.sh >> /var/log/sync.log 2>&1

See crontab.

Input redirection

mysql mydb < dump.sql                 # file as stdin
wc -l < access.log                    # count lines (no filename in the output)

Here-documents feed multiple lines as stdin, which is great for writing config files from scripts:

cat > /etc/motd <<'EOF'
Welcome to web1. Unauthorized access is prohibited.
EOF

sudo tee /etc/sysctl.d/99-custom.conf > /dev/null <<'EOF'
net.core.somaxconn = 1024
EOF

Quoting 'EOF' prevents variable expansion inside. Here-strings: grep foo <<< "$text".

Pipes: connect commands

A pipe | sends one command’s stdout into the next command’s stdin:

ps aux | grep nginx
cat access.log | awk '{print $1}' | sort | uniq -c | sort -rn | head    # top IPs
journalctl -u nginx --since today | grep -i error | wc -l
du -sh /var/* 2>/dev/null | sort -h | tail

Only stdout goes through a pipe. Add 2>&1 (or use |& in Bash) to include stderr. See grep, awk, and sed.

tee: see output and save it

./deploy.sh | tee deploy.log                 # screen + file
./deploy.sh 2>&1 | tee -a deploy.log         # include errors, append
echo "1" | sudo tee /proc/sys/net/ipv4/ip_forward

The last example shows why sudo echo 1 > /root-owned-file fails: the redirection is done by your shell, not by sudo. sudo tee writes the file with root privileges.

Exit status in pipelines

A pipeline’s exit status is that of the last command, so failures earlier in the pipe are hidden. In scripts:

set -o pipefail      # the pipeline fails if any command fails

See Bash error handling.

Process substitution

Treat a command’s output as a file:

diff <(ssh web1 cat /etc/nginx/nginx.conf) <(ssh web2 cat /etc/nginx/nginx.conf)

See diff command.

Quick reference

Syntax Meaning
> / >> stdout to file (overwrite / append)
2> stderr to file
2>&1 / &> merge stderr with stdout
< file file as stdin
<<'EOF' here-document
a | b a’s stdout into b’s stdin
tee write to file and screen
/dev/null discard

Related: wildcards and globbing and Bash scripting tutorial.

  • Linux
  • Bash
  • Command Line