A common secure layout exposes only one server (the bastion or jump host) to the internet over SSH. Database and app servers accept SSH only from the private network. To reach them, you hop through the bastion. OpenSSH’s ProxyJump makes this a single command, with authentication still done from your own computer.

Your laptop  ──SSH──▶  bastion (public IP)  ──SSH──▶  db1 (10.0.0.12, private)

One-off: ssh -J

ssh -J [email protected] [email protected]

Your SSH client connects to the bastion, opens a TCP tunnel from there to 10.0.0.12:22, and runs a second, end-to-end encrypted SSH session through it. Your keys stay on your laptop. The bastion only forwards encrypted bytes. That’s safer than logging into the bastion and SSHing onwards, and safer than agent forwarding. See SSH agent forwarding.

Multiple hops:

ssh -J user@hop1,user@hop2 user@target

Custom ports: ssh -J admin@bastion:2222 [email protected].

Permanent: ~/.ssh/config

Host bastion
    HostName bastion.example.com
    User admin
    Port 2222
    IdentityFile ~/.ssh/id_ed25519

Host db1
    HostName 10.0.0.12
    User admin
    ProxyJump bastion

Host app-*
    User deploy
    ProxyJump bastion

Host app-1
    HostName 10.0.0.21
Host app-2
    HostName 10.0.0.22

Now ssh db1 and ssh app-2 just work. See SSH config file.

Older OpenSSH (before 7.3)

Host db1
    ProxyCommand ssh -W %h:%p bastion

ProxyJump is the modern, simpler form of the same idea.

File transfer and tools through the jump host

Everything that uses SSH works through the config:

scp backup.sql db1:/tmp/
rsync -avz ./site/ app-1:/var/www/site/
sshfs db1:/var/log ~/mnt/db1-logs

Without a config entry:

scp -J admin@bastion file [email protected]:/tmp/
rsync -e "ssh -J admin@bastion" -avz ./dir [email protected]:/srv/

Port forwarding to a private database through the bastion, for a local GUI client:

ssh -L 5432:127.0.0.1:5432 db1
# then connect your DB client to localhost:5432

Ansible picks up ProxyJump from your SSH config, or set ansible_ssh_common_args='-J admin@bastion'. See SSH tunnels and Ansible inventory.

Setting up the bastion

The bastion should be small, simple and hardened:

  1. Minimal software: SSH only, no web apps, no databases.
  2. Key-only authentication, with root login disabled. See disable root SSH login.
  3. Restrict who can use it: AllowUsers admin or AllowGroups ssh-users.
  4. Allow forwarding (needed for ProxyJump), but nothing else:
# /etc/ssh/sshd_config on the bastion
AllowTcpForwarding yes
X11Forwarding no
AllowAgentForwarding no
PermitTTY yes

For “jump-only” users who should never get a shell on the bastion:

Match User jumper
    PermitTTY no
    ForceCommand /usr/sbin/nologin
    AllowTcpForwarding yes
    PermitOpen 10.0.0.*:22
  1. Firewall it: allow SSH to the bastion from your admin IPs if possible. Add Fail2Ban or CrowdSec.
  2. Log centrally, so access is auditable even if the bastion is compromised. See rsyslog.

Setting up the private servers

On db1, app-1 and the rest, allow SSH only from the bastion’s private IP:

sudo ufw allow from 10.0.0.5 to any port 22 proto tcp
sudo ufw deny 22/tcp

Or bind sshd only to the private interface (ListenAddress 10.0.0.12). Put your public key in ~/.ssh/authorized_keys on each private server. The bastion doesn’t need it.

Bastion vs VPN

Bastion + ProxyJump VPN (WireGuard, Tailscale)
Setup Just SSH VPN server and clients
Covers SSH and anything tunnelled over it All protocols (web admin UIs, databases…)
Per-command convenience Very good with ssh config Transparent once connected
Exposure One SSH port One UDP port (WireGuard) or none (Tailscale)

Many teams use both: a VPN for general access, and a bastion for audited SSH. See WireGuard and Tailscale.

Troubleshooting

ssh -v db1        # shows each hop: connecting to bastion, then "channel … open" to 10.0.0.12
  • channel 0: open failed: administratively prohibited — the bastion has AllowTcpForwarding no, or PermitOpen doesn’t include the target.
  • Connection timed out during the second hop — the bastion can’t reach the target. Check the private network and the target’s firewall from the bastion with nc -zv 10.0.0.12 22. See netcat.
  • Permission denied on the target — your key isn’t in the target’s authorized_keys, or User is wrong in the config.

Quick reference

Task Command / config
One-off jump ssh -J user@bastion user@target
Multiple hops ssh -J h1,h2 target
Permanent ProxyJump bastion in ~/.ssh/config
scp through jump scp -J bastion file target:/path
rsync through jump rsync -e "ssh -J bastion" …
Bastion needs AllowTcpForwarding yes

Related: SSH multiplexing and Linux server hardening.

  • SSH
  • Security
  • Networking