Advanced SSH Key Management on a VPS: Multiple Keys, Agents, Jump Hosts, and Hardening

SSH keys are the foundation of secure VPS access — but most administrators only scratch the surface of what key-based authentication can do. Managing multiple keys for different users and purposes, forwarding agent credentials through jump hosts, restricting keys to specific commands, and auditing who accessed the server and when are all standard practice in well-secured infrastructure. This guide covers the full SSH key management workflow.

Generating Strong SSH Keys

# Ed25519: best option — small keys, fast, secure (use this)
ssh-keygen -t ed25519 -C "deploy@myproject" -f ~/.ssh/myproject_ed25519

# RSA 4096: compatible with older systems that don't support Ed25519
ssh-keygen -t rsa -b 4096 -C "deploy@myproject" -f ~/.ssh/myproject_rsa

# Always set a passphrase — protects the private key if the file is stolen
# The passphrase is handled by ssh-agent so you only type it once per session

# View public key (this is what you copy to servers)
cat ~/.ssh/myproject_ed25519.pub

Deploying Keys to a VPS

# Method 1: ssh-copy-id (simplest)
ssh-copy-id -i ~/.ssh/myproject_ed25519.pub deploy@YOUR_VPS_IP

# Method 2: Manual (when ssh-copy-id isn't available)
cat ~/.ssh/myproject_ed25519.pub | ssh deploy@YOUR_VPS_IP \
  "mkdir -p ~/.ssh && chmod 700 ~/.ssh && \
   cat >> ~/.ssh/authorized_keys && \
   chmod 600 ~/.ssh/authorized_keys"

# Method 3: Copy file directly
scp ~/.ssh/myproject_ed25519.pub deploy@YOUR_VPS_IP:/tmp/
ssh deploy@YOUR_VPS_IP "cat /tmp/myproject_ed25519.pub >> ~/.ssh/authorized_keys && rm /tmp/myproject_ed25519.pub"

Managing Multiple Keys with ~/.ssh/config

The SSH config file lets you define aliases, key assignments, and connection options per host — so you never need to remember which key goes with which server:

nano ~/.ssh/config

# Production web server
Host prod-web
    HostName 203.0.113.10
    User deploy
    IdentityFile ~/.ssh/prod_ed25519
    Port 22

# Production database server (only via jump host)
Host prod-db
    HostName 10.0.0.5
    User postgres
    IdentityFile ~/.ssh/prod_ed25519
    ProxyJump prod-bastion

# Bastion / jump host
Host prod-bastion
    HostName 203.0.113.1
    User bastion
    IdentityFile ~/.ssh/bastion_ed25519
    Port 2222

# Development server
Host dev
    HostName dev.yourdomain.com
    User ubuntu
    IdentityFile ~/.ssh/dev_ed25519

# GitHub (separate key per service is good practice)
Host github.com
    HostName github.com
    User git
    IdentityFile ~/.ssh/github_ed25519

# Global defaults
Host *
    ServerAliveInterval 60
    ServerAliveCountMax 3
    AddKeysToAgent yes
    IdentitiesOnly yes   # Only use explicitly configured keys

Now connect using aliases:

ssh prod-web          # Uses correct key automatically
ssh prod-db           # Tunnels through bastion transparently
git push origin main  # Uses GitHub key

SSH Agent: Enter Passphrase Once Per Session

# Start ssh-agent (usually auto-started by your OS/desktop)
eval "$(ssh-agent -s)"

# Add keys to agent (prompts for passphrase once)
ssh-add ~/.ssh/prod_ed25519
ssh-add ~/.ssh/dev_ed25519

# List loaded keys
ssh-add -l

# After adding, SSH connections use the agent — no passphrase needed
ssh prod-web

Agent Forwarding for Jump Hosts

Agent forwarding lets you use your local SSH keys from a remote server (e.g., to pull from GitHub while SSH’d into a VPS) — without copying private keys to the server:

# Enable agent forwarding in ~/.ssh/config for specific hosts only:
Host prod-bastion
    ForwardAgent yes   # Only enable for trusted hosts you fully control

# Or use -A flag for one-off connections:
ssh -A deploy@bastion.yourdomain.com

# From bastion, SSH to internal servers using your local key:
ssh postgres@10.0.0.5    # Your local key forwarded through bastion

Security note: Only forward the agent to hosts you fully trust and control. A compromised server with agent forwarding can use your forwarded keys to access other servers.

Restricting Keys on authorized_keys

The authorized_keys file supports per-key restrictions — limit what each key can do:

nano ~/.ssh/authorized_keys

# Restrict key to a single command (useful for backup scripts, monitoring)
command="/usr/local/bin/backup.sh",no-port-forwarding,no-X11-forwarding,no-agent-forwarding ssh-ed25519 AAAA... backup@monitoring

# Restrict to specific IP address
from="203.0.113.0/24",no-port-forwarding ssh-ed25519 AAAA... office@work

# Allow port forwarding but restrict to one command
command="rsync --server -avz . /backup/",no-pty ssh-ed25519 AAAA... rsync-backup

# Full access key (no restrictions — for human admin use)
ssh-ed25519 AAAA... admin@laptop

Hardening sshd_config

sudo nano /etc/ssh/sshd_config

# Disable password authentication entirely
PasswordAuthentication no
ChallengeResponseAuthentication no
UsePAM no

# Disable root login
PermitRootLogin no

# Limit SSH to specific users
AllowUsers deploy alice bob

# Change default port (obscures from automated scanners)
Port 2222

# Disable unused features
X11Forwarding no
AllowAgentForwarding no   # Only enable if you need it
AllowTcpForwarding no

# Use only modern algorithms
KexAlgorithms curve25519-sha256,diffie-hellman-group16-sha512
Ciphers chacha20-poly1305@openssh.com,aes256-gcm@openssh.com
MACs hmac-sha2-512-etm@openssh.com,hmac-sha2-256-etm@openssh.com

# Log verbosely for audit trail
LogLevel VERBOSE

# Disconnect idle sessions
ClientAliveInterval 300
ClientAliveCountMax 2
MaxStartups 3:50:10

sudo sshd -t   # Test config before applying
sudo systemctl reload ssh

Auditing SSH Access

# View all successful SSH logins
sudo grep "Accepted publickey" /var/log/auth.log | tail -30

# View failed attempts (brute force monitoring)
sudo grep "Failed password\|Failed publickey" /var/log/auth.log | \
  awk '{print $11}' | sort | uniq -c | sort -rn | head -20

# Who is currently logged in
who
w

# Recent login history
last | head -30

# Login history for specific user
last deploy | head -20

Rotate Keys Safely

# Step 1: Generate new key pair
ssh-keygen -t ed25519 -C "deploy-new-$(date +%Y%m)" -f ~/.ssh/deploy_new

# Step 2: Add new public key to server (while old key still works)
ssh-copy-id -i ~/.ssh/deploy_new.pub deploy@YOUR_VPS_IP

# Step 3: Test new key works
ssh -i ~/.ssh/deploy_new deploy@YOUR_VPS_IP "echo 'new key works'"

# Step 4: Remove old key from authorized_keys
ssh deploy@YOUR_VPS_IP "sed -i '/old_key_comment/d' ~/.ssh/authorized_keys"

# Step 5: Update ~/.ssh/config to use new key
# Step 6: Securely delete old private key
shred -vzu ~/.ssh/old_key

Getting Started

SSH key management is the security foundation for every Ubuntu VPS at VPS.DO. The key steps are: generate Ed25519 keys with passphrases, use ~/.ssh/config for multi-server management, disable password authentication in sshd_config, and restrict individual keys in authorized_keys to the minimum necessary permissions. VPS.DO’s emergency KVM console provides recovery access if you accidentally lock yourself out during hardening.

Conclusion

Advanced SSH key management — per-service keys, config file aliases, agent forwarding through jump hosts, and per-key command restrictions — transforms SSH from a basic login mechanism into a fine-grained access control system. Combined with sshd hardening and audit logging, key-based SSH provides both convenience (no password typing with agent) and security (no passwords to brute-force, per-key restrictions, complete audit trail).