When one application server isn’t enough — or you want the site to survive one server failing — Nginx can distribute requests across several backends. The open-source version includes everything most setups need.

A basic load balancer

upstream app_servers {
    server 10.0.0.11:3000;
    server 10.0.0.12:3000;
    server 10.0.0.13:3000;
}

server {
    listen 80;
    server_name app.example.com;

    location / {
        proxy_pass http://app_servers;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

By default Nginx uses round robin: requests go to each server in turn.

Use private IPs for backends where possible, and make sure backends accept connections only from the load balancer (firewall rules).

Balancing methods

upstream app_servers {
    least_conn;              # send to the server with fewest active connections
    server 10.0.0.11:3000;
    server 10.0.0.12:3000;
}
Method Behaviour Good for
(default) round robin In turn, respecting weights Similar, short requests
least_conn Fewest active connections Requests of varying duration
ip_hash Same client IP → same server Simple session stickiness
hash $key consistent Same key → same server; minimal remapping when servers change Caches, sharding by URL or user
random two least_conn Pick two at random, choose the less busy Large pools, multiple balancers

Weights

Send more traffic to bigger servers:

upstream app_servers {
    server 10.0.0.11:3000 weight=3;   # 8-core server
    server 10.0.0.12:3000 weight=1;   # 2-core server
}

Passive health checks

Nginx open source marks a backend as failed when requests to it fail:

upstream app_servers {
    server 10.0.0.11:3000 max_fails=3 fail_timeout=30s;
    server 10.0.0.12:3000 max_fails=3 fail_timeout=30s;
}

After 3 failures within 30 seconds, the server is skipped for 30 seconds, then tried again. What counts as a failure is set in the location:

location / {
    proxy_pass http://app_servers;
    proxy_next_upstream error timeout http_502 http_503 http_504;
    proxy_next_upstream_tries 2;
    proxy_connect_timeout 3s;
}

proxy_next_upstream retries the request on another server. Be careful with non-idempotent requests (POST): by default Nginx doesn’t retry them after the request was sent, which is the safe behaviour.

Active health checks (polling a /health URL) are an NGINX Plus feature. With open source, use passive checks, or an external tool that edits the upstream list, or HAProxy, which includes active checks — see HAProxy load balancer.

Backup and maintenance

upstream app_servers {
    server 10.0.0.11:3000;
    server 10.0.0.12:3000;
    server 10.0.0.20:3000 backup;    # used only when all others are down
    server 10.0.0.13:3000 down;      # temporarily removed
}

Mark a server down and reload Nginx to take it out of rotation for maintenance — existing connections finish normally.

Keepalive to backends

Opening a new TCP connection for every proxied request adds latency. Reuse connections:

upstream app_servers {
    server 10.0.0.11:3000;
    server 10.0.0.12:3000;
    keepalive 32;
}

location / {
    proxy_pass http://app_servers;
    proxy_http_version 1.1;
    proxy_set_header Connection "";
}

Both lines in the location are required for keepalive to work.

Sessions and state

Load balancing is easiest when backends are stateless: sessions in Redis or a database, uploaded files in shared/object storage, caches shared or disposable. If an app keeps sessions in memory, you need stickiness:

  • ip_hash — simple, but uneven when many users share an IP (offices, mobile carriers).
  • hash $cookie_sessionid consistent; — stick by session cookie.

Fixing the app to be stateless is better long term.

WebSockets through the balancer

location /ws/ {
    proxy_pass http://app_servers;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "upgrade";
    proxy_read_timeout 1h;
}

WebSocket connections stay on one backend for their lifetime; least_conn keeps them evenly spread.

TCP and UDP load balancing

The stream module balances non-HTTP traffic — databases, game servers, DNS:

stream {
    upstream db_read {
        server 10.0.0.31:5432;
        server 10.0.0.32:5432;
    }
    server {
        listen 5432;
        proxy_pass db_read;
    }
}

The load balancer itself is a single point of failure

One Nginx in front of three app servers still fails if that Nginx server goes down. Options:

Testing

Add a header that shows which backend answered:

add_header X-Upstream $upstream_addr always;
for i in $(seq 1 6); do curl -sI https://app.example.com | grep -i x-upstream; done

Remove it in production if you don’t want to reveal internal IPs.

Several small VPS in the same location, connected over private networking where available, make a cost-effective load-balanced cluster. Our KVM VPS plans let you start with two app servers and grow from there.

  • Nginx
  • Load Balancing
  • High Availability