When one server can’t handle the load, or you can’t afford it going down, you spread traffic across several servers with a load balancer. There are several kinds, working at different layers with different trade-offs. Here’s a map of the options, and when each one fits.
Why load balance?
- Scale out: handle more requests than one server can.
- High availability: keep serving when a server fails or is being updated.
- Zero-downtime deploys: take servers out of rotation one at a time. See blue-green deployments.
- Geographic distribution: send users to nearby servers.
Layer 4 vs layer 7
| Layer 4 (transport) | Layer 7 (application) | |
|---|---|---|
| Sees | IPs, ports, TCP/UDP | Full HTTP: host, path, headers, cookies |
| Routing decisions | By connection | By request (/api → API servers, /static → cache) |
| TLS | Usually passed through | Usually terminated (can re-encrypt) |
| Protocols | Any TCP/UDP: databases, mail, games | HTTP(S), gRPC, WebSocket |
| Overhead | Very low | Higher (parses requests) |
| Features | Fast, simple | Header rewrites, caching, compression, WAF, retries |
| Examples | HAProxy TCP mode, Nginx stream, LVS/IPVS |
Nginx http, HAProxy HTTP mode, Envoy, Traefik, Caddy |
Use L7 for web apps, where content-based routing, TLS termination and observability are worth the cost. Use L4 for non-HTTP services, or when TLS must stay end to end. See HAProxy TCP mode.
DNS-based load balancing
Round-robin DNS: several A records for one name, and clients pick one:
www.example.com. A 203.0.113.10
www.example.com. A 203.0.113.11It’s simple and free, but with no health checks: a dead server stays in the rotation until you change DNS and caches expire. Client behaviour varies.
GeoDNS / latency-based DNS: managed DNS providers answer with different IPs depending on the client’s region or measured latency, often with health checks that remove failed endpoints. Good for multi-region setups. Failover speed is bounded by TTLs. See DNS propagation and anycast explained.
DNS balancing is often combined with a real load balancer in each region.
Software load balancers on a VPS
The most common choice for self-managed infrastructure:
- Nginx: a web server plus an L7 (and L4
stream) balancer. See Nginx load balancing. - HAProxy: a dedicated balancer with excellent performance, health checks and stats. See HAProxy load balancer.
- Traefik / Caddy: automatic HTTPS and service discovery, great with Docker. See Traefik.
- Envoy: modern L7 proxy for service meshes and gRPC.
- LVS/IPVS with keepalived: kernel-level L4 balancing for very high throughput.
A modest VPS running Nginx or HAProxy can balance thousands of requests per second for several backends.
Cloud and hardware load balancers
- Cloud LBs (AWS ALB/NLB, GCP, Azure): managed, scalable and integrated with autoscaling, billed per hour and per traffic.
- Hardware appliances (F5, Citrix ADC): enterprise data centres, very high throughput, expensive.
- CDN-based (Cloudflare Load Balancing): balancing at the edge, across origins in different places.
Balancing algorithms
| Algorithm | Behaviour | Good for |
|---|---|---|
| Round robin | Each server in turn | Similar servers, short requests |
| Weighted round robin | More traffic to bigger servers | Mixed server sizes |
| Least connections | Fewest active connections | Long or uneven requests, WebSockets |
| IP hash / source | The same client goes to the same server | Simple stickiness |
| Consistent hashing | Stable mapping with minimal reshuffling | Caches |
| Random with two choices | Pick two, use the less loaded | Large pools |
Health checks
A load balancer must stop sending traffic to unhealthy servers:
- Passive: mark a server down after failed requests (Nginx open source:
max_fails,fail_timeout). - Active: probe an endpoint regularly (HAProxy, Nginx Plus, Traefik). Ideally a
/healthendpoint that checks the app’s dependencies, such as database connectivity.
Sticky sessions (and why to avoid them)
If an app stores sessions in local memory or files, users must keep hitting the same server (“sticky sessions” via cookie or IP hash). That works, but it unbalances load and loses sessions when a server dies. Better: store sessions centrally (Redis, the database) or use stateless tokens, so any server can handle any request. See install Redis and JWT explained.
Don’t make the load balancer a single point of failure
One load balancer in front of three servers moves the single point of failure to the load balancer. Common fixes:
- Two load balancers with a floating IP (keepalived / VRRP). See keepalived floating IP.
- DNS with two load balancer IPs and health-checked failover.
- Managed or edge load balancing.
Choosing
| Situation | Suggestion |
|---|---|
| A few web servers on VPSes | Nginx or HAProxy (L7) |
| Docker hosts with many services | Traefik or Caddy |
| Databases, Redis, mail, game servers | HAProxy or Nginx stream (L4) |
| Multiple regions | GeoDNS or anycast in front, regional balancers behind |
| No ops time, cloud-native | A managed cloud LB |
Related: vertical vs horizontal scaling and high availability.