Reading · 5 min · Module 8, lesson 1 of 420 min left in this module
Module 8 · Load balancers and private networksLesson 1 of 4
Goal: Explain how a load balancer spreads requests across servers and keeps traffic away from unhealthy ones.
Load balancers and private networks at a glance
Load balancer
One public entry that spreads requests.
Health check
Keeps traffic off failing servers.
Private network
Addresses the internet can't reach.
Internal name
How services find each other inside.
Allowlist
Only listed addresses may connect.
What sits between the internet and your servers.
Key idea
A load balancer is the one address the internet talks to. It passes each request to one of several servers behind it, and only to servers that pass their health check, so you can add servers, lose one, or replace them all without changing your address.
One server has limits: it runs out of CPU at some point, and when it restarts, your site is down. Running several identical copies fixes both, but a DNS name should lead to one stable entry. The load balancer is that entry.
One public entry, a private network behind it
Clients
HTTPS
Load balancer
203.0.113.10:443The only public address
Private network10.0.0.0/16
web-110.0.1.11
web-210.0.1.12
web-310.0.1.13
Unhealthy: no traffic
Database10.0.2.20:5432
Internal service10.0.2.30:8080
No route in from the internet
Only the load balancer faces the internet. It sends traffic only to servers that pass their health check, and everything behind it talks over private addresses.
How it picks a server
Two common rules:
Round robin: each request goes to the next server in turn.
Least connections: each request goes to the server with the fewest open requests, which helps when some requests are slow.
Either way, any server may get any request. That's why the copies must be identical and keep nothing important in their own memory or disk (lesson 1.3.1). A login session stored in one server's memory disappears when the next request lands elsewhere; keep it in a database or a shared store instead.
Health checks
Every few seconds, the load balancer asks each server something simple, such as GET /healthz. A server that answers 200 stays in rotation. One that times out or returns errors several times in a row is taken out, and put back once it passes again.
A check that returns 200 before the app can reach its database lets broken servers take traffic. Make the check test what the app needs to serve.
What the server sees
The load balancer often ends the TLS connection itself, then forwards the request to a server over the private network. So the server sees the load balancer as the client. The visitor's real IP address arrives in a header, usually X-Forwarded-For.