What is Load Balancing?

Load balancing spreads incoming requests across two or more servers so no single machine carries all the traffic, and removes a failed server from rotation automatically so its failure becomes a routing change rather than an outage. Think of the attendant at a toll plaza waving each vehicle to the shortest lane.

• Distribution: an algorithm such as round robin or least connections picks the server for each request.
• Health checks: the balancer probes each backend and stops sending traffic to one that fails.
• Failover: the remaining servers absorb the failed server's share.
• One address: visitors reach a single IP or hostname; the pool behind it can grow or shrink.

Most sites meet load balancing at a threshold: a sale-day spike, a server that cannot keep up, or a requirement that the site survive one machine failing.

A health check is a periodic probe — an HTTP request to a status URL, a TCP connect, or watching live traffic for errors — and a server that fails it stops receiving new requests until it passes again.

Active checks test each server on a schedule before traffic arrives. Passive checks mark a server down when real connections start failing. Either way, visitors see no error page; the pool simply shrinks by one and rejoins the server once it recovers. Make the status endpoint check the things that matter, such as the database connection, or a server can look "healthy" while serving errors.

Open-source Nginx balances HTTP traffic with an upstream block listing the backend servers and a proxy_pass that sends requests to it. This example uses least connections, marks a server down after three failures in 30 seconds, and keeps a backup server in reserve.

upstream app_pool {
    least_conn;
    server 10.0.0.11:80 max_fails=3 fail_timeout=30s;
    server 10.0.0.12:80 max_fails=3 fail_timeout=30s;
    server 10.0.0.13:80 backup;
}

server {
    listen 80;
    server_name example.in;

    location / {
        proxy_pass http://app_pool;
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

Open-source Nginx performs passive health checks through max_fails; scheduled active checks need NGINX Plus or a tool such as HAProxy. Test with sudo nginx -t before sudo systemctl reload nginx. The same proxy pattern in front of Apache is shown in Nginx reverse proxy for Apache.

A CDN serves cached static files from edges near users, a cache stores computed pages so they are not rebuilt, and a load balancer decides which origin server handles whatever traffic is left. Large sites run all three together.

The edge layer is explained in what is a CDN.