What is a Load Balancer?
In my “What is a website?” post, the request example I used was one client communicating with one server. In practice, popular sites run multiple copies of the same server behind the scenes, and something needs to decide which copy handles each incoming request. We call that a load balancer.
Why do they exist?
A single server can only handle so many concurrent requests before it runs out of CPU, memory, or network capacity. Running multiple identical servers (“horizontal scaling”) solves that, but it introduces a new problem since clients only know about one address (a domain name or IP). Thus, something must redistribute incoming traffic across them.
A load balancer solves some problems:
- Distributes traffic so single servers don’t get overwhelmed
- Improves availability by redirecting traffic being sent to servers that are down or unhealthy
- Enables zero-downtime deployments since traffic can be shifted away from a server before it’s offline
Layer 4 vs Layer 7 Load Balancing
Load balancers are usually described by which layer of the network stack they operate at.
Layer 4 (Transport layer) load balancers make routing decisions based on IP address and TCP/UDP port alone. They don’t look at the actual content of the request, just where it’s coming from and going to.
This makes them fast, but limited since they can’t route based on information inside the request itself, like a URL path or a header.
Layer 7 (Application layer) load balancers understand HTTP. They can inspect the actual request, things like the URL path, hostname, headers, or cookies, and make smarter routing decisions based on that content. This is what allows a single load balancer to route /api/* to one set of servers and /images/* to another, or route based on a Host header to support multiple domains behind the same load balancer.
The tradeoff is roughly speed and simplicity (L4) versus flexibility and content-awareness (L7). Most modern web infrastructure relies heavily on L7 load balancing since HTTP-level routing is usually what’s needed.
Health Checks
A load balancer is only useful if it knows which servers are actually able to handle traffic. It does this through health checks: periodic requests (often a simple GET /health endpoint) sent to each backend server to confirm it’s responsive.
If a server fails its health check, the load balancer stops sending it traffic until it starts passing again. This is what allows a fleet of servers to tolerate individual failures without any single server going down taking the whole service with it.
Where this shows up
Load balancers appear in different forms depending on the application environment:
- Cloud provider load balancers, like AWS’s Application Load Balancer (L7) and Network Load Balancer (L4)
- Software load balancers, like nginx or HAProxy, often run manually or as part of a larger platform
- Kubernetes Services and Ingress controllers, which act as load balancers for traffic within and into a cluster
A Kubernetes Service is essentially a load balancer with a virtual IP and DNS name attached to it, and an Ingress controller is essentially an L7 load balancer for routing traffic into the cluster.
Actual implementations
This depends on the individual websites’ architectural design. Some websites deploy a Layer 4 load balancer that redistributes traffic to a Layer 7 load balancer. While others do it the other way around. There are many different ways to implement these network tools and custom algorithms for them too.
In EKS, traffic typically flows from the client -> ALB -> internal NodePort/kube-proxy (L4) -> Pod. That internal load balancer (kube-proxy) uses iptables mode which, by default, uses random selection. IPVS mode can also be used to enable round robin.