Load Balancers, Reverse Proxies, and the L7 Stack

Last updated: May 1, 2026

Every production web service in 2026 has at least one load balancer in front of it. The load balancer is also: the TLS terminator, the WAF, the rate limiter, the bot mitigation, the health checker, and the source IP from the application’s perspective. Misconfigure it and you have a single point of failure, a privacy hole, or a bypass for everything that came before. This module is the working introduction to load balancers, reverse proxies, and the L7 stack with security front-of-mind.

L4 vs L7 — the fundamental choice

Layer 4 load balancing routes by IP and TCP/UDP port; the LB does not inspect application bytes. Fast (line-rate possible), simple, and protocol-agnostic. The original LVS, AWS NLB, GCP TCP/UDP LB. Layer 7 load balancing understands HTTP (or other application protocols): can route by URL path, host header, cookies; can rewrite headers; can terminate TLS; can apply rate limits per user; can run a WAF. nginx, HAProxy, Envoy, AWS ALB, Azure Application Gateway, Cloudflare.

The choiceL7 for HTTP/HTTPS workloads (almost everything in 2026); L4 for non-HTTP (databases, gRPC sometimes, custom TCP protocols), for ultra-high-throughput (>10M PPS), or where end-to-end TLS without termination is required. Modern stacks often combine: L4 LB at the edge for DDoS, L7 LB inside the perimeter for routing.

Algorithms — round-robin, least-connections, consistent-hash

Round-robineach backend gets the next connection. Simple, fair on uniform backends.

Weighted round-robinbackends get traffic proportional to weight (used during canary deploys, mixed-capacity fleets).

Least-connectionssend to backend with fewest active connections; good for long-lived connections (websockets, gRPC streams). Least-time /

least-response-timebackend with lowest latency wins; beware oscillation if probe interval is wrong.

Consistent hashhash a client identifier (IP, cookie, header) and map to a backend; same client always lands on same backend (sticky without stateful tracking). Used heavily in caching tiers (Memcached, Redis fronted by twemproxy).

Randomsurprisingly effective; used in some Envoy configs to avoid synchronisation pathologies.

The mistakechoosing round-robin without considering backend heterogeneity, or sticky-session without an off-host session store (now you cannot bounce a backend without dropping users).

Health checks — the difference between graceful failover and outage

Load balancers detect backend failure by health checks — periodic probes that the backend “is healthy”. Active: LB initiates HTTP/TCP probe every N seconds. Passive: LB observes real traffic, marks unhealthy on N consecutive failures. Best practice: combine both. Mistakes:

TLS termination — where to terminate, and the trust boundary

Three patterns: Pass-through (TLS terminates at backend; LB only forwards by SNI). Used when end-to-end encryption is mandatory; LB cannot inspect or apply L7 features. Termination at LB (TLS ends at LB; LB-to-backend is plaintext or re-encrypted with internal cert). Standard pattern. Re-encryption (terminate, inspect, re-encrypt to backend). Required for anything sensitive in regulated environments.

Trust boundarieswhen LB terminates TLS, the LB sees plaintext credentials, payment data, PII. The LB is now part of your DPDP-compliant data perimeter; logs and access controls must reflect that.

Header injectionterminating LBs typically inject X-Forwarded-For, X-Forwarded-Proto, X-Real-IP so backends know the client. The backend MUST validate the source of these headers (only trust LB IPs) — otherwise a client can inject headers and bypass IP-based controls.

Sticky sessions — when, why, and the pitfalls

Session affinity / sticky sessions: the LB ensures a client always reaches the same backend.

Why you might need itin-memory session state, websockets, long-running operations with state on the backend.

How implementedcookie inserted by LB (most common, controllable), source IP hash (works behind NAT but breaks for CGN), or backend-set cookie that LB observes.

Why you should avoid it when possiblebouncing a backend drops all stuck users, scaling out is harder, hot-spots emerge if affinity meets uneven traffic.

The right architecturestateless backends + external session store (Redis, database) + cacheable responses. Sticky sessions are a 2010s pattern that 2026 architectures move away from. Where you must use them, use cookie-based with a sensible expiry (1-2 hours) and ensure the backend also handles the “no affinity” path gracefully.

L7 routing — host, path, header rules

Modern L7 LBs support rich routing: by host header (multi-tenant routing), by path prefix (/api/* → api backend; /static/* → CDN), by header (X-Customer-Tier: premium → premium pool), by cookie, by query string.

Use casesAPI versioning (route v2 to new backend), canary deployments (5% → v2, 95% → v1), feature flags, A/B testing, blue/green deploys.

Security implicationsrouting rules become part of the security policy. A rule that says “if path starts with /admin, route to admin backend” is also a security perimeter — anything that bypasses path normalisation can route to admin. Path-traversal in routing rules (/admin/../ after URL decode) has caused real bypasses. Always normalise paths before routing decisions; explicit prefix matches over regex.

Operational pitfalls — what kills load balancers in production

gRPC and HTTP/2 — what changes for load balancing

gRPC runs over HTTP/2. HTTP/2 multiplexes many streams onto one TCP connection. Naive L4 load balancing routes the connection — and all its streams — to one backend, which defeats load balancing. The right answer: L7 load balancing aware of HTTP/2 semantics, which can route individual streams. Envoy, Contour, NGINX (with appropriate config), AWS ALB all support gRPC properly.

Connection management caveatslong-lived HTTP/2 connections + sticky load balancing means uneven backend distribution as connections accumulate over time. Mitigation: max-connection-age forces reconnect periodically (~5-15 minutes typical). Without this, scaling up new backends sees little traffic until a thundering herd arrives.

Authentication and gRPCgRPC metadata carries auth tokens like HTTP headers — same logic applies, JWT validation, mTLS, etc. The pattern is consistent; the framing is different.

Service mesh — when LBs become east-west fabric

A service mesh (Istio, Linkerd, Consul Connect) deploys a proxy (typically Envoy) as a sidecar to every workload. Each service-to-service call goes through the local sidecar, which handles TLS, retries, circuit breaking, observability, and policy.

What this buysmTLS everywhere (every service-to-service call encrypted with workload-identity certs), uniform observability (every call logged with trace ID), centralised policy (“service A may call service B endpoint X but not endpoint Y”), traffic shaping (canary, weighted routing, fault injection).

What it costsoperational complexity, control-plane availability, latency overhead (typically 1-5 ms per hop), resource overhead per pod.

Choosingfor clusters with 50+ services and meaningful inter-service security/observability requirements, service mesh is justified. Below that, simpler patterns (cert-manager + ingress + NetworkPolicy) cover most needs without the operational burden.

Caching at the LB — the performance and security tradeoffs

L7 LBs (especially CDN-backed) cache responses to reduce backend load and improve latency. What can be cached safely: static assets (images, JS, CSS — long TTL), public API responses without personalisation, idempotent GET responses with proper cache headers. What must NOT be cached: anything containing user-identifying data, authenticated content (without cache-key including auth context), responses to mutations. Common security holes:

Diagrams

  L4 LB (sees only TCP/IP):
     Client ──TCP──▶ LB ──TCP──▶ Backend
     LB knows: src IP, dst IP, ports, TCP flags
     LB does not know: HTTP path, headers, cookies

  L7 LB (terminates HTTP):
     Client ──HTTPS──▶ LB ──HTTP──▶ Backend
     LB knows: everything in the request — path, host, headers, body
     LB can: rewrite, route by path, run WAF, rate limit per user
     LB sees: plaintext (TLS terminated here)

  Pass-through (no inspection):
     Client ─[TLS]─▶ LB ─[TLS]─▶ Backend (cert lives here)

  Termination + plaintext to backend:
     Client ─[TLS]─▶ LB ─[HTTP]─▶ Backend
     ^                ^
     LB has cert      Backend trusts LB to identify client
     plaintext on the LB-to-backend hop (must be a trusted segment)

  Re-encryption (typical regulated):
     Client ─[TLS public]─▶ LB ─[TLS internal]─▶ Backend
     LB inspects in plaintext; encrypts again with internal CA cert

References & deeper reading

• nginx documentation
• HAProxy documentation
• Envoy proxy
• AWS ALB documentation
• OWASP CRS — ModSecurity rules
• Caddy server
• Traefik

FAQ

nginx, HAProxy, Envoy — which should I pick?

For traditional web stacks, nginx is the workhorse — battle-tested, fast, ubiquitous. For complex routing and high feature ceiling, Envoy is the modern choice (default in service meshes). For ultra-high-throughput TCP/HTTP load balancing where stability matters more than features, HAProxy. They overlap heavily; choose for ecosystem fit (cert-manager + nginx-ingress, Istio + Envoy) rather than pure technical merit.

When does a WAF help, and when does it just create false positives?

A WAF helps against opportunistic attacks (mass exploitation of CVEs, automated SQLi probes) and as a quick patch for newly-disclosed vulnerabilities. It rarely stops a determined targeted attacker. Run in monitor-only for 30-60 days, tune false-positives, then enforce; treat WAF as defence-in-depth, not the primary control.

Should I terminate TLS at LB or pass through to the backend?

Terminate at LB unless you need backend-controlled cert (some compliance regimes, or backends that need to read TLS-extracted client identity directly). Termination unlocks every L7 capability — without it the LB is a fancy L4. The trust boundary moves to the LB-to-backend hop, which should be on a trusted segment or re-encrypted.

How do I avoid LB single-point-of-failure?

Active-active LB pair with anycast or DNS-based failover; or cloud LB (which is internally HA). For self-hosted: keepalived + VRRP for VIP failover; HAProxy in pairs; or DNS-based load balancing across multiple LBs. Test failover quarterly — the time to discover the failover is broken is not during an incident.

Are LBs the right place to do authentication?

For TLS client cert validation, yes — terminate TLS at LB and pass authenticated identity to backends via header (with backends validating the LB chain). For OAuth/SSO, increasingly yes — LB-fronted auth proxies (oauth2-proxy, AWS ALB native OIDC, Cloudflare Access) offload auth from apps. For per-request authz with rich policy, no — that belongs at app/service level.

Should I use a service mesh from day one?

No. Start with Kubernetes Services + cert-manager + Ingress + NetworkPolicy. Add service mesh when you have specific needs that those primitives do not solve — usually around east-west mTLS, traffic shaping, or fine-grained authz. Premature mesh adoption adds operational burden without proportional benefit.

What is the difference between an Ingress and an API Gateway?

Ingress (Kubernetes term): L7 routing into the cluster — host/path to service. API Gateway: more featureful — auth, rate limiting, transformations, schema validation, monetisation. Modern projects (Kong, Ambassador, Traefik) often combine both. For internal cluster-only routing, Ingress; for external API surface, API Gateway.

⚖️ Legal: Use any techniques described here only on networks you own or have explicit written authorisation to test. In India, unauthorised access is punishable under IT Act §66 (up to 3 years + fine). Pair offensive testing with a signed Statement of Work / Rules of Engagement; pair forensic activity with §65B-aligned chain of custody.