Load Balancers

Model Answer

Round-robin: equal distribution, works well when requests have similar cost and servers have similar capacity. Simple and predictable. Least-connections: better when request durations vary significantly (e.g., WebSocket connections alongside short HTTP requests) — avoids overloading servers with long-lived connections. Random: surprisingly effective when all servers are identical — avoids coordination overhead and has good behavior under simultaneous burst load. Netflix uses weighted random. Power of Two Choices (P2C) — pick two random servers, route to the one with fewer connections — achieves near-optimal balance with O(1) overhead.

Model Answer

Options: (1) Consistent hashing on user ID or session ID — the same user always maps to the same backend without any state in the LB, (2) Signed cookie containing the backend server ID — the LB reads the cookie and routes accordingly, no server-side state, (3) JWT tokens that contain all session state — any backend can validate and use them, removing the need for affinity entirely. Option 3 is the most scalable: eliminate the statefulness rather than routing around it. IP hash is the least reliable (NAT, mobile IPs change).

Model Answer

Possible causes: (1) Health check is too shallow — it checks if the port is open, not whether the backend is actually healthy (the DB connection pool could be exhausted). Improve health check to exercise actual dependencies. (2) Slow backends — the LB routes traffic to a server that passes health checks but is running slowly (GC pause, I/O saturation). Use latency-aware routing or least-response-time. (3) Partial failures — some routes fail, not all. The health check hits a healthy endpoint. (4) Race condition — backend is being drained/restarted, passes one health check but fails during the drain window. Use pre-stop hooks and connection draining.

Model Answer

L4 (transport layer): routes based on IP/TCP/UDP. Cannot see HTTP content. Extremely fast, minimal latency overhead, handles any TCP protocol. Use for: raw throughput workloads, non-HTTP protocols, DDoS mitigation (AWS NLB). L7 (application layer): inspects HTTP headers, URL, cookies. Enables content-based routing, header rewriting, JWT validation, A/B testing. Higher computational overhead per request. Use for: HTTP APIs, microservices routing, SSL termination with certificate management, WebSocket upgrades. Modern stacks often use both: L4 at the edge for DDoS/high-throughput, L7 inside the cluster for intelligent routing.

Model Answer

Rolling deployment: bring up new version instances; add to LB pool; drain old instances (stop new connections, wait for in-flight to complete); remove old instances from pool; shut down. Key steps: (1) Liveness vs readiness probes — do not add an instance to the pool until it passes the readiness check, (2) Pre-stop hook — allow a grace period before termination to drain in-flight requests, (3) Connection draining — LB waits for existing connections to complete before marking the backend as removed, (4) Health check passes before traffic — new instance serves a few test requests before receiving full traffic (can combine with canary or blue-green). Kubernetes handles this via readinessProbe and terminationGracePeriodSeconds.