Proxies

Model Answer

A reverse proxy sits in front of backend servers, receiving client requests and forwarding them on behalf of the servers. It can do TLS termination, compression, caching, authentication, request transformation. A load balancer is a specific type of reverse proxy that distributes requests across multiple backends. The distinction is blurry — Nginx and HAProxy do both. "Load balancer" emphasizes traffic distribution; "reverse proxy" emphasizes the request manipulation and forwarding role.

Model Answer

A service mesh provides network-level features (retries, timeouts, circuit breaking, mTLS, observability) via sidecar proxies, without any changes to application code. Reasons to use over application-level implementation: (1) language-agnostic — works for every language/framework in your infrastructure, (2) consistent policy enforcement — the same retry/timeout configuration across all services, (3) observability without instrumentation — L7 metrics for every service-to-service call automatically, (4) security — mTLS between all services without application changes. Cost: operational complexity, added latency per hop (~1ms for Envoy), large memory footprint at high pod counts.

Model Answer

Envoy's circuit breaker is resource-based, not error-rate-based (unlike Hystrix). It tracks: max_connections (concurrent TCP connections to upstream), max_pending_requests (queued requests waiting for a connection), max_requests (concurrent active requests), max_retries (concurrent active retries). When any limit is exceeded, new requests fail immediately with 503. For error-rate-based outlier detection, use Envoy's outlier_detection config: consecutive 5xx errors, success rate thresholds, and latency percentile thresholds trigger ejection of hosts from the load-balancing pool.

Model Answer

Without resilience: the timeout propagates up. The backend waits for the downstream timeout (e.g., 30s), returns a 500, gateway returns 500, user waits 30s and sees an error. With resilience: (1) The downstream has a short timeout (e.g., 2s), not 30s. (2) The backend uses a circuit breaker — if the downstream has been failing, fail fast without waiting. (3) The backend returns a degraded response (partial data) rather than a 500 if the failed downstream is non-critical. (4) The gateway adds a hedge or retry. This is the resilience pattern Netflix called "graceful degradation" — the system stays available with reduced functionality rather than fully failing.

Model Answer

Request mirroring: send a copy of live production requests to a shadow service, discard responses, without affecting the user. Use cases: (1) testing a new service version with real traffic before cutover — any differences in behavior are logged but do not affect users, (2) load testing a new implementation under real traffic patterns, (3) comparing old and new algorithm outputs. Envoy supports request mirroring natively via the request_mirror_policy config. Netflix's Diffy tool performs differential testing this way. The shadow service should not have side effects (no emails, no payment processing) — typically use a read-only mode or mock dependencies.