ARCHIVED from builddistributedsystem.com on 2026-04-28 — URL: https://builddistributedsystem.com/tracks/migrator/tasks/task-25-2-3-graceful-degradation
TASK

Implementation

Graceful degradation keeps your API functional even when downstream services are failing. Instead of returning errors, you serve cached data, disable non-critical features, or queue requests for later processing. The user gets a slightly degraded experience instead of a complete outage.

Implement a node that degrades gracefully under failure conditions:

// Service failing -> open circuit breaker, return cached data
{ "type": "get_user", "msg_id": 1,
  "user_id": 123, "force_failures": 6 }
-> { "type": "user_response", "in_reply_to": 1,
    "user": {"id": 123, "name": "John Doe"},
    "circuit_state": "open", "from_cache": true }

// Dependency down -> return stale cached product
{ "type": "get_product", "msg_id": 2,
  "product_id": 123, "service_unavailable": true }
-> { "type": "product_response", "in_reply_to": 2,
    "product": {"id": 123, "name": "Product"},
    "_cached": true }

// High load -> disable expensive features
{ "type": "set_mode", "msg_id": 3,
  "mode": "degraded", "cpu_usage": 85 }
-> { "type": "mode_changed", "in_reply_to": 3,
    "mode": "degraded",
    "disabled_features": ["recommendations", "search"] }

Sample Test Cases

Circuit breaker opens after failuresTimeout: 10000ms
Input
{
  "src": "client",
  "dest": "api",
  "body": {
    "type": "get_user",
    "msg_id": 1,
    "user_id": 123,
    "force_failures": 6
  }
}
Expected Output
{"type": "user_response", "in_reply_to": 1, "user": {"id": 123, "name": "John Doe"}, "circuit_state": "open", "from_cache": true}
Fallback to cached dataTimeout: 5000ms
Input
{
  "src": "client",
  "dest": "api",
  "body": {
    "type": "get_product",
    "msg_id": 1,
    "product_id": 123,
    "service_unavailable": true
  }
}
Expected Output
{"type": "product_response", "in_reply_to": 1, "product": {"id": 123, "name": "Product"}, "_cached": true}

Hints

Hint 1
Circuit breaker: after threshold failures, return cached data instead of calling the failing service
Hint 2
Cached fallback: return stale data with _cached: true when a dependency is down
Hint 3
Feature flags: disable expensive non-critical features (recommendations, search) under high load
Hint 4
Request queuing: accept the request and queue it for later processing when the service is down
Hint 5
circuit_state: "open" means the circuit breaker is tripped and using the fallback
OVERVIEW

Theoretical Hub

Concept overview coming soon

Key Concepts

graceful degradationcircuit breakerfallback cachefeature flagsrequest queuing
main.py
python
Implement Graceful API Degradation - The Migrator | Build Distributed Systems