ARCHIVED from builddistributedsystem.com on 2026-04-28 — URL: https://builddistributedsystem.com/tracks/messenger/tasks/task-1-3-5-chaos-mode
TASK

Implementation

Real networks drop messages. Netflix pioneered chaos engineering — deliberately injecting failures to test resilience. Your task is to add a "chaos mode" to your node.

When chaos mode is enabled, the node randomly drops a configurable percentage of outgoing messages (does not send them to stdout). This simulates network packet loss.

Implement these message types:

  1. chaos_on — Enable chaos mode with a given drop rate:
Request:  {"type": "chaos_on", "msg_id": 1, "drop_rate": 0.1}
Response: {"type": "chaos_on_ok", "in_reply_to": 1, "drop_rate": 0.1}
  1. chaos_off — Disable chaos mode:
Request:  {"type": "chaos_off", "msg_id": 2}
Response: {"type": "chaos_off_ok", "in_reply_to": 2}
  1. chaos_stats — Report chaos statistics:
Request:  {"type": "chaos_stats", "msg_id": 3}
Response: {"type": "chaos_stats_ok", "in_reply_to": 3, "enabled": true, "drop_rate": 0.1, "total_sent": 50, "total_dropped": 5}

Use a fixed random seed (42) for reproducibility in tests. The drop decision uses random.random() < drop_rate.

Chaos mode should NOT drop control messages (init_ok, chaos_on_ok, chaos_off_ok, chaos_stats_ok) — only application messages like echo_ok.

Sample Test Cases

Init and echo work without chaosTimeout: 5000ms
Input
{"src":"c0","dest":"n1","body":{"type":"init","msg_id":1,"node_id":"n1","node_ids":["n1"]}}
{"src":"c1","dest":"n1","body":{"type":"echo","msg_id":2,"echo":"safe"}}
Expected Output
{"src": "n1", "dest": "c0", "body": {"type": "init_ok", "in_reply_to": 1, "msg_id": 0}}
{"src": "n1", "dest": "c1", "body": {"type": "echo_ok", "echo": "safe", "in_reply_to": 2, "msg_id": 1}}
Chaos on responds with drop_rateTimeout: 5000ms
Input
{"src":"c0","dest":"n1","body":{"type":"init","msg_id":1,"node_id":"n1","node_ids":["n1"]}}
{"src":"c1","dest":"n1","body":{"type":"chaos_on","msg_id":2,"drop_rate":0.5}}
Expected Output
{"src": "n1", "dest": "c0", "body": {"type": "init_ok", "in_reply_to": 1, "msg_id": 0}}
{"src": "n1", "dest": "c1", "body": {"type": "chaos_on_ok", "drop_rate": 0.5, "in_reply_to": 2, "msg_id": 1}}

Hints

Hint 1
Use a random number generator to decide whether to drop each outgoing message
Hint 2
The drop rate should be configurable (default 10%)
Hint 3
Log dropped messages to stderr so you can observe chaos effects
Hint 4
Track how many messages were dropped vs sent
Hint 5
Chaos mode should be toggleable via a message
OVERVIEW

Theoretical Hub

Concept overview coming soon

Key Concepts

chaos engineeringfault injectionresilience testingnetwork partitions
main.py
python
Add Chaos Mode with Random Message Dropping - The Messenger | Build Distributed Systems