Implement Idempotency in Sagas - The Coordinator

                      TASK
                    

Implementation

Idempotency ensures that retrying saga steps doesn't cause duplicate operations like double-charging payments.

Idempotency key:

Each saga step is tagged with:

saga_id: "saga42"
step_id: 2
idempotency_key: "saga42:step2"

Service-side idempotency tracking:

processed_steps = new Map<string, PaymentResult>();

chargePayment(saga_id: string, step: number, params: ChargeParams) {
  const key = `${saga_id}:step${step}`;

  // Check if already processed
  if (this.processed_steps.has(key)) {
    console.log(`Already processed ${key}, returning cached result`);
    return this.processed_steps.get(key);
  }

  // Process and cache result
  const result = this.doCharge(params);
  this.processed_steps.set(key, result);
  return result;
}

Example: Retry without double-charge:

// First attempt:
{"type": "ChargePayment", "saga_id": "saga42", "step": 2, "params": {"user_id": "u42", "amount": 99.99}}
Response: {"type": "ChargePayment_ok", "saga_id": "saga42", "step": 2, "result": {"payment_id": "p1", "charged": 99.99}}

// Network timeout, orchestrator retries:
{"type": "ChargePayment", "saga_id": "saga42", "step": 2, "params": {"user_id": "u42", "amount": 99.99}}
Response: {"type": "ChargePayment_ok", "saga_id": "saga42", "step": 2, "result": {"payment_id": "p1", "charged": 99.99}, "note": "cached_result"}

// User is only charged once (payment_id = "p1")

Compensating transactions must also be idempotent:

// First compensation:
{"type": "RefundPayment", "saga_id": "saga42", "step": 2, "compensating": true, "params": {"payment_id": "p1", "amount": 99.99}}
Response: {"type": "RefundPayment_ok", "saga_id": "saga42", "step": 2, "result": {"refund_id": "r1", "refunded": 99.99}}

// Retry:
{"type": "RefundPayment", "saga_id": "saga42", "step": 2, "compensating": true, "params": {"payment_id": "p1", "amount": 99.99}}
Response: {"type": "RefundPayment_ok", "saga_id": "saga42", "step": 2, "result": {"refund_id": "r1", "refunded": 99.99}, "note": "already_refunded"}

Sample Test Cases

Idempotent charge on retryTimeout: 5000ms

Input

{"src":"c0","dest":"payment","body":{"type":"init","msg_id":1}}
{"src":"c1","dest":"payment","body":{"type":"ChargePayment","msg_id":2,"saga_id":"saga42","step":2,"params":{"user_id":"u42","amount":99.99}}}
{"src":"c1","dest":"payment","body":{"type":"ChargePayment","msg_id":3,"saga_id":"saga42","step":2,"params":{"user_id":"u42","amount":99.99}}}

Expected Output

{"src": "payment", "dest": "c0", "body": {"type": "init_ok", "in_reply_to": 1, "msg_id": 0}}

Idempotent refund on retryTimeout: 5000ms

Input

{"src":"c0","dest":"payment","body":{"type":"init","msg_id":1}}
{"src":"c1","dest":"payment","body":{"type":"RefundPayment","msg_id":2,"saga_id":"saga42","step":2,"params":{"payment_id":"p1","amount":99.99}}}
{"src":"c1","dest":"payment","body":{"type":"RefundPayment","msg_id":3,"saga_id":"saga42","step":2,"params":{"payment_id":"p1","amount":99.99}}}

Expected Output

{"src": "payment", "dest": "c0", "body": {"type": "init_ok", "in_reply_to": 1, "msg_id": 0}}

Hints

Hint 1▾

Tag each step with a unique saga_id + step_id combination

Hint 2▾

Services track processed steps to avoid duplicate work

Hint 3▾

If a step is retried, the service should return the same result

Hint 4▾

Use idempotency keys: the service checks if it already processed this step

Hint 5▾

Example: ChargePayment(saga42, step2) should only charge once, even if retried

Resources

Idempotency Patterns

AWS blog on idempotency patterns

                      OVERVIEW
                    

Theoretical Hub

Concept overview coming soon

Key Concepts

idempotencydeduplicationexactly-once semanticsmessage retriessaga_id + step_id

main.py

python