TASK
Implementation
Implement data migration between replica groups:
- Source group: stop accepting writes for migrating shard
- Create snapshot of shard data + client sessions
- Send to destination group
- Destination: install snapshot, start serving shard
- Source: delete shard data after confirmation
Handle failures: retry, idempotency, rollback.
Sample Test Cases
Prepare shard for migrationTimeout: 5000ms
Input
{"src":"c0","dest":"g1","body":{"type":"init","msg_id":1,"node_id":"g1","node_ids":["g1","g2"]}}
{"src":"c0","dest":"g1","body":{"type":"seed_shard","msg_id":2,"shard":3,"data":{"x":1,"y":2}}}
{"src":"c0","dest":"g1","body":{"type":"prepare_migration","msg_id":3,"shard":3,"target_gid":"g2"}}
Expected Output
{"src":"g1","dest":"c0","body":{"type":"init_ok","in_reply_to":1,"msg_id":0}}
{"src":"g1","dest":"c0","body":{"type":"seed_shard_ok","in_reply_to":2,"msg_id":1}}
{"src":"g1","dest":"c0","body":{"type":"prepare_migration_ok","in_reply_to":3,"msg_id":2,"shard":3,"target_gid":"g2","snapshot":{"data":{"x":1,"y":2}}}}
Hints
Hint 1▾
Stop serving shard during migration
Hint 2▾
Transfer all key-value pairs
Hint 3▾
Include client session state
OVERVIEW
Theoretical Hub
Data Migration
Moving shards requires moving data. This must be atomic per shard and consistent. During migration, the shard may be unavailable or served by source (stale reads OK) until transfer completes.
Client Session Transfer
Don't forget client deduplication state. If sessions aren't migrated, clients may see duplicate execution on retry. Transfer the client session table with the shard data.
Key Concepts
migrationdata transferconsistency
main.py
python
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
#!/usr/bin/env python3
import sys
import json
import threading
class MigrationManager:
def __init__(self, group):
self.group = group
self.outgoing = {} # shard -> snapshot
self.incoming_complete = set()
self.lock = threading.Lock()
def start_outgoing(self, shard, target_gid):
# TODO: Prepare shard for migration
pass
def get_shard_snapshot(self, shard):
# TODO: Return shard data + client state
pass
def receive_shard(self, shard, snapshot):
# TODO: Install shard data
pass
def confirm_received(self, shard):
# TODO: Delete local copy after confirmation
pass
def is_shard_available(self, shard):
# TODO: Check if shard is ready for operations
pass