Implement Protocol Evolution - The Migrator

                      TASK
                    

Implementation

As services evolve, they add new fields and change message schemas. Protocol evolution strategies ensure old services can still read new messages (by ignoring unknown fields) and new services can still serve old clients (by transforming messages down to old schemas).

Implement a node that handles protocol encoding, decoding, negotiation, and transformation:

// Encode a message with Protocol Buffers
{ "type": "encode", "msg_id": 1, "format": "protobuf",
  "data": {"id":1,"email":"user@example.com","full_name":"John Doe"} }
-> { "type": "encoded", "in_reply_to": 1,
    "format": "protobuf", "encoded": "<base64>",
    "fields": ["id","email","full_name"] }

// Old schema decodes v2 message: unknown fields are silently dropped
{ "type": "decode", "msg_id": 2, "format": "protobuf",
  "schema": "v1", "data": "<base64-v2>" }
-> { "type": "decoded", "in_reply_to": 2,
    "schema": "v1",
    "data": {"id":1,"email":"user@example.com"},
    "unknown_fields_ignored": true }

// Negotiate highest common version
{ "type": "negotiate", "msg_id": 3,
  "client_versions": ["1.0","2.0"] }
-> { "type": "negotiated", "in_reply_to": 3,
    "version": "2.0",
    "server_versions": ["1.0","2.0","3.0"] }

Sample Test Cases

Protocol Buffers backward compatibilityTimeout: 5000ms

Input

{
  "src": "client_v1",
  "dest": "service",
  "body": {
    "type": "encode",
    "msg_id": 1,
    "format": "protobuf",
    "data": {
      "id": 1,
      "email": "user@example.com",
      "full_name": "John Doe"
    }
  }
}

Expected Output

{"type": "encoded", "in_reply_to": 1, "format": "protobuf", "encoded": ".*", "fields": ["id", "email", "full_name"]}

Decode with old schema (ignores unknown fields)Timeout: 5000ms

Input

{
  "src": "service",
  "dest": "decoder",
  "body": {
    "type": "decode",
    "msg_id": 1,
    "format": "protobuf",
    "schema": "v1",
    "data": "BASE64_ENCODED_V2"
  }
}

Expected Output

{"type": "decoded", "in_reply_to": 1, "schema": "v1", "data": {"id": 1, "email": "user@example.com"}, "unknown_fields_ignored": true}

Hints

Hint 1▾

Protocol Buffers: unknown fields from a newer schema are ignored by older readers

Hint 2▾

Decode with old schema: ignore fields the old schema does not know about

Hint 3▾

Protocol negotiation: choose the highest version that both client and server support

Hint 4▾

Transform v2->v1: map new field names back to old names (full_name -> name)

Hint 5▾

Encoding returns the field names actually serialised into the message

Resources

Protocol Buffers

Protocol Buffers: language-neutral, extensible serialization format

API Protocol Evolution

API Protocol Evolution guide

                      OVERVIEW
                    

Theoretical Hub

Concept overview coming soon

Key Concepts

Protocol Buffersbackward compatibilityunknown field handlingversion negotiationmessage transformation

main.py

python