When I started Zue, I thought: “I’ll just use JSON! It’s debuggable, easy, and everyone loves it.”
Then I realized that serializing { "key": "user:123", "value": "..." } for every single request is:
Slow: String parsing is CPU-intensive.
Verbose: The keys take up ton of space than the data.
So I went lower level. I designed a simple Length-Prefixed Binary Protocol.
The Packet Structure
Every message in Zue looks exactly like this on the wire. No delimiters. No newlines. Just raw bytes.
%%{init: {'theme':'base', 'themeVariables': {'fontSize':'16px'}, 'flowchart':{'nodeSpacing': 60, 'rankSpacing': 60}}}%%
flowchart LR
subgraph Packet ["Binary Packet Structure"]
direction LR
Len["Length (4 Bytes)<br/>(Little Endian)"]
Type["Type (1 Byte)<br/>(Enum)"]
Pay["Payload (Variable)<br/>(Protobuf / Struct)"]
end
Len --> Type --> Pay
style Len fill:transparent,stroke:#10b981,stroke-width:3px
style Type fill:transparent,stroke:#f59e0b,stroke-width:3px
style Pay fill:transparent,stroke:#3b82f6,stroke-width:3px
style Packet fill:transparent,stroke:#6366f1,stroke-width:2px
Why length-prefixed? Because TCP is a stream.
The “Partial Read” Trap
Beginners (read: me, two weeks ago) think that if you call send("Hello") on the client, you will receive “Hello” in one recv() call on the server.
Wrong.
You might get “Hel”. Then “lo”. Or you might get “HelloWor” if two messages got stuck together.
Length prefixing solves this. My server reads 4 bytes first. It gets the number 50. It then loops read() until exactly 50 bytes have arrived.
%%{init: {'theme':'base', 'themeVariables': {'fontSize':'16px'}}}%%
sequenceDiagram
participant Net as Network
participant Buf as App Buffer
Net->>Buf: "Length: 50" (4 bytes)
Note right of Buf: App knows: Wait for 50 bytes.
Net->>Buf: "Payload: Hello..." (10 bytes)
Note right of Buf: Total: 10/50. Keep reading.
Net--xBuf: *Pause* (Network Lag)
Net->>Buf: "...World!" (40 bytes)
Note right of Buf: Total: 50/50. Done!
Buf->>App: Deserialize()
The Async Event Loop
The second biggest mistake I made was using blocking I/O.
In v1, if a Follower node was slow to respond, the Leader would just… wait.
The client would wait.
The universe would wait.
To fix this, I rewrote the entire engine to use a single-threaded Event Loop using poll().
Leader Loop
The Leader never blocks. It checks sockets. If they have data, it reads. If they don’t, it moves on. It runs a tickRepair function periodically to fix broken followers in the background.
The Followers are simpler. They just do what they’re told. If a client tries to write to them, they politely yell “I am not the Leader!” and close the door.