A model that checks itself

Austin Wagner

I reverse-engineered the private protocol of an online game that publishes no documentation, then built an instrument that keeps proving the model is still right. It has since caught three of its own conclusions being wrong, including the ending of its best result.

Problem
No documentation and an encrypted wire, with a key table derived per session from a value the server injects into the page.
Result
32 message types decoded, each pinned to a signature, driving an autonomous bot that plays full sessions.
Best session
84 kills, 0 deaths, 149 minutes, unbroken. It ended because a fresh map held no affordable enemy.
Stack
Python, Playwright and the Chrome DevTools Protocol, a hierarchical state machine, a simulated server.
Gate
7,007 tests at full statement and branch coverage, no mocks. A partial mutation sweep on top, not a standing check.

One decoder per wire byte

Traffic is captured through the browser's own socket, because synthetic events do not work: commands have to travel the real connection. The codec is XOR with a static key and a per-session magic key, and I reversed both directions, encode and decode.

The container envelope has one dispatcher: subtype-first for everything it tunnels, structural for the few shapes that carry no unique subtype byte. The detail worth an interview is what a corpus did to it. Six length-based fallbacks were deleted after a sweep of 150 sessions and 48,304 container bodies showed zero production fires for any of them. Decoders are removed on evidence, not kept in case.

def decode_0x2e_message(data: bytes) -> BinaryMessage:
    """Decode a 0x2E container body - single source of truth.

    Subtype-first dispatch covers every protocol-tunneled type (handed
    to the protocol decoder for that subtype). Container-only subtypes
    with length-based variants (...) and container types without a unique
    subtype byte (...) are handled by `decode_container_message`, called
    as a fallback.

clients/TankpitBot/src/tankpit_bot/protocol/decoders/tank.py, the dispatcher's docstring, two parenthetical lists elided.

Three conclusions the instrument refuted

The marathon session was real. Its ending was not: I had written that the bot ran out of opponents, and the operator said that was impossible because enemies respawn and there are always 27. He was right. The record showed every enemy rejected as stale map data, including one ten tiles away, because a map open had completed in 12 milliseconds by consuming an orphan flag from an earlier stalled open.

Then my explanation of that was refuted the same night. I had written that the connection was dying. It was not. The last frame the session ever received was the answer to its final question, arriving 5.7 seconds after dispatch and 3.7 seconds after the bot had already quit. What degraded was latency, not the wire.

The same disease then turned up a third time, in a different subsystem: every reported movement stall was fuel-starved latency crossing a fixed local deadline. In the larger run the over-budget tail count matched the stall count exactly, 102 to 102. The stalls were never stalls.

One mechanism, three disguises: a slow answer crossing a fixed deadline, minting evidence that something downstream then believes.

Measurements that fail the build

Beliefs carry provenance and confidence, so every observed entity records where the knowledge came from. Validators price the model against the real capture archive, and every archived binary message must survive a decode and re-encode unchanged. Invariants raise where they are violated rather than after several observations of the consequence.

The part I would keep in any project: each machine-checked claim about the game's physics is one executable symbol, verified against the value or probe grid its wiki claim records on every build, and re-derived from the run archive by a separate audit. A measured claim that stops holding fails the build instead of quietly rotting.

It earned that discipline. An optimisation probed ten clean runs out of ten, so I built and gated it; a second probe minutes later dropped 7 of 10, and it was reverted the same night, before it ever ran live. Round one alone would have shipped a change that cost time on every teleport.

Record

Every figure above is bound to a named section of the wiki page tankpitbot and checked mechanically before this page can deploy. The project keeps its own wiki of 6 hubs and 83 pages as the source of truth for wire protocol and mechanics; the session figures are in its log with their artifact ids.

Known limits. The session figures are one session, not an average over many. The mutation sweep covered 37 of 483 invariant guards once, found 6 survivors, and killed them; it is not run on every build. Until 2026-09-14 this page said the container's subtype byte was scrambled per session and could not be trusted, a claim the decoder and the project's cipher page do not make; re-measuring the numbers twice had not caught a wrong sentence.

Sources