Citations that check themselves

Austin Wagner

18 research wikis share one audit engine, and each declares which kind of proof its citations owe: a PDF hash, a captured-page hash, a resolving document query, or a git blob. A claim that cannot be checked that way fails the write.

Problem
Notes rot silently. A citation that pointed at something real when it was written is indistinguishable, later, from one that never did.
Result
5 source-contract kinds, one per class of evidence, dispatched by the wiki's own declaration, plus a separate guard that checks the container rather than the claims.
Scale
One wiki of papers, 2 of captured technical and legal pages, one civic document vault, 10 codebase wikis, and 4 that pin nothing and say why.
Stack
TypeScript, PostgreSQL, hybrid dense + BM25 retrieval, MCP tools.
Where it runs
The same rules run on the filesystem, inside the database write transaction, and through a tool an agent calls.

One contract per kind of evidence

"Cite your source" means different things depending on what the source is, and a single rule for all of them either blocks honest writing or waves through unverifiable claims.

ContractWikisWhat a page must pin
PDF archive1A local PDF, its SHA-256, and the vault document id, so the paper a claim rests on is the one still on disk
Captured HTML2The saved page, its hash, its URL and the date captured, for sources that change under you
Document vault1A query that must resolve to at least one hit in the document store, re-checked rather than assumed
Code paths10Source paths plus each cited file's git blob hash, so editing the code invalidates the citation
External URLs4Nothing, and the contract says so explicitly, because those wikis have no local archive to pin; their sources are live URLs and repository paths

The last row is the one worth arguing about. Declaring "these wikis pin nothing, and here is why" is a stronger position than quietly exempting them, because the exemption is then visible and has to be defended. The rules that remain universal still apply, so every footnote must name a page, section, URL, line anchor or commit.

A code-paths page looks like this in the public monorepo, copied from its frontmatter: the audit re-hashes each path and fails the page when a pin no longer matches.

source_paths:
  - services/Model-Trainer/src/model_trainer/cli/cartridge_composition_sweep.py
  - services/Model-Trainer/src/model_trainer/core/services/model/cartridge_measurement.py
source_git_blobs:
  "services/Model-Trainer/src/model_trainer/cli/cartridge_composition_sweep.py": 2abd00e4e999a0b87a27f40d6f3650f7629b0684
  "services/Model-Trainer/src/model_trainer/core/services/model/cartridge_measurement.py": 9856d519c51064f045c27e30c1ab65192cfb99e2

API/wiki/pages/model-trainer-composition-ceiling.md, frontmatter at c447b0c; two of its four paths and their pins shown.

The rule that made this page possible

Change a cited file and the pin stops matching, so the page is flagged as describing code that has moved rather than continuing to read as current. It is the same mechanism the article you are reading depends on: every figure on this site is bound to a named wiki page and section and re-resolved before the page can deploy.

Writing and then auditing the first 14 articles found 14 of their 20 source pages carrying a wrong claim, every one marked high confidence. One described an actively running project as mothballed. One said "eight commands" while naming seven, against a real 15. Re-counting a page's numbers had not caught its sentences.

The audit checks the claims; something else has to check the container. A separate guard verifies each wiki's index counts against the files on disk, and it caught a page moved between two wikis that left four counts behind: 858 declared against 857 on disk, 70 against 69, 37 against 38, 11 against 12, in one red run.

Why an agent needs this more than a person does

A person writing from a source they just read is unlikely to invent a figure. A language model asked to summarise a corpus will produce something plausible whether or not the corpus supports it, and plausible is the failure mode that survives review.

So the check is not advice, it is a gate: the same rules run in the write transaction, on the filesystem, and through the tool an agent calls, and a claim that fails them does not land.

Record

Every figure above is bound to a named section of a wiki page and checked mechanically before this page can deploy: the contract kinds and counts to wiki-check-audit-chain, the page move to wiki-structure-guards (both in the codebase wiki that is itself under the code-paths contract, so they pin the blob hashes of the audit engine they describe), and the stale-page tally to public-dashboards.

Known limits. The engine ran 77 rules on 2026-09-14 and had grown by 11 between two readings without the audit noticing, because a pin covering one file attests to one file. None of this checks whether a claim is true, only whether it is anchored to something a reader can open.

Sources

  • The audit engine and the three wiki pages named above live in a private repository; the wiki pages cannot be linked from here.
  • A wiki under the code-paths contract, public: API/wiki at c447b0c, every page carrying source_git_blobs.
  • The gate this site runs: Dashboards/scripts/provenance_gate.py, whose docstring records why CI cannot run it.
  • This article's own manifest: provenance.json, one entry per figure above.