REFERENCE CODING MAKES CLAUDE CODE
6.5× CHEAPER — MEASURED.

HOW IT WORKS, IN DEPTH.

• 1 · The prompt resolves to llms.txt. The one-liner points the agent at xerj.org/llms.txt — ordered machine-readable steps: install from xerj.org/get, start the single binary, index, query over the Elasticsearch wire, and the reference-coding rules (retrieve before writing, cite what you use, respect licences).
• 2 · The corpus is chosen by problem domain, not language. The agent reads the project's dependencies and domains, picks the open-source repos that already solve those problems, and git clone --depth 1s them into one folder per domain. Record each repository's licence alongside the clone; GPL/AGPL entries are approach-only — read the design, never copy the code.
• 3 · Indexing is AST-aware. xerj autoindex <clone>: every file is sniffed by content, and source files go through tree-sitter grammars (13 languages) that emit the language, every symbol with its kind and line number, a searchable defs field, and the full body. One index per dataset, incremental on re-runs — re-index when the references change.
• 4 · Retrieval is hybrid, with a stated fallback. The retrieval step is a BM25 multi_match over body, defs and title — with a measured (not hand-picked) boost on defs so exact symbol names outrank prose — RRF-fused with a vector arm where an index supports it. When no index does, it degrades to BM25-only and says so; a silently lexical-only "hybrid" would be a lie. The default embedder is lexical feature-hashing — vocabulary overlap, not neural understanding.
• 5 · The loop has discipline. Before writing non-trivial code against an unfamiliar API the agent retrieves, reads the returned definition with its file:line, adapts it, cites the source, and checks the licence. If nothing relevant comes back it says so and falls back to normal work — forcing a bad match in is worse than not retrieving.
• 6 · Why the tokens collapse. A retry loop on an unmemorised API burns output tokens — the kind priced 5× above input on Claude models. Grep recovery pulls source into context: up to 1.06M input tokens on one corpus in this study. A retrieved passage is a few hundred tokens and carries the one thing a compiler can never leak — the runtime contract. The per-task table below shows the collapse: thousands of tokens from memory, hundreds with retrieval.

EVERYTHING WE BUILT TO TEST IT.

You cannot measure retrieval on code the model has memorised — it reproduces even a 256-value table from memory. So the unfamiliar corpus is 13 real libraries written for this study (each compiles and passes its own tests), each carrying a runtime contract the compiler cannot warn about. Each task returns the library's own type, so a hand-rolled workaround cannot pass. bare = from memory; XERJ = same model, reference retrieved.

* bare "passes" some cases by recovering an API name from a compiler/runtime error at 20–150× XERJ's tokens, or because the task spec necessarily stated the one non-obvious rule — never by recalling a genuine runtime contract. On the two memorised cases (quill varint, garner trie) the model already knows the algorithm and retrieval saves nothing — which is the point of including them. Source for every library: docs/case-studies/reference-coding →

THE LINE IS SHARP.

Two controls on real, public code the model has trained on — valkey + memcached (a KV server), and tantivy (a search engine) — pin down exactly where retrieval stops paying. The value is gated by memorization.

EVERY LANGUAGE, SAME SHAPE.

Median output tokens to solve, XERJ vs answering from memory. Verdict is the real toolchain in each language (python3, node, cc, javac).

THE SAME TASK, TWO WAYS.

One Java task: build an append-only ledger that must be sealed before replay, truncated to a checkpoint. bare cannot read the library, so it reinvents the whole thing — 503 lines — and still fails the seal contract. XERJ, handed the reference, writes it in four lines and cites the rule it learned:

// reinvented an entire append-log + checkpoint + seal
// state machine from memory, guessing the API surface …
// 503 lines … wrong truncation semantics
// → hidden test FAILED.

public static Ledger upTo(long[] vals, long seq) {
    Ledger l = Ledger.open();
    for (long v : vals) l.append(v);
    l.checkpoint(seq);   // keep 0..seq
    l.seal();            // required before replay
    return l;
}

CAPTURED VERBATIM · docs/case-studies/reference-coding/generated →

VERIFIED: THE PROMPT WORKS.

We ran the opener above verbatim on a fresh task — implement a top-3 heavy-hitters function using the unfamiliar sift crate (whose API is not new()/push()/top_k()). Following the prompt, Claude Code indexed the reference, retrieved sift's real API, and wrote code that compiles and passes a hidden test — using furnish / absorb / seal / crest, the seal-before-read contract a from-memory attempt cannot recover. The full transcript and the generated program are in the repo.

REPRODUCE THE VERIFICATION ITSELF · docs/case-studies/reference-coding →

DO IT YOURSELF.

Four commands, no tooling beyond the xerj binary and git. The method works on any repository — including your own private code, which is the real use case.

1. Start XERJ. xerj -d ./.xerj-data --insecure & — the ES-compatible API comes up on :9200. autoindex is a client of a running node; without one it exits 1.
2. Build a corpus. git clone --depth 1 https://github.com/tokio-rs/tokio ref/tokio (and the same for each peer) — group by problem domain, not language.
3. Index it. xerj autoindex ref/tokio — a 100k-record tree indexes in ~10 s; re-run when the references change and the journal makes it incremental.
4. Retrieve before you write. One query against the new indices returns the matching definition with file:line:

   curl -sXPOST 'localhost:9200/ax-*/_search' -H 'content-type: application/json' \
     -d '{"size":3,"query":{"multi_match":{
           "query":"graceful shutdown with a broadcast channel",
           "fields":["defs^3","body","title"]}}}'

   defs, symbols, body, title and language are the fields autoindex writes for source files. ax-* matches every dataset on the node, your own project included — check GET /_cat/indices after each index run and narrow the pattern to the reference datasets when you want your own code kept out of the results.
5. Or let the wrappers do it. xc-corpus.sh / xc-index.sh / xc.py ship in the repo at tools/xerj-code/ and wrap exactly the four steps above. They add a corpus definition: xc-corpus.sh --from hub/xerj-search.json rebuilds a corpus someone already vetted — URLs, pinned commit SHAs and licences; no source, so nothing to redistribute, and the same file gives two machines the same bytes.
6. Measure it honestly. Three arms of the same agent — memory only, grep-driven, retrieval-injected — one hidden-test verdict, real tokens from claude -p --output-format json, no API key. The numbers and every per-run record are in CASE_STUDY.md; the harness itself is not shipped.

Set me up for reference-coding with XERJ and then use it automatically for the
rest of our work, so I stop burning output tokens re-deriving APIs I could just
look up.

1. Make sure a local XERJ instance is running and the `xerj-code` skill is
   available. If XERJ isn't installed, get it from https://xerj.org/get
   (curl -fsSL https://xerj.org/get | sh) and start it.
2. Look at this project — its dependencies, the domains its code touches — and
   decide which well-known open-source libraries already solve those problems.
   Clone them into a reference corpus with the skill's xc-corpus.sh, grouped by
   problem domain, not by language.
3. Index that corpus with xc-index.sh (it runs `xerj autoindex`).
4. From now on, for every coding task this session: before writing non-trivial
   code against an unfamiliar or external API, first run
   xc.py <corpus> "<what you need>" to retrieve the real implementation, read the
   returned definition, and adapt it — cite file:line, check the licence. If
   retrieval returns nothing relevant, say so and fall back to normal work.
5. Re-index whenever the references change.

Do this proactively for the rest of the session without me asking again.

Kept for reproducibility, not as current instructions — the one-line prompt at the top of this page is what to use today.

WHAT WE WON'T OVERCLAIM.

POINT IT AT YOUR CODE.