Trust · Methodology
How a claim gets made
Seven stages between a public document and a sentence in an answer. The sixth stage can fail, and when it does the sentence is withheld rather than softened. This page describes each stage in plain terms.
01
Source
A source is an organisation or feed we have catalogued: a regulator, a company, an agency, a court. Tracked means listed; producing means at least one document has arrived.
02
Ingestion
Documents are fetched on a schedule, deduplicated by content hash, and parsed to text. Headings and character offsets are kept so a chunk can be located in its document later.
03
Chunk · embed
Text is split into overlapping chunks that keep sentences whole. Chunks are embedded and indexed with their document, heading and sector. Before launch, embedding runs under a small daily budget, so newly ingested chunks can wait days before they are searchable.
04
Retrieval
A question is routed to one or more sectors, then matched against chunks by vector similarity and reranked. If the embedding service is unavailable, a slower keyword match is used instead. The top results are kept with their identifiers.
05
Synthesis
A model drafts the answer from retrieved chunks only. Every sentence that asserts a fact is marked as a claim span and tied to the chunk it came from.
06
Validator
Each claim span is checked for a verbatim match in its chunk. One retry with a stricter quoting prompt; then the claim is withheld. Nothing is rewritten to pass.
07
Citation
Passing claims receive a numbered citation with the full chain: claim, source, document, chunk, verbatim span. Citation rows are append-only; a citation that stops resolving is counted as unresolved, not removed.
Validation, published
What passed and what was withheld
A withheld brief is not an error to hide. It is a measurement of how often the corpus could not support what the model drafted. We publish the rate because a reader cannot judge the passed briefs without it.
1,747
briefs passed validation
all time
queried 2026-10-21 09:37 UTC
601
withheld
26% failure rate
queried 2026-10-21 09:37 UTC
a brief counts once, in the state it finished in · the weekly series will be published here when the rollup carries it
Citation discipline
Three parts of the check
The matcher decides whether a sentence is supported. The fallback decides what happens when it is not. The bibliography records what was used, in a form a reader can follow without our software.
The matcher
Whitespace is normalised, then the quoted span must appear as a substring of the retrieved chunk. Case is preserved. Punctuation is preserved. Paraphrase does not count.
claim span …cleared at $269.92/MW-day RTO-wide…
chunk …RTO clearing price of $269.92/MW-day for the 2025/2026…
resultmatch: exact → verified
The fallback
A first miss triggers one more pass with a stricter quoting prompt. A second miss withholds the claim and the paragraph renders as insufficient evidence. There is no third pass and no softened rewrite.
The bibliography
Every brief exposes a bibliography of the documents actually matched, with stable identifiers for source, document and chunk, in APA, MLA, Chicago or Harvard. Exports read from the append-only citation rows.
GET /api/briefs/{brief_id}/bibliography?style=apa
Fail-closed conditions
What fails closed
Three conditions under which the system declines to say something, and what you see when it does.
| Condition | What happens | What you see |
|---|---|---|
| No verbatim match after two passes | The claim is removed from the answer and listed as withheld. The rest of the answer stands. | WITHHELD block · insufficient_evidence · validate 2 of 2 |
| Numeral differs from the source | A number that does not appear as written in the chunk fails the match even if the sentence otherwise agrees. $1.2B does not match $1.2 billion. | withheld · no substring match |
| Source document no longer resolves | The citation keeps its verbatim span and is counted as unresolved in the published citation-integrity rate. | counted in the resolve rate on /transparency |
conditions as implemented by the citation validator
More on the check: Citation discipline·Methodology
The brief shown live at the top of the sample page is a published brief that went through every stage above. The prose beneath it there is an illustrative sample of the format.
See a sample brief Start free