Technical Design Document

Design

A document version is ingested once and stored as immutable raw text plus a SHA-256 of that text (document_versions.source_sha256). Versions are never edited in place — a correction is a new version, never a mutation, because every downstream offset assumes the text under it cannot move. app/pipeline.py enforces this: a version id already stored with different bytes raises CorpusChanged rather than being repaired, since every passage, change span and citation derived from it addresses text that no longer exists.

Ingestion then segments the raw text into passages: the unit diffing and citation both work over. Each passage is stamped with (version_id, ordinal, char_start, char_end, text, section), where the offsets index directly into the stored raw text — not into a re-flowed or re-normalized copy — computed in a single pass so a passage's offsets can never drift from a later re-parse.

The SHA-256 is there so that this is the document the citation points at is checkable independently of anything the database says. If the stored source ever diverges from what was ingested, the hash catches it — which is a different guarantee from the one the offsets give, and the two are kept apart deliberately (§5).

Why offsets are the unit

Every claim in the product — a diff segment, a citation, a piece of obligation evidence — resolves in the end to (document_version_id, char_start, char_end) against an immutable, hashed source. Offsets are the unit rather than passage ids, sentence ids or a similarity score, for one reason: an offset is checkable by exact substring lookup. No embedding maths, no fuzzy threshold, no model in the loop — read the bytes between two integers and compare them to what was claimed. That is what makes verification a pure function instead of another place where the product has to trust something.

Two consequences follow and both are load-bearing. Where a passage ends is a retrieval convenience, not the unit of truth — app/ingestion/ingest.py says so in its own words, which is why a crude segmentation is survivable and a wrong offset is not. And offsets are addressed against the frozen version, never the current document, so a citation minted against version 3 stays valid after version 4 exists. Versions are never edited in place, only appended. §5 carries the rule that follows: an offset is never projected across versions through the alignment computed in §2.

Alternatives considered

Trade-off accepted. Paragraph-level segmentation is coarse when a long paragraph contains one small change: the passage offsets bracket the whole paragraph. The plan was to compensate with a character-level diff inside each aligned pair. That is not built. A Change row carries whole-passage offsets on both sides, and the change screen shows the two paragraphs side by side and lets the reader find the difference. Narrowing the span to the changed characters is a diff/-internal improvement, not a redesign, because nothing downstream depends on how wide the span is — only that it is exact.

PDF extraction, stated honestly

The plan was to treat PDF extraction as its own tested stage — page headers and footers, hyphenation across line breaks, column reflow all resolved once, before any offset exists. No such stage exists. grep -rn "pdfminer\|pypdf\|pdfplumber" app/ returns nothing, and ingest_version is handed decoded text in every path.

The real corpus widens what is untested rather than closing it. data/real/ holds 102 real public filings retrieved from 8 jurisdictions, each beside a .provenance.json naming its source URL, docket, filer, filing date, retrieval time and a SHA-256 of the bytes (ADR-058, ADR-059). That text was extracted from the commissions' PDFs outside this product and stored as text, so the extractor is still somebody else's, still untested here, and the risk is unaddressed rather than mitigated. What the real corpus does buy is input the normalizer had never met: line-numbered testimony, certificates of service, words hyphenated across line breaks, soft hyphens and ligatures, in documents nobody on this side shaped to suit the code.

One piece of PDF reality is handled. Soft hyphens and hyphen-against-line-break are folded by app/text/normalize.py, which runs on both sides of every comparison at verification time (§5) — not at ingestion, before offsets exist. Normalization that ran at ingestion would rewrite the text the offsets index, and verification must never fix text out from under an offset it did not compute. tests/test_pdf_text.py exercises the occurrence check over text shaped the way an extractor emits it — soft hyphens, ligatures, wrapped lines — which is a test of the normalizer, not of an extractor the product does not have.

Design

Diffing is deterministic code and the model is not part of it. diff(before, after) normalizes both passage sequences, runs difflib.SequenceMatcher over them, and turns each opcode into typed changes — added, removed, modified. Inside a replace opcode the two runs are paired positionally and any surplus on either side is reported as a pure addition or removal. Given the same two sequences it returns the same list every time, which is what lets the corpus's deliberate edits be asserted exactly rather than approximately.

Every modified pair carries an alignment confidence, and that number is where the design does its real work. Text similarity alone is the wrong measure, and wrong in the dangerous direction: when a section is renumbered and relocated, the words barely move, so similarity runs high precisely when the structural identity has changed. So a disagreement in section label caps the score at RESTRUCTURE_CONFIDENCE_CEILING = 0.5, below any plausible escalation threshold. The passages may well correspond. The alignment is now an inference rather than an observation, and it may not present itself as settled.

A reviewer can answer the inference, and the answer is kept. 16 of the 27 changes in the demonstration corpus carry the caution — more than half of it — so printing the doubt and offering no control over it would be asking for judgement and dropping every answer. app/state/alignment.py records one of three verdicts against the change: the pairing is right, the pairing is wrong, or cannot tell, with optional reasoning. It is written into the hash chain as change.alignment_confirmed, change.alignment_disputed or change.alignment_uncertain, filed under the change, carrying the actor, the account and the moment. POST /changes/{id}/alignment is the route and it is gated on action.propose. The computed number is never overwritten: it stays on Change.alignment_confidence and is quoted into the audit reason as it stood when the person answered, so what the machine thought survives the person disagreeing with it. An unanswered pairing keeps its caution, and so does one answered "cannot tell" — an alignment nobody could resolve is exactly as uncertain as one nobody has read. A confirmed or disputed pairing loses it, because the caution's own words are "read the pairing as a guess, not a fact" and a named person has just made it something else.

One predicate, two surfaces, and that is the load-bearing part. pairing_caution_stands holds the rule for the screen and the clerk together, and tests/test_alignment_decision.py::test_the_screen_and_the_clerk_agree_on_every_answered_state walks all four states through the page and the tool. The test for which pairings are in doubt lives in one function, pairing_is_uncertain: the screen and the clerk each held their own copy of alignment_confidence <= 0.5, and 8 of the cautioned changes sit at exactly that value, so a one-character difference between the two copies would have decided half the cautioned corpus in one surface and not the other. The clerk carries the verdict beside the number on both the change list and the change detail, because a tool that reports the machine's doubt and drops the person's answer can tell somebody a pairing is unsettled after it was settled.

The section label is a proxy and it was measured rather than assumed. _section_label takes the leading integer of a passage, and among the labels it hands back across data/real are 2025, 207 and 480. Those are years and testimony line numbers, and on the real version pairs a large share of the modified changes escalated for no reason better than a printed gutter number shifting by one. app/diff/structure.py was written to fix that: it reads a filing's own numbering into a tree where it can and refuses with a stated reason where it cannot. It is not wired in, and no confidence in this product moves because of it. Measured on the real pairs, the two branches it was built for — a hierarchy contradicting agreeing labels, a hierarchy confirming one node — never fired, and every confidence that moved moved because the parser had failed to read both documents and that failure was being used to withhold the cap. A parse refusal raising a score is absence licensing certainty. So the parser, the measurement and the refusals were kept and the wiring was taken out; the alarms the proxy raises on line-numbered pages are still there and still worth fixing, on the proxy's own terms. Every figure is in structure-measurement.html, which is re-derived from the corpus by one command and guarded by a test.

Alternatives considered

Trade-off accepted (ADR-004). Sequence alignment assumes the document's order is stable enough to follow. Wholesale restructuring breaks that assumption for many passages at once, not one — and in the corpus it does exactly that. Renumbering section 6 to 5.4 accounts for some of the capped pairings; the section-7-to-6 ripple that no manifest change names accounts for the rest, which nothing in the manifest predicted. The cap is not a scalpel here: the capped pairings run contiguously from section 6.1 through 7.4, and a reviewer handed a caution on every one of them has been handed a list of everything. This is not a rare edge case in regulatory drafting; a final order routinely renumbers a draft's sections. See §14.

Design, and what of it exists

Extraction is the stage where a model reads source text and produces a structured claim, downstream of a diff that has already enumerated what changed. The package is two files and only one of them calls a model: action.py, the status-to-vocabulary table with no model in it (§7), and propose.py, which asks claude-opus-5 whether a change is material and hands every answer to the verifier (ADR-037).

Interpretation was designed to do four jobs. Three are answered somewhere other than here, and the fourth is answered nowhere. One, judge materiality — does this change matter. propose.py does it, on the change screen's first view of an unjudged change, and Change.materiality stays NULL until something judges it rather than carrying a default that would read like a judgement. Two, extract obligations with evidence spans — read a duty out of the filing and bind it to the characters that state it. This one is not built at all and has no half anywhere. The obligations table holds the company's own duties, loaded from data/company_context.json by scripts/seed_demo_gaps.py; no code reads an obligation out of a docket, so no obligation in this build carries an evidence span. Three, map a change to the company-context obligations, projects and documents it affects — change_obligations holds that edge, app/state/mapping.py proposes candidates from shared words and a person confirms one, and nothing derives an edge from meaning; documents have no table (§4). Four, recommend an action and a reviewer — action.py supplies the vocabulary by dispatching on the version's status, and app/state/routing.py picks the reviewer off obligations.owner_user_id (§7).

The shape of that module is the part worth reading. Every proposal must name a (version_id, char_start, char_end, quoted_text), and judge_materiality re-reads the stored source at those offsets through app/verification/verifier.py before anything survives. Nothing is repaired. A proposal whose offsets sit two characters off the quote is withheld, not nudged into place — snapping an offset to the nearest matching span would make the gate a formality, because every citation would be adjusted until it passed and the verifier would end up checking the repair. A malformed proposal is dropped one at a time with its reason recorded, so one bad entry does not discard the good ones beside it. The model is never asked whether a version is draft or final: that is read from the field and dispatched on in Python (ADR-005), and the permitted actions go to the model as a closed list.

Three limits. The module writes no claim — nothing in it creates a Claim row, because there is no action code for "a model proposed this" and inventing a second spelling of one is how the two that already drifted got that way. It does write a verdict: _record_verdict writes the verdict, the reason and the citation that earned it onto the Change row and appends ACTION_MATERIALITY_SET to the chain under ACTOR_MODEL. And its request has been sent for real once deliberately, and once more by accident — every test drives a deterministic fake through an injected transport, so what the suite proves is the gate and not the request. The deliberate request went to the endpoint on 2026-08-05 through scripts/probe_live_transport.py: the parameters were accepted, the structured output came back in the shape _SCHEMA asks for, and the transcript sets out what that settles and what it leaves open — including the second call the same day, whose output was piped away and lost, so nothing at all can be said about it. The claims a reviewer actually sees are assembled deterministically: app/seed.py writes the synthetic ones by rule from data/manifest.json, and scripts/ingest_real.py carries exactly one hand-written statement about the real Kentucky filing, cited at the offsets where it sits.

That leaves this section a contract as well as a description, and the contract is the useful part. Whatever produces a claim — the model, the seed, a person typing one into a script — emits the same typed object, and that object is the only thing that crosses into verification. Nothing else about the producer is visible downstream, which is what makes the model swappable and, more importantly, what makes the gate in §5 independent of it.

Design

Three things carry the company side of the join. An obligation is a regulatory duty the company holds, recorded in the company's own words. A project is an internal initiative that fulfils or is constrained by obligations. A document is one of the company's own artefacts — a tariff, a compliance policy, a procedure — evidentiary rather than regulatory.

The corpus (data/company_context.json) holds the obligations, projects and documents, deliberately worded the way the company would word them rather than the way the docket does. That is not incidental realism. It is the reason a lexical match is not enough. OBL-001 reads Post security for new large-load interconnection work before construction starts. The clause it answers to, section 6.1, reads A Large Load Customer shall post collateral in an amount equal to 100% of the binding cost estimate produced by the Facilities Study, prior to the Utility's commencement of construction of Interconnection Facilities or Network Upgrades. The words common to both are post, large, load, interconnection and construction — and none of them is the one that matters. The company posts security; the docket posts collateral. A lexical match has to bridge that word, and the ones it does share are the generic terms this docket uses throughout.

How much of that is a table. Projects are rows. obligations is a loaded table: scripts/seed_demo_gaps.py reads them out of the JSON and resolves each owner_name to a real account by name, and where no account matches it writes the obligation without an owner rather than with a guessed one — an unowned obligation is visible and unroutable, which is the honest state. change_obligations exists and the change screen writes it: propose_obligations_for_change offers candidates on every render and confirm_obligation_for_change commits the one a person accepts, through map_change_to_obligation. Documents have no table at all. The candidates are proposed from words the two sides share, which is the half of the join that does not need the word that matters.

Contract

A citation is (version_id, char_start, char_end, quoted_text), plus cited_occurrence where the quote is not unique. verify_citation slices the source at the offsets, runs both the slice and the quoted text through the same normalization, and compares them for equality — never a similarity threshold. It applies no threshold, and the reason is not this document's own: "A paraphrase is exactly what an auditor will not accept, so a threshold would defeat the point of having the gate at all," says the docstring at the top of app/verification/verifier.py. So "close enough" is not a verification outcome this system produces, and a reader who wants to check that the code agrees with the document reads that docstring rather than taking this paragraph's word for it. It then counts how many times the quote occurs in that version, and if it occurs more than once the claim must name which occurrence it relied on or it is refused.

The module calls no model and makes no network request, on purpose twice over: the code that decides whether the product may assert something has to be auditable by a reviewer who trusts nothing about the AI, and it has to run in CI with no API key. It only compares. It cannot itself misquote fluently, because it does not generate anything.

Normalization — stated explicitly, because an unstated normalization rule is a silent one

The same function runs on both sides of every comparison. A normalization applied to one side only is not a normalization, it is an edit.

Outcomes

There are two: verified, or not. A refusal carries the reason and the bytes the source really holds at those offsets, so the analyst sees the mismatch itself rather than a status word. Most come from the canonical function; the ones only the scoped entry point can raise say so in their own row.

Graded outcomes were designed and none is built. verified_exact, verified_normalized, near_match and unverified, with a stored verification_status column and a bounded re-anchor search that would hunt for the quote a few characters either side of the given offsets. The stored column and the re-anchor were both bad ideas. A stored status is a promise about bytes that may since have changed; no claim carries a stored verdict, and the verdict is recomputed on every read. And a re-anchor window is a tunable that trades a false reject for a false accept, which is the one direction this design cannot fail in. Rejecting an off-by-a-little citation costs a review. Confirming a coincidental identical phrase elsewhere in the document costs the reason the product exists.

The gate is a read, not a write — and this is the one place that is told at length

The function every reader goes through is verified_claims(session, company_id, change_id) in app/state/claims.py. It fetches the change's claims, re-reads each cited source, calls the verifier, and returns two lists: what may be asserted, and what may not. It writes nothing and stores no verdict. Edit a source after a claim was written and that claim flips to withheld on the next page load, with no migration and no background job.

A write gate would be the weaker design. Verification between interpretation and state, with a verified flag going into the database, makes a promise about bytes that may have changed since. The cost of the arrangement that ships is arithmetic — verification runs once per claim per view instead of once per claim ever — and A7 names what to do about that at volume, which is a cache keyed on the source hash and still not a stored boolean.

There is one verified column in the schema: share_opens.verified, not nullable and with no default. It is not a cache of whether a claim is good today — it is the record of what a named recipient was shown at a named moment, and a writer that says nothing about verification has recorded nothing, so the insert fails rather than filing silence as a measurement. The rule's exact form: no claim carries a stored verdict, and there is no verified column on claims, changes or escalations for a stale one to hide in.

Every reader uses this function and none has its own copy. The readers are the change view in app/web/views/changes.py; the escalation queue and the claim view in review.py; the review centre; the project workspace; the actions screen; the public share page through app/state/sharing.py, which re-verifies at open time on every open; app/state/review.py; the assistant's change_detail and search_claims tools; and app/seed.py, which asks the same question when it writes the demonstration rows. Scripts under scripts/ call it too, outside app/. No total is printed here. One was, for three review rounds, and it was wrong in a different direction each time; the list is the claim and grep -rn "verified_claims(session" app/ scripts/ settles it. A withheld claim's wire form is built in one function and carries a reason code, a reason text and where it sits — never the statement, never the quote. Everywhere else in this document that touches the read gate — §6, §12, §13, A2 — states the fact and points back here.

Two entry points, and which one production uses

verify_citation(citation, source_text) is the canonical one. It trusts the caller for one thing it cannot check: that the text it was handed is the text of the version the citation names. That is a real guarantee to have to make — a citation naming v2 handed v1's text verifies whenever the quote sits at the same offsets in both, which repeated boilerplate across versions makes reachable rather than theoretical. On the path a rendered claim takes the pairing is made once, in verified_claims(), against a map keyed on version id, so there is one place to check it rather than one per call site. Off that path, app/interpretation/propose.py and app/evals/metrics.py each build their own version-to-source map and call verify_citation directly, so "once" holds only for the rendered path and this sentence is the qualifier.

verify_citation_for_version(session, citation, company_id) makes the pairing itself and adds the check the other cannot: it hashes the version's stored text and refuses when the digest disagrees with the one ingestion recorded. Nothing in app/ calls it. It is not on the path a rendered claim takes, because it finds its version by reading every version the company owns, and adopting it would repeat that read once per claim. Two consequences follow and neither is hidden: the hash check does not run in production, and closing that needs a single-row scoped read in app/state/queries.py that does not exist yet.

What that costs, precisely. The bytes under a citation are re-read on every render, so an edit landing on a cited span withholds the claim immediately. An edit elsewhere in the same version does not: the quote still matches at its offsets, and the claim still asserts itself against a document that is no longer the one ingested. That is the freshness principle unclosed on the path the product actually uses, and it is stated here rather than left to be found.

The harder case: a true substring that reverses meaning

A citation can pass exact-match verification and still misrepresent the source, if the quote is a real, correctly-offset substring that omits an adjacent qualifier — dropping "shall not" to leave "shall", or quoting a clause without the "unless" three words later that governs it. Verification proves the text exists at that location. It does not prove the text supports the claim. These are different guarantees and this design does not pretend otherwise.

The plan was a cheap adjacency check scanning immediately outside the cited span for negation and qualifier words. It is not built. There is no such scan in app/verification/verifier.py, so the truncation case is caught by nothing today — not the adjacent one, not the distant one. §14 carries it as the open risk it is.

Offsets are scoped to a version, never projected across versions

A citation into version 1 stays a citation into version 1's immutable text. It is never re-projected onto version 2's text through the alignment computed in §2. The alternative — a document-level coordinate space every version maps into, so a citation could follow a passage across versions — was rejected because that mapping is the alignment problem, and alignment has its own confidence and its own failure mode (§14); routing citation correctness through it would let an alignment error silently corrupt a citation that was otherwise exactly right. The accepted cost: when a change says a deadline moved, the interface shows two citations — the old span in the old version, the new span in the new version — not one citation that claims to have moved.

Adversarial test discipline

The verifier is tested against deliberately corrupted citations. A fabricated quote at real offsets — one numeral altered inside a real cost floor — must come back as a quote mismatch. A real repeated quote cited without stating its occurrence must come back as ambiguous. Both are scored in make eval (§9) and both must be rejected for the right reason: a rejection for the wrong reason is not a pass. The passing bar is every probe, not most. There is no partial credit for a citation verifier, because the product's claim to the reader is "if it is shown as fact, it verified," not "it usually verified."

tests/test_verification.py also fuzzes the occurrence scan over randomized adversarial text. It asserts the property — the fast boundary test never clears a span the exact re-read would refuse — and measures that the fuzz pool is strong enough to be worth asserting over: the run fails if fewer than a thousand hits reached the assertion, and fails again if no hit took the slower path, so the fallback cannot become dead code without the suite saying so. The scan re-reads what it refuses and never re-reads what it clears, which is the direction that keeps the guard from switching itself off quietly.

What that scan costs. Hits scale with the source, and re-normalizing each one was the cost. The boundary test — a hit is valid exactly when it does not share a raw offset with the character either side of it — made the check flat in the quote's length. It is still not O(n) in the worst case: the exact re-read survives for the hits the boundary test refuses, so a source made entirely of expanding characters still costs O(n·m). Filing text is not that; a page of accented capitals would be. The milliseconds this paragraph used to print came from one run on one laptop with nothing checking them, which is the class of number this document has stopped publishing.

Design

The verdict on a claim is not stored. verified_claims() re-runs the whole decision against the stored source every time a screen asks for it and returns two lists: what may be asserted and what may not. The argument for that, and the one verified column that does exist and is not a verdict, are in §5.

The order of the checks is deliberate. The citation is tested before the confidence, so a claim that misquotes reports the misquote rather than a number — that is what the analyst needs in order to act.

Confidence is attached at each stage that makes a judgement, and the sources are kept distinct rather than collapsed into one number: alignment confidence from §2, which is rule-derived and capped; the citation outcome from §5, which is a gate rather than a score and always forces refusal on its own; and a model-reported confidence on interpretation, which does not exist because app/interpretation/propose.py does not ask for one and would not accept one. That is a decision rather than an omission — a number a model produces about its own certainty is a number the gate would then have to argue with.

Where a model does eventually report a confidence, it will be treated as an ordinal signal, not a probability. Raw numbers from language models are poorly calibrated, and trusting a value like 0.83 at face value would be fake precision dressed as rigor. The bar would be tuned against the eval set's known answers (§9) — against outcomes, not against the model's own stated certainty.

Trade-off accepted (ADR-006). A review queue nobody empties is its own failure. That is why the escalation record carries enough context to be actioned in one look — the reason in plain words, the quote that was made, and the bytes the source really holds at those offsets — rather than making the reviewer re-derive it. It is also why the alignment cap in §2 does not raise an escalation: capping is common in a renumbered filing, and a queue that filled with every capped pairing would be a list of everything.

What routing means here, and how much of it runs

Two halves. The vocabulary half decides what kind of action a change may produce, from the version's explicit status and nothing else: a draft change offers monitor or comment, a final change offers comply and requires an effective date. It is a lookup table with no model in it (app/interpretation/action.py), and an unknown status raises rather than defaulting, because guessing this wrong is the most expensive error available in this domain (ADR-005). That half is built, tested and scored.

The routing half decides who is asked, and it is built. app/state/routing.py walks an escalation to its claim, the claim to its change, the change to the company obligations it bears on, and the obligation to its owner, then writes Escalation.assigned_to_user_id and assigned_at together. Obligations are rows, with an owner column that resolves to a real account.

The design work is in the refusals, not the routing. Every step can fail to name a person and none of them may guess when it does: no obligation maps to the change, the mapping behind it is the proposer's rather than a person's, the obligation has no owner, the owner's account is inactive or belongs to another tenant, two obligations name two different people. Each is a distinct code — ROUTE_NO_OBLIGATION, ROUTE_MAPPING_UNCONFIRMED, ROUTE_OBLIGATION_UNOWNED, ROUTE_OWNER_INACTIVE, ROUTE_OWNERS_DISAGREE and one per remaining way the walk can end without a person, all in grep -n "^ROUTE_" app/state/routing.py — and after each one the escalation is still in the shared queue with nobody's name on it (ADR-036, ADR-056). The temptation this file exists to refuse is the default assignee: the admin, the person who raised it, the last person who touched anything nearby. Each is one line of code and each is worse than the refusal, because a wrong assignment looks handled — it leaves the queue, sits on a desk that will not act on it, and nobody finds out until the deadline has gone. shared_queue() re-derives every verdict on read rather than trusting a stored one, for the same reason verified_claims() does: suspending an owner this morning has to change what the queue says this afternoon with nothing having re-run.

A mapping the pipeline proposed does not route, and that refusal fires only where a name would otherwise have been handed out. resolve_change_owner reads ChangeObligation.mapped_by_kind: where any mapping behind the change is a person's, only the person's mappings are considered and the proposer's are set aside entirely, because resolving over all of them lets a guess overrule a judgement by arithmetic. Where none is, a name that would have been handed out becomes ROUTE_MAPPING_UNCONFIRMED instead. Every other refusal stands: a proposed mapping onto an unowned duty still answers ROUTE_OBLIGATION_UNOWNED, because "give this duty an owner" is true whoever wrote the mapping, and app/state/invites.py branches on that code to decide an invitation is the fix. The cost, exactly. An owner-gap invitation on a proposed mapping completes — the gap is real, the invitation is written, the account is created — and the last step does not happen: the escalation stays in the shared queue until somebody confirms the mapping.

No screen assigns an item. app/web/views/review.py, changes.py, proceedings.py and projects.py all read escalations_for_company() — the flat company list, in id order, that everyone with the screen sees. resolve_change_owner is reached from app/state/invites.py, when an invited owner accepts or an invitation is withdrawn; from the assistant's tools; and from changes.py, which resolves an owner on every change render and assigns nobody. So the queue is routable and is not yet routed, and a reviewer clicking through the demonstration sees the shared list. The callers are grep -rn resolve_change_owner app/.

Beside routing sits the approval gate, which decides whether a given person may sign off on a given claim, and it is the part worth defending. Holding action.approve gets a user past its second check and no further. The fourth asks whether this person already acted on the claim underneath the action, on the change beneath it, or on an escalation raised against it — and refuses if they did, whatever their permissions say. Authorship is read out of the audit chain rather than from an authored_by column, because a second record of who did what, kept beside the first, is free to disagree with it.

Three system roles exist as data — analyst, obligation_owner, admin — and PERMISSION_CODES in app/state/models.py holds the permission vocabulary. Beside them, scripts/seed_roles.py carries role templates named for a function rather than a rank — regulatory analyst, regulatory counsel, interconnection, load forecasting, certifying officer, auditor and the rest — every code in every template checked against PERMISSION_CODES so a template cannot grant a permission the product does not define. They are templates, not a vocabulary. An administrator grants any code to any person; a template is copied, never edited, because a role called "analyst" that one company has quietly changed means something different on every screen and every ADR that names it. And no template holds both action.propose and action.approve — not because a four-person team may not have both, but because a default must not hand that over in silence.

The owner sits on the obligation record rather than the project, because one project carries obligations different people answer for, and a final change routes to the analyst and to that named owner. The column is obligations.owner_user_id, nullable, because an owner who leaves takes their account's usefulness and not the duty. What is missing is the surface: there is no screen that renders obligations, so an obligation nobody owns is visible only as the reason an escalation refused to route.

What the log is, and what it is not

Every decision the system takes appends one row to a hash-linked chain, scoped to a company: ingesting a version, recording a change, withholding a claim, granting a role, refusing an approval, resolving an escalation, composing a synthesis. One log, not two. A separate table for security events would drift from this one, and the log nobody reads is the one that goes wrong first — silently, because nothing compares them.

This is not an event fold. State tables are written directly — the pipeline inserts a Change row and then records an audit event beside it — and the log is a parallel, append-only record of what was written, not the thing that produces it. There is no fold and no projection. The guarantee that costs: with a real event fold a state table cannot silently disagree with its history, and here it can.

Taking one decision back is built, and it is a row rather than an undo. app/state/rollback.py restores the state the decision changed and then appends an event naming the one it undoes, in that order, inside the caller's transaction, and never edits or flags the original — writing anything onto it would be the rewrite the whole table exists to prevent, and a decision somebody later reversed is precisely what an auditor asks to see. The order and the shared transaction are the guarantee, not an implementation detail. revert_event() calls the registered restorer and only afterwards calls record_event(), and it opens no transaction of its own: it must be called inside one the caller aborts on failure, which app.state.db.session_scope does. Called outside one, a failure between the two writes would leave the state put back with nothing in the log to say why — a change nobody can account for, which is the exact condition this table exists to make impossible. The pointer sits inside the digest under scheme 3 (ADR-053), so re-pointing a reversal at a different event turns the chain red rather than quietly changing what the log says was undone. Two rules are load-bearing. A reversal may not make an unverified citation verify — only the source agreeing with the quote does that, and there is a test that reads this module's own source to stop a later edit blurring it. And taking back a reversal reinstates nothing (ADR-054): the subject stays where the reversal left it and waits for a fresh decision, because putting the original back would write a resolver's name onto state nobody re-decided.

The append-only rule binds at the point of writing rather than living in a comment. A SQLAlchemy before_flush listener on the Session class raises AuditTamperError on any UPDATE or DELETE of an audit row, in every session in the process — not only the ones a careful caller remembered to protect. verify_chain(session, company_id) then walks the rows in sequence order and raises on a sequence gap, a broken prev_hash, an altered row, or a digest scheme it cannot compute.

One limit on the reversal. Every entry in RESTORERS is the same transition — closing an escalation, by resolution, approval or rejection. An action with no entry there is refused rather than recorded, because a row saying a decision was taken back while no state moved is a fallback that did not announce itself.

What the log itself provides is narrower and true: it cannot be edited or deleted through application code, its rows verify against each other, and a revision anywhere in the product is a new row that points at the one it replaces.

Why append-only, against the alternative

Trade-off accepted. The rule is a convention, not a constraint: nothing outside the intended write path should touch a state table, and nothing in the schema enforces it. A direct update against a row would leave state and history disagreeing with no alarm — the failure mode of a derived value that is wrong and announces nothing. The log is unwritable; the tables it describes are not.

What is measured, on what data

All deterministic, all scored against an oracle the code did not produce. Expected answers come from data/manifest.json, whose offsets were computed by a separate script and read back from the bytes; no metric asks the code under test what the right answer is. Nothing here calls a model or the network, which is a deliberate limit and is printed in the caveat rather than left for a reader to discover. Every figure in the Result column is marked, so this table cannot go on reporting a scorecard the harness has stopped printing.

No percentage is printed, and the mechanism is the interesting part. report.rate() raises below a sample of ten. It does not return "n/a" and it does not soften. The sample is derived from the identity of every row of evidence rather than declared by the caller, so a call site cannot inflate it: the citation metric re-reads 20 recorded offsets, most of which quote the same boilerplate sentence or a wording a revision left alone, and they resolve to 6 subjects. The scorecard prints 20 of 20 manifest offsets verify [n = 6 subjects over 20 recorded offsets; no rate, n < 10]. Every one of those offsets verifying is a real result and it is not that many samples — and the gap between them is the argument. Deflation a call site has to remember is deflation a call site will forget, and it will forget it in the direction that flatters. Precision cannot be computed at all: the manifest labels some changes, not every change, so the corpus carries no negative labels, and the run prints the shortfall — how many changes were reported across both pairs and how few of them carry a label. Every verdict is bound to the corpus digests printed at the head of the run: change the corpus and the verdicts expire.

Design

Every company-scoped table carries company_id from the start, even though the build has exactly one synthetic company. Reads go through one chokepoint — app/state/queries.py — which refuses a call with no scope rather than answering it. A query that returns everything when the caller meant nothing is how tenant isolation fails in practice, and it fails silently. The guard is imported by every module that needs it, never copied: two copies of a tenant check drift, and the copy nobody audited is the one that leaks.

Building the query layer as if a second tenant existed, while only one does, is cheap now and expensive later. Retrofitting isolation after the schema exists is the mistake this convention is written against.

Two tables carry no company_id on purpose: permissions and role_permissions, which are vocabulary rather than tenant data — the codes the product defines and the grid mapping them onto role names. proceedings are company-scoped: Proceeding.company_id exists and every read filters on it. And passages carries no company_id either, deliberately: tenancy sits on the version that owns it, and the join to document_versions applies it, because adding the column would give two writers a chance to disagree about who owns a passage.

A read that resolves to another company's row returns None rather than raising, and the web layer answers 404 — the same answer an id that was never issued gets. Telling the two apart tells a caller which ids exist. Where a read joins to a parent, both company columns are filtered rather than one: a write bug that stamped the wrong company on a child row would otherwise hand one tenant another's rows, and the column meant to prevent that is the column that let it through.

What runs, and what is a named gap

Authentication is built: scrypt password hashing with the cost parameters stored per user so the cost can be raised without locking anybody out; session tokens held in the database as a SHA-256 of the bearer token, never the token itself; a middleware that refuses an anonymous request to everything except the login pages, the health check, the stylesheet and a share link; three system roles and the permission vocabulary beside them; and an approval gate that a permission alone cannot pass (§7). Every login, every failure, every denial and every waived approval lands in the same audit chain as the state changes.

Refusing at the middleware rather than per route is the load-bearing choice: a route added next week is protected because it exists, not because somebody remembered a decorator. A login refusal says one thing whatever went wrong — unknown address, wrong password, suspended, locked — because a message that distinguishes them turns the form into an account oracle. And a refusal costs the same as a success: the password check runs against a dummy hash even for an address with no account, so timing does not answer the question the message refused to.

The share prefix is in the public list on purpose, and it got there by way of a defect worth keeping in the document. /s/<token> was not public, so an anonymous open answered 303 to /login?next=%2Fs%2F<token> — which puts a live bearer token into a query string, where it reaches the access log, the Referer header and browser history. A redirect that carries the thing it was protecting is worse than no guard, because it looks like one working. PUBLIC_PREFIXES in app/web/deps.py holds the prefix with its trailing slash; a bare /s and /settings are still guarded, because prefix-matching the rest would make /login-as-somebody-else public by accident.

Where the key comes from, and the bug that made this worth a paragraph

app/config.py reads .env into os.environ once, at startup, in a short function of standard library — no new dependency (ADR-014). Most of what is in the file is docstring and comment rather than code. Two rules govern it. The real environment always wins: a name already set is never overwritten, because a container, a systemd unit or an export on the command line is a deliberate act and a file on disk is a default; read the other way round, a deployment could not override its own checked-out configuration and the surprise would land in production. It never logs a value: it reports which names it set, because the file holds an API key, an OAuth refresh token and a client secret, and a helpful startup line would put all three into every log aggregator the host ships to.

Before it existed, nothing put .env into the process. The key sat in the file the README told a reviewer to put it in and never reached os.environ, so the assistant, the proposer and the mail transport were all permanently on their fallback paths — a deployment holding a perfectly good key behaving exactly like one holding none. Every fallback fired, every message it printed was honest, and the product was still wrong. §14 carries that as a failure mode in its own right, and A8 carries where in the deploy the file is read.

44 tables, marked so a test re-derives it, because it is this section's premise rather than decoration: drawing all of them would teach nothing, so the diagram carries the spine — the seven a reader has to hold in order to follow every other section — with the keys that make each one work. The rest are named by cluster underneath.

The rest, by cluster

How a schema change reaches a database that already has rows

requirements.txt carries no migrations tool. No ordinary test can catch a missing migration, and the reason is the shape of a suite: tests build their schema from the current models every time, so they never meet yesterday's database. The failure exists only in production, only on the deploy, in front of whoever is looking. A test that guards it has to do what no other test in this repository does — construct yesterday's database on purpose — which is what tests/test_migrate.py does.

app/state/migrate.py is the fix, and it is derived rather than listed. Five ALTER TABLE lines would have closed the instance and left the class open: the next column anybody adds breaks the deploy the same way and they will not think of this file. So it asks SQLAlchemy what the models declare, asks the database what it has, and adds the difference. Stated as the SQL it emits, because "adds the difference" is a shape and a reader deserves the mechanism:

• A new table arrives through create_all. Base.metadata.create_all(engine) creates what is missing and leaves what exists untouched. That is also why the column loop below it has to exist: create_all will not add a column to a table it already sees.
• A new column arrives through a generated ALTER TABLE … ADD COLUMN. The loop walks Base.metadata.sorted_tables, compares each table's declared columns against inspect(engine).get_columns(), and for each missing one executes ALTER TABLE {table} ADD COLUMN {column} {type}. The name and the type come from the model, never from a caller, because a column name cannot be a bind parameter and DDL has to be built as text.
• A new index arrives through CREATE INDEX IF NOT EXISTS. Indexes on a table that already existed are not created by create_all either, so the last block emits SQLAlchemy's CreateIndex(index, if_not_exists=True) for every declared index the database does not report. An index the dialect refuses — a partial index it will not render — is recorded as a refusal rather than swallowed.

The audit-chain migration runs between the first and the second, and the order is the fix rather than a preference: run migrate_audit_schema after the generic column loop and the loop would add digest_version NULL first, the audit migration would then see the column present and never backfill it to 1, and verify_chain would refuse the whole log. app/state/migrate.py carries that reasoning in a comment beside the call. What the migration will not do is the design:

• It never drops, renames or retypes. Those are destructive, and this product keeps an append-only hash chain a rewrite would silently invalidate.
• It never backfills. A new column is NULL on old rows, because NULL says "the schema of the day did not record this" and a default says something the record cannot support.
• It refuses rather than guessing. A NOT NULL column with no default is reported as a refusal and scripts/migrate.py exits non-zero, so a deploy stops rather than starting against a schema it cannot read.

It is safe to run on every start, and it has to be: an idempotent migration nobody calls is the same as no migration. The second run reports that the schema is already current. A8 draws where in the deploy it runs, and why it runs before the seed rather than beside it.

One thing it deliberately does not migrate. The passage index (§15) is a derived corpus, and derived data half-migrated does not fail — it answers every query, plausibly, and wrongly. So the migration creates the index objects if they are absent and never backfills rows into them; building the index is its own step, and a read that finds the index missing or stale refuses to answer from it and says which path answered instead.

One real request, followed through the code: an analyst opens a change and sees which claims the product will make about it.

The load path, which is the other half

Ingesting a version runs through one chokepoint, app/pipeline.py, so exactly one place owns the order of operations. It ingests, diffs against the previous version, writes the Change rows and records an audit entry for each. Three properties are deliberate: it is idempotent, so a second run with the same bytes writes nothing — no second copy of the passages, no second set of changes, no second audit entry; it refuses a corpus that moved under it rather than rebuilding half of it; and it calls no model — app/pipeline.py passes materiality=None, which is why Change.materiality is NULL rather than defaulted. Change ids are derived from the two version ids and the position in the diff output, so a re-run reuses them; a UUID there would be correct and useless, because the second run would mint new ids for the same changes and double the row count.

The same chokepoint takes the real corpus. scripts/ingest_real.py hands it Kentucky PSC 2025-00113 — one witness's direct testimony, filed and then corrected — as two versions of 1,024,409 and 1,024,536 characters, a 127-character difference across a million. Every version carries the address it was fetched from, so a claim drawn from that docket links back to the Commission's own copy of the PDF, which is the one thing a regulatory reader wants from a citation: not our text, theirs. The URL goes through the same check the templates use, so a provenance file naming a javascript: address is refused at ingest rather than in a template.

Wrong-occurrence citations — the one that is closed

Regulatory documents repeat boilerplate, and a quote that matches at its offsets matches whichever occurrence the writer was reasoning about. A claim built on occurrence B while citing occurrence A has a real citation, real offsets, a matching quote — and is wrong. Verification would report green.

This is closed. verify_citation counts the occurrences of the normalized quote in the source, and where there is more than one the claim must state which it relied on or it is refused. The corpus is built around the trap: one sentence, three sections, three versions, and make eval asks three questions of every span it produces (27 of 27 above). The remaining honesty is about what the check costs: a claim over repeated text cannot be made at all unless the producer knows which occurrence it read, which pushes work back onto whatever writes claims.

The model misquotes, or quotes truthfully and misleads

Fabrication and shifted offsets are caught by the verifier (§5) at a bar of every probe on the corruption suite. The truncation case is caught by nothing. A real, correctly-offset substring that drops a governing qualifier — a clause quoted without the "unless" three words later — passes exact-match verification, and the adjacency scan that would catch it is not built. Citation verification proves a quote exists at a location. It does not prove the interpretation drawn from that quote is correct. Conflating those two guarantees would be the easiest way for this design to overstate what it delivers.

Wholesale restructuring

Sequence alignment assumes document order is stable enough to follow, and a final order that renumbers a draft's sections breaks that for many passages at once. The mitigation is real and partial: the section-label cap (§2) stops a renumbered pairing presenting itself as settled, the change screen says so, and a person can answer the doubt. What it does not do is raise an escalation, so a capped pairing is visible to whoever opens that change and to nobody else.

Retrieval — what can fail now, and what it is still not allowed to decide

Every unavailable state of the index falls back to a complete scan of every passage in scope, saying which path answered and why (§15); a short list with no announcement is the failure this refuses. The load-bearing choice underneath is that "did something change" is complete and deterministic; "what does it touch" is lossy — the diff, not retrieval, enumerates changes, so a retrieval miss degrades explanation and never change coverage. The index can hand the model the wrong passage or miss the right one; neither puts a wrong fact on screen, because a candidate is not evidence. What it can do is make an answer thinner than the record supports, and the thinness is announced by count.

The change-to-obligation edge has a producer, and it is lexical. app/state/mapping.py::propose_obligations_for_change proposes candidates from the words an obligation and a passage share; confirm_obligation_for_change lets a person stand behind one; map_change_to_obligation writes the row either way and mapped_by_kind records which. The gap that remains is the one the two wordings in §4 are about: the lexical rule cannot reach a duty that says "post security" where the docket says "post collateral", so a change whose only link is that word has no mapping at all and routes to nobody. ADR-085 carries it.

A fallback that announces itself correctly, about a state that is not true

This is the failure mode this repository actually produces, and no test goes red for it. Three modules degrade safely with no API key: the assistant says it is unavailable, the proposer proposes nothing and carries FALLBACK_NO_API_KEY, the mail transport declines to send and says so. That is the right design — best-practices §26, a fallback must announce itself. It is also what a wiring bug looks like from outside. With nothing loading .env, a deployment holding a good key behaved exactly like one holding none: every message printed was honest and every one was about a state that was not the state, and nothing failed, which is why nothing found it.

The guard has to be a test that asserts the connection — that the name the caller resolves is the name the callee publishes, that the router is mounted, that the environment a module reads is the environment something fills — not a test that the fallback works. Three such tests exist for the three places this shape has already appeared: app/config.py (§11), the router-mounting check in tests/test_app_wiring.py, and the engine-name seam in app/chat/engine.py.

Tables

A table flattened to text loses its row structure: a cell's meaning depends on the header above it, and a citation landing on a bare cell offset is not gradable without that header. Segmentation compounds it — one inserted row shifts every offset after it, and a diff over the flattened text can present a whole table as changed when one row moved. The intended fix is to segment a table as its own passage type, row tied to header, so a cell's offsets carry its header by construction. Not built. Tables are ordinary text and inherit every problem above.

Footnotes

Segmentation has no passage type for a footnote and no anchor tying one back to the clause it qualifies. A footnote in regulatory drafting often carries the exception that changes the obligation: the clause reads as absolute and the footnote is where "unless" lives. Losing the link means a duty can be read as unqualified when the source qualifies it a few lines down. Not built.

A verified fact with a shelf life

An edit landing on a cited span withholds the claim on the next render. An edit elsewhere in the same version does not, so a claim can go on asserting itself against a document that is no longer the one ingested. The hash check that would catch it exists in verify_citation_for_version and nothing on the run path calls it (§5). Closing it needs a single-row scoped read that does not exist yet, and a decision about what the interface should show for a source that has drifted — which is wider than one module.

At 10x latency, and at 10x scale

Every stage on the path a claim takes to the screen is bounded string work: ingestion, segmentation, diffing and verification call no model and make no network request, so none is latency-sensitive. Two surfaces are. The assistant makes a model call inside the request that renders the answer, with a step cap and nothing else — no timeout of our own, no queue, no background worker — so a slow endpoint is a slow page and a rate-limited one is a failed turn. The change screen's first view of an unjudged change does the same for the materiality verdict. That is acceptable for a chat surface, where a person is waiting anyway; it is less acceptable on the change screen, and the fix when that path carries load is background work, so a request returns "changes detected, interpreting" rather than hanging, and a call that times out leaves its item visibly needing interpretation rather than dropped.

What breaks first under load is SQLite (ADR-028), and A7 works through where it breaks. This section carries only the half that is a failure mode: the deployment runs one uvicorn worker on purpose, because a second buys concurrency the database cannot honour and turns a slow write into a locked one.

When the model is simply wrong

Not a hallucination and not a bad citation: a correct reading of a verified passage, reaching a judgement a domain expert would dispute. This is what confidence and escalation exist for, and the honest limit is that confidence is an imperfect proxy for correctness — a model can be wrong while reporting itself certain. Two parts of the mitigation stack are real today and neither involves a model: draft-versus-final is removed from judgement entirely and read from a field (ADR-005), closing off one whole class of wrong guess; and a person sits between a refusal and its resolution.

What is not solved. There is no mechanism that samples confident, non-escalated output and checks it against ground truth. A review queue only catches what was flagged; it cannot catch what was confidently wrong. At production scale that needs sampled human audit of confirmed output, not just of the queue. It is unresolved because closing it needs exactly the real-user, real-volume evidence a 48-hour build does not have — not because it was overlooked.

Three words a reader should not have to look up

An index is work done once at build time so a lookup is instant instead of linear in the corpus. The cost is that it is a copy of an arrangement of the text, and a copy can fall out of step with the thing it describes — which is the whole of the risk in this section.

FTS5 is that index, compiled into the SQLite this project already runs, so it is a table in the database the product already opens. No service to run, no process to supervise, no package in requirements.txt, nothing for a reviewer to install before make run works. That is ADR-007 paying off rather than a coincidence: choosing SQLite for the clone-and-run property bought a full-text engine for free, where a Postgres or Elasticsearch choice would have made this section a dependency negotiation.

bm25 is the ranking rule, and it is three ideas with no magic in them: a rare word counts for more than a common one, repetition counts with diminishing returns, and length is normalised so a paragraph is not out-ranked by a chapter that happens to contain the same word. SQLite reports it as a number where lower is better, which is its convention and not the one most people expect; the field is named rank rather than score for the same reason the product refuses to call anything a confidence unless it is one.

And the part that matters here: a ranked hit is a candidate, not an answer. bm25 says a passage uses the query's words often and other passages do not. It says nothing whatever about whether the passage supports a claim. Passages carry exact character offsets, so a hit arrives already in citation shape — and the verifier still checks the quote against the source bytes (§5). A top-ranked passage whose quote does not verify is still withheld. Ranking finds it; verification decides whether anything may be said about it. Every candidate handed to the model carries the key is_evidence: false, written out explicitly rather than left absent, because a missing key reads as "not applicable" and a present false reads as "asked and answered no".

The three things that make this riskier than it looks

One — tenancy

An FTS5 virtual table is its own object, and the guard in app/state/queries.py does not reach inside it. Passage carries no company_id at all — tenancy lives on the DocumentVersion that owns it — so an index queried on its own terms hands back every company's rowids. The read therefore lives in app/state/queries.py with every other tenant read, joins passages to document_versions, and filters on the caller's company. Nothing in app/state/search.py queries the index directly. The ledger table beside the index deliberately does not carry a company_id, though it easily could: a denormalised tenant column is a second answer to "who owns this row", and on the day the two disagree there is no way to say which is right (§10).

Two — the index is a derived corpus

Derived data is comparable only within the scheme that produced it, and half-migrated it does not fail — it answers every query, plausibly, and wrongly. So the index is not a migration and migrate.py never backfills into it. It has its own build step, it is written inside a single transaction so it lands whole or not at all, and a read that finds it missing or stale refuses to answer from it: three states, three named reasons, and every one of them degrades to a complete answer from a scan rather than a short one. Returning fewer results with no announcement is the worst available outcome, because the caller cannot tell "nothing matched" from "the index was empty".

Three — the tokenisation has to agree with the verifier

Regulatory text arrives from PDF extraction full of ligatures, soft hyphens, hyphenated line breaks, full-width digits and section marks. If the index folds them differently from the verifier, retrieval and citation stop describing the same document. The rule that makes "agree" precise: the index may be more permissive than the verifier and never less. A permissive index offers extra candidates and the gate rejects them, which costs a little work. A restrictive index cannot find a passage that exists and would have verified — a silent miss, invisible to everyone, and the failure this whole design is arranged against.

So both sides of the index run through normalize(), the same function that runs on both sides of every citation comparison, and FTS5 is left to do nothing but split words on already-folded text. All the folding is ours; the word splitting is symmetric. Measured on this machine against the raw text:

Three of the four shapes above are silent misses, all in the restrictive direction, all removed by normalizing first. In the other direction the index keeps what the verifier keeps: a superscript two is a footnote marker, normalize() refuses to fold it, and a query for "202" does not reach "20²". A blanket NFKC index would have joined them (§5). Two differences remain and both are permissive, which is the safe side: FTS5 case-folds where normalize() deliberately does not, and the tokeniser splits on stops and section marks, so § 5.4.1 is searched as a phrase — the tokens 5, 4 and 1 adjacent — rather than as three separate numbers that would match every table reference in the corpus. The scheme is named in SCHEME and recorded on every indexed row, so an index built under an older scheme is refused rather than mixed with a newer one.

Alternatives considered

Trade-off accepted. The index can be wrong in one direction only — it can offer a passage that does not matter, or fail to offer one that does — and neither can put an unverified sentence on the screen. What it costs is a build step somebody has to run, a scheme string two files have to agree on, and one more piece of derived data to keep honest. What it buys is the thing the citation story needs on a corpus bigger than a page: a way to find the passage worth citing before deciding whether it may be cited.

Where the wiring is. The assistant's search_claims tool calls search_passages, the migration calls ensure_passage_index, and app/state/queries.py holds both index reads beside every other tenant read, and .venv/bin/python -m pytest tests/test_search.py -q is the file that covers it. What is not true and would be easy to write: that ADR-008 is implemented. Its docket-text half is; its company-context half is not.