Uscha: A Fact-Gated, Tool-Agnostic Methodology for
Specification-Driven Development with LLM Coding Agents

1  Introduction

Two recent empirical results frame the problem this methodology addresses. First, when task instructions are underspecified, production coding agents do not fail closed: they act on unstated assumptions, and a majority of runs violate at least one action boundary [1]. Second, when a verification oracle is visible to the agent, near-perfect scores can coexist with an undelivered artifact: the agent builds to the test, optimizing the check rather than the request, and does not on its own validate what it ships as a user would [2]. Both failures are dispositional rather than capability failures, and both worsen as agents are given more autonomy.

Existing agent development frameworks converge on persistent artifacts and human review as mitigations [3], but they largely encode their process as prose instructions that the agent may ignore, and their verification signals rarely distinguish between what is mechanically verifiable and what is a model's opinion. Uscha (“Uscha” is the name of the methodology; the pattern it operationalizes is a spec-loop — a specification-driven build-and-verify loop. The name was chosen partly to avoid collision with spec-loop [18], an independently developed, actively maintained open-source project that shares the term and the spec-driven, review-per-increment philosophy while taking a different implementation path (shell-based reusable agent skills, task-local specifications); it was found late in this work's development, and the two are convergent rather than derivative) starts from that distinction and makes it the organizing principle of the whole process. Figure 1 shows the resulting pipeline: eight stations from idea to human gate, under a constitution of invariants that no station may violate, with an inner convergence loop over the build and an outer proposal loop that returns to discovery — while the merge remains human.

A useful way to see the resulting shape is as a choice of control flow. The default agentic pattern is an open loop — investigate, edit, test, and then let the agent answer “is it resolved?” itself — and that self-assessment is exactly the signal the two results above show to be unreliable. Uscha instead runs the same work through a measured graph: the steps become defined stations whose transitions are gated by computational evidence in the ledger rather than by the agent's own verdict. The loop is not removed but bounded — it lives inside one station (build–verify convergence) that either converges or escalates to a human when it stalls, instead of iterating on the model's judgment of its own doneness. What the practitioner authors is not the topology, which is fixed, but the guardrails the graph may not cross: the specification, the acceptance oracle, and the constitution of invariants.

The methodology is built on one discipline and three architectural commitments:

Facts block; guesses advise.

Every check in the process is classified as computational (it reads an artifact — a byte comparison, a parsed test report, a diff structure) or inferential (it judges prose or code quality). Computational checks are allowed to block progress mechanically. Inferential checks may only advise, unless a human explicitly declares them gating. A natural-language heuristic that blocks produces false positives, gets disabled, and dies; keeping guesses advisory is what keeps them alive.

Measured beats narrated.

The evidence ledger records two tiers: measured records parsed from real artifacts (test reports, coverage files, linter output) and self-reported agent counts. A measured red always overrides a narrated green. Acceptance criteria close per criterion only when a green, name-tagged test case exists in the ingested reports — a checked checkbox without a test is recorded as narrated-only and does not close.

The human at three fixed points.

The human approves the shape (specification front), approves the golden baseline (the one artifact the agent is mechanically forbidden from authoring), and merges. The agent proposes, measures, and stops.

Tool-agnosticism by contract.

The engine is dependency-free Python that never runs an LLM: it validates structures and ingests reports. Wherever a model's judgment participates (e.g. rubric grading), the interface is a JSON contract; who produced the JSON — one vendor's agent, another's, or a human — is irrelevant to the ledger.

This paper describes the methodology (§3), its evidence engine (§4), the reference implementation (§5), and the design-validation evidence accumulated so far (§6), followed by limitations (§7) and future work (§8).

2  Related Work

Underspecification. Ji et al. [1] show that underspecified operational instructions cause agents to guess rather than stop, violating action boundaries in 55.8–67.8% of runs. Uscha's greenfield front is a direct response: the specification is produced through a structured interrogation in which the agent proposes and the human decides, and inviolable constraints are captured in a constitution file whose breach is a first-class blocker, never a trade-off.

Building to the test. Ma et al. [2] demonstrate that agents deliver what is checked, not what is requested, and name the missing disposition validation self-awareness. Uscha treats this as its central adversary: acceptance closes per criterion on measured test evidence; findings cannot be closed without new test material (find bugs once); qualitative verdicts require file-and-line evidence or they do not count; and the workflow state that authorizes a pull request is derived from ledger facts, never self-declared by the agent.

Process taxonomies. de Macedo [3] surveys agent development frameworks along six dimensions (specification, context, roles, execution, validation, portability) and identifies recurring risks: specification drift, artifact fragility, and platform dependence. Uscha addresses each explicitly: drift through a rebuild test that scores whether the specification package alone can regenerate the system; fragility through an integrity-checksummed ledger; and platform dependence through the contract architecture and a dependency-free engine.

Field-level evidence and the autonomy trajectory. A systematic review of agentic AI across the software development life cycle [16] finds agentic support concentrated downstream — implementation, testing, maintenance — while the upstream phases (requirements, specification, design) remain the least mature, and it identifies output verifiability as the primary enabler of adoption. Uscha is a direct wager on that gap: it invests its heaviest machinery precisely upstream, where the review reports support is thinnest, and makes verifiability — not autonomy — the axis it optimizes. The framing is deliberately narrow: the review establishes that the upstream, verifiability-first niche is real and underserved; it does not, and cannot, measure Uscha itself, whose own evidence remains design validation and self-application (§6). The complementary direction is the autonomy-first multi-agent framework, of which ALMAS [17] is representative: specialized agents aligned with agile roles drive the loop toward less human involvement. Uscha takes the opposite stance on the same problem — the human stays at three fixed decision points (shape, golden, merge), and the reviewer's verdict is a recorded ledger fact backed by file-and-line evidence, never one more agent's self-report.

Practitioner doctrine. The methodology's check classification adapts the computational-versus-inferential distinction from Böckeler's sensor framework for coding agents [11, 6] — the axis on which “facts block, guesses advise” rests. Its gate-integrity stance — that a change must not weaken the apparatus that measures it, and that a high-blast-radius change needs a checker uncorrelated with the maker — follows the agentic-code-review red flags catalogued by Osmani [12]. Architecture decision records follow Nygard [5], extended with machine-checkable implementation plans. Golden-master capture follows approval-testing practice [7], with one addition central to the method: the agent may author the capture harness but is mechanically forbidden — by a pre-tool-use hook, not by instruction — from writing an approved fixture. The simplicity gate operationalizes the “Reduce” law of Maeda [13]. The methodology was also systematically cross-examined against classical doctrine [4]; §6 reports the ten resulting improvements. Two agent-native failure modes complete the lineage: the premature “done” token — Huntley's Ralph Wiggum loop [14] — is exactly what the derived state machine and objective gates refuse to accept, and the maker-checker separation the review loop enforces is the evaluator-optimizer pattern [15]. The design stance throughout is an application of Goodhart's law [8] to agent verification: any signal the agent can see, it will eventually optimize.

3  The Methodology

3.1  Two specification fronts

Greenfield (discovery). The human brings an idea; the agent brings the shape. The front proceeds one question at a time, each question carrying the agent's recommended answer, walking a fixed agenda: purpose, domain model, operation surface, architecture options with trade-offs, dirty cases and failure behavior, inviolable constraints, out-of-scope, acceptance criteria with stable traceable identifiers, a declared quality bar, and residual risks. High-uncertainty risks trigger a time-boxed spike whose only legitimate output is a decision record with lessons — spike branches are mechanically refused at the pull-request gate. The front emits a versioned specification package: context glossary, constitution, domain model, specification, decision records with implementation plans and verification checklists, acceptance file, risks, and handoff. Figure 2 shows how the package's artifacts feed the evidence engine and how the engine verifies the result across six layers of truth.

Brownfield (reverse discovery). For an existing system, the direction of trust inverts: observable behavior is ground truth, and the agent may produce only facts — a system map from static analysis and a golden suite captured mechanically at the boundaries. The agent never authors a specification of what the old system does or why: a specification written by the agent about legacy code encodes the same partial reading of the code that loses behavior silently, which is precisely the blind spot the golden exists to counter. The human writes the migration specification by reading the facts.

3.2  The development loop

The loop consumes the specification package and proceeds through phases, each guarded: plan (constitution first; no acceptance criteria, no build); a coverage gate that triggers boundary characterization tests when the safety net is insufficient; build under decision-record discipline (the agent consults records before touching governed areas and must propose — never silently take — new architectural decisions); a simplicity gate scoring the diff against declared budgets, with test code excluded so that writing tests is never penalized; a severity-gated review loop that converges instead of chasing zero (findings below the declared gate are deferred, not fixed forever); an integration pass across repositories; late test-writing against stabilized code; an optional rebuild test measuring specification completeness; and a pull request opened only when the derived workflow state says the facts allow it. The loop stops at merge: merging is human.

An escalation contract enumerates the situations in which the agent must stop and ask — iteration cap, oscillating findings, a previously-passing test now failing non-trivially, contradictory tool directives, decisions of architectural weight, and any constitution breach. Escalations and their closures are recorded events; an open escalation caps the readiness score until a human resolves it, and resolving a blocker requires an escape analysis: which gate or test should have caught this, and what was done about it.

3.3  A worked example in ten steps

Figure 3 walks a deliberately small feature — a discount calculation — through the whole methodology, because the small case makes the failure modes legible. Six of the ten steps are failures or refinements, and that is the point: each deviation is caught by a specific gate rather than by hope. Two advisory passes tighten the specification before any code exists (a vague criterion becomes a testable oracle; a missing out-of-scope section is added). During the build, mutation testing exposes a test that runs but asserts nothing (coverage was lying); the gate-integrity check catches and blocks the agent lowering a threshold to pass; the rebuild test diverges on an implicit input case, and the response is to amend the specification, never the code; the simplicity gate cuts a speculative abstraction. Only then does the readiness score — capped by facts, dominated by measured acceptance — reach the human gate, where a person reads the diff and approves the merge.

4  The Evidence Engine

The engine is a single dependency-free Python file. It never runs an LLM and never runs the tools it scores: it ingests their reports (JUnit-family XML, Cobertura and lcov coverage, nine linter formats across nine language stacks) and validates structures. Every verdict is persisted in a deterministic ledger with an integrity checksum; external mutation or a truncated write blocks loading with a recovery message. Table 1 summarizes the complete check family and the verdict class of each.

Fact gates (block). The blocking gates read artifacts, not opinions. Two design choices deserve emphasis. First, the golden gate's masking of volatile fields (timestamps, request identifiers) is itself governed: masking rules are declared in a versioned file that the human approves together with the fixtures, every masked match is reported separately, and any later edit to the rules file is flagged — masking is never invisible. Second, the workflow state machine is derived: the state (plan, build, qa, escalated, pr-ready) is computed from ledger facts. A self-declared state machine would be narrated state — the exact category of signal the methodology distrusts — so there is nothing to declare and no illegal transition to police.

Structured guesses (advise). The advisory layer includes regression capture (closing findings without adding a single non-blank test line is flagged as narrated closure; the failing test goes before the fix); plateau and stop-signal advisories over the finding history (findings flat or rising across three complete cycles means more iteration is not approaching the solution — return to the decision records; everything converged with zero blocking facts means cut and ship); and the rubric layer: a versioned file of weighted qualitative criteria — conventions, error-handling sanity, interface ergonomics, documentation quality — with anchor examples, negative criteria, and a threshold, graded by any runner against a JSON contract with evidence-or-nothing semantics: a verdict that affects the score without a file-and-line citation does not count, duplicate verdicts for one criterion break the contract, and unevaluated criteria count as failed (unevaluated is not approved). The advisory layer also carries reuse detection: a deterministic Type-1/Type-2 clone check of the diff against the existing repository. This targets the duplication that a diff-local simplicity gate is blind to by construction — the muda most characteristic of generated code [9, 10]. It too reports a fact (a byte-identical window already exists at file:line) while leaving the verdict “wasteful” advisory, because a normalized-line proxy has honest false positives (boilerplate, data-transfer objects, embedded SQL) and a check that blocks on those would be disabled and die.

Readiness and the anti-ceremony invariant. A weighted 0–100 score reports the state of the result, never effort (Table 2). The dominant dimension is measured acceptance — criteria closed by green, name-tagged tests — with checkbox completion demoted to a narrative dimension. Hard caps override the weighted average, and every biting threshold states its provenance. The score reports; it does not gate — the gates are the facts above. As the gate set grows, the dominant risk shifts from any one bad gate to the sum of good ones making the loop feel like an audit — over-processing, the ceremony form of waste. The methodology answers this reflexively with a meta-invariant every future gate must satisfy (it runs without human input, speaks only when it matters, collapses into readiness, and a trivial change skips it), mechanized as a single-verdict view: readiness collapses every persisted gate into one line, expandable on demand. This is the same Lean lens the method applies to code, turned on the method itself [9].

5  Reference Implementation

The reference implementation packages the methodology as seven agent skills (the two fronts, a precision interview for known features, the loop orchestrator, golden capture, a two-audience system-documentation generator, and the rubric grader adapter) plus the engine (24 subcommands). Three properties matter for portability. First, the skills are markdown instruction files readable by any instruction-following agent; the engine is invoked identically from any of them. Second, every LLM-judgment interface is a contract: the rubric grader ships as a neutral prompt usable by any vendor's agent, a raw API call, or a human filling the JSON by hand — the implementation's test suite exercises exactly that path, with no LLM in the loop. Third, the one mechanical prohibition (the agent must not write approved golden fixtures) is enforced by a pre-tool-use hook at the runtime boundary, not by instruction — during development of the implementation itself the hook blocked one of its author's own commands, which is the property working as designed. Table 3 maps the seven skills to the phase each serves.

The implementation carries its process in executable form. A smoke suite of 161 checks exercises the engine against a synthetic ledger — including, deliberately, the agnosticism claim: the rubric-layer tests fill the grader's JSON by hand, with no model in the loop. The engine also reports a passive Lean process metric — first-time yield, the fraction of repositories that cleared QA on the first cycle with no rework or escalation — derived from ledger facts and kept strictly informational, never a gate: it measures the process, not the state of the result. Version consistency across the five release artifacts (version file, configuration, changelog, plugin manifest, marketplace manifest) is itself a suite check, converting a release convention into a verifiable fact. An installation-diagnosis subcommand in the spirit of flutter doctor verifies interpreter, per-stack toolchains, hook presence and registration, skill integrity, and per-project configuration — and every failing check carries its remedy (an installation link or command), so the workflow is: run, install what is listed, re-run until green. Three installation modes are supported: per-project, per-user, and as a packaged plugin for one agent runtime — packaging, not dependency; the other runtimes install by copying files.

6  Design Validation and Self-Application

No controlled external evaluation has been conducted yet; we report the design-validation evidence honestly and completely.

Adversarial audits. Before stabilization, the methodology and its implementation were subjected to two adversarial audits totaling 231 agent evaluations: a seven-lens audit (54 agents; findings attacked by skeptics defaulting to refuted) confirmed 33 weaknesses and refuted 13, with the central finding that the core principle was inverted in the code — fact gates existed but were not wired to block. A second audit verified 171 claims across documentation, code, and design intent. The response re-wired every fact gate into the engine and re-passed all documentation against the implemented reality (a recorded truth-pass over the affected documents), converting the principle from slogan to enforced property.

Doctrine cross-examination. The methodology was systematically read against classical software-engineering doctrine [4], producing 8 validations, 12 tensions with recorded resolutions, and 10 actionable improvements — all ten implemented, each as a separate release with its own regression checks. The validations were often word-for-word: “don't assume it — prove it” is the ledger's two-tier contract; shared mutable resources including files motivated the ledger's integrity checksum; and the doctrine's warning against trusting enforcement to good intentions is exactly why the golden prohibition is a hook rather than an instruction. In the two deepest tensions the methodology deliberately departed from the letter of the doctrine to defend its own principle: readiness caps continue to bite by default (their existence is a definition; only the numbers are opinion, and they now state their provenance), and the workflow state machine is derived rather than declared.

Self-application. The implementation is developed under its own process. Across the eighteen releases of the self-application arc (versions 1.10 through 1.27), each engine change carried smoke checks in the same commit, a fresh-context review before merge, and the five-way version consistency check enforced by the suite. These reviews repeatedly caught real defects prior to merge — the changelogs record the resulting hardening across the series — including: test-file classifier gaps that made a documented guarantee an overclaim; a blank-line loophole in regression-capture evidence (any non-blank test line counts as evidence — a tripwire, not a judge — but a blank line must not); a review-suite check that passed for the wrong reason (its precondition never held, so the assertion was vacuous); silent last-wins semantics on duplicated grader verdicts — a grader gaming vector, now a broken-contract error; and, in the single-verdict release itself, documentation that under-enumerated what the collapsed view actually shows. We take the consistency of this defect stream as evidence for the methodology's core premise: independent, fresh-context checking against recorded criteria catches what the authoring context cannot see.

7  Limitations and Threats to Validity

No controlled evaluation. All evidence is design validation and self-application by the methodology's author; effect sizes on team productivity, defect escape rates, or comparison against the frameworks surveyed in [3] are unmeasured. Single-operator bias. Self-application shares authorship context; although reviews run in fresh contexts, they share the model family and the author's configuration. LLM non-determinism. The rubric layer's grades vary across runs; the methodology bounds the blast radius (advisory by default, evidence-or-nothing, human-declared gating) but does not eliminate variance. Scope. The loop targets a single operator driving one non-trivial change; multi-operator concurrency is out of scope, and the ledger is single-writer by design. Heuristic residue. Some gates carry documented approximations (path-based test classification, stale-report windows); each is disclosed at the point of use, but disclosure is not elimination.

8  Future Work

The central gap is controlled evaluation. Concretely, we plan a study that assigns matched non-trivial changes to operators with and without the methodology and measures three outcomes the ledger already instruments: defect-escape rate at human review, rework (first-time yield across cycles), and time-to-merge — against two baselines, ad-hoc agent use and the frameworks surveyed in [3] along their six taxonomy dimensions (with particular attention to the validation and portability axes). The missing pieces are operators other than the author and a controlled task set; the process instrumentation is already in place. Supporting steps: read-only dry runs of all nine stack adapters against real repositories; a multi-project dogfooding phase; and an evaluation of the rubric layer's inter-run variance under anchored versus unanchored criteria, connecting to the validation self-awareness agenda of [2]. A further direction extends the Lean lens beyond the inner loop: the reuse and anti-ceremony gates address two forms of waste (duplication and over-processing), but the “shine” discipline of 5S — scheduled cleanup that keeps the legacy layer from silently freezing — is a candidate outer-loop routine [9], deliberately out of scope here because it acts on the repository over time rather than on a single change under review.

9  Conclusion

Uscha's contribution is not a new agent architecture but a discipline for placing trust: facts block, guesses advise, the measured overrides the narrated, and the human decides at exactly three points. The methodology's own history — a central principle found inverted by adversarial audit, then wired into an engine that now blocks its own author when he crosses the line — suggests that for LLM-assisted development, the difference between a methodology and a slogan is whether the checks are executable. The reference implementation will be released as open source (MIT).

References