Four assurance properties, and four artifacts every enterprise already runs on which the corresponding controls must be built.
That phrasing is deliberate, and it is the distinction the rest of this readout depends on. A semantic layer, an eval suite, an access model, and a set of logs are control-bearing artifacts — they carry assurance load today. They are not yet implemented controls. A semantic layer with no change gate and no independent approver is not a control in the assurance sense, and an unsigned, incomplete trace does not become a provenance control because somebody possesses logs. What the enterprise owns is the substrate. What it does not yet own is the specification, the validation, and the accountability that would turn each substrate into a control.
Each of the four was specified against a different requirement, built by a different team, for a purpose that had nothing to do with certifying that an agent’s answer can be defended. Each now sits in the path of exactly that.
An artifact that performs control work without having been specified as a control has three predictable properties. Nobody gates changes to it. Nothing validates it independently. And no one is accountable for it in its new assurance capacity — not because governance functions are absent, since model risk, data governance, and IAM owners frequently exist, but because each of those owners is accountable for the artifact in its original capacity and considers that job done.
The four questions those artifacts are being asked to answer: does everyone get the same answer? Is the answer right? Was this person allowed to see it? Can you show how it was produced?
Any executive can ask these in a meeting. The first is usually the easiest to answer. Published control guidance already treats the underlying exposure as a named risk class, describing excessive agency as an agent determining that the best solution extends beyond its intended scope, and rating that exposure high where least privilege and permission boundaries are not established [01]. The gap it names is not a tooling gap. Products exist that address parts of it; no purchased product resolves accountability by itself.
Semantic consistency rides on the semantic layer
The artifact is dbt models, metric definitions, grain decisions, and the accumulated tribal knowledge about which refunds count and when the vertical mapping changed. It was specified for reporting coherence. Its job was to make the numbers in the Monday deck agree with the numbers in the quarterly.
It is now the thing determining whether a coding agent, a BI tool, a chat interface, and a coworker return the same value for the same question. That is a different job with a different failure mode, and the layer was never specified against it.
The failure it cannot detect is a definition that is consistently wrong. Suppose the definition of net revenue retention in a domain document excludes a class of downgrades it should include. Every interface reads that document. Every interface is faithful to it. Four surfaces, one number, complete agreement, wrong answer. A consistency check reads clean, because consistency is what it measures.
Disagreement between interfaces detects definitions that differ, not definitions that are wrong. Once every surface converges on the same definition, shared error becomes maximally invisible, and a consistency metric reading zero cannot distinguish a definition everything got right from one everything got wrong together.
What is missing is not a better semantic layer. It is a change gate on definitional values, and a separation between the person who authors a definition and the person who approves it.
Empirical validity rides on the eval suite
The artifact is evals, rubric scoring, and increasingly an LLM acting as judge. It was specified to catch regressions during development — did this change make things worse than last week. It is now being asked to certify that a production answer is correct.
The distinction that matters is between an instrument that returns the same answer and an instrument that returns the right one. A pre-registered program asked judges to predict the outcomes of hundreds of real experiments, then measured two things separately: how much the judges agreed with each other, and how much they agreed with what actually happened. They agreed strongly with each other and barely at all with reality. A sixteen-vote panel carried roughly two independent votes. The authors then ran the same test on fifteen human domain experts and found the same shape — the experts agreed with each other and scored at chance against real outcomes [05]. Their conclusion is the one this section needs: consensus, human or model, is reproducible, persuasive, and not evidence.
That result is not isolated. The largest systematic evaluation of LLM judges to date finds that the metric the field validates on does not correct for chance and overstates how well judges discriminate, to the point that judge rankings reorder substantially depending on the benchmark [03]. Another separates two properties usually conflated — whether a verdict holds steady under cosmetic edits, and whether it changes when the thing being judged actually changes — and finds judges strong on the first and weak on the second [04]. Judges agree with human experts mainly on questions they could have answered themselves, which aggregate scores conceal [06]. The standard the literature converges on is chance-corrected agreement, with instrument consistency and human alignment validated as separate properties [07][08].
What is missing is independent validation evidence for the instrument itself, in whatever form the domain supports — a gold set, an adjudicated baseline, a reported agreement figure against human labels. An unvalidated instrument cannot certify a control, and an eval suite scoring its own outputs is not validation.
Authorization correctness rides on the access model
The artifact is RBAC, OAuth scopes, row and column policy, and whatever masking sits in the warehouse. It was specified for humans and applications with stable identities making bounded requests. It is now governing an agent acting on a principal’s behalf, composing across systems, at a granularity nobody designed for.
The precision here carries the argument, because correctness and entitlement are easy to collapse and must not be. The underlying factual answer to a revenue-retention question is not role-dependent. What is role-dependent is the authorized representation: whether this principal may retrieve the value at all, at what granularity, with which rows and columns filtered, and under what masking or aggregation. So the invariant is: same semantic question, same interfaces, different principal, potentially different authorized representation. Where that representation must legitimately differ, a universal consistency target becomes a false assurance signal, because a clean consistency reading now certifies that policy was ignored uniformly.
The enforcement literature is specific about where the check must sit, and the figure above traces the placements against a single execution chain [14]. Detection after the action is not control. One line of work reframes the problem as authority binding, observing that injection becomes dangerous not when untrusted content appears in context but when it determines an authority-bearing argument, and moves enforcement inside the tool call so external data may fill a content field while being blocked from a destination or command field [09]. A network-operations benchmark reaches the same conclusion from the operations side, concluding that agents need execution-time authorization boundaries alongside prompt-level instruction hygiene [10]. Its transferable design point: artifact text may describe an approval, but it does not create one. Where a single authority exists the invariant is clean — a platform’s own MCP governance documentation states that users and apps cannot use the server to reach more resources than they could reach through the API [02]. A federated context layer spanning warehouse, BI, transcripts, and documents has no equivalent single authority, which is the gap.
The obvious shortcut is to let the agent derive its own scope. Measured against a mechanical solver across ten real applications, LLM-inferred least privilege reaches only 70 to 83 percent optimality with proprietary models and considerably lower with open-weight ones, degrading further as permission hierarchies deepen and context lengthens; the same authors report six overprivileged configurations among live commercial connector deployments [11]. Those are the authors’ observations in a preprint whose acknowledged funding includes several of the vendors examined, and they are cited as such. Bounded to the evaluated conditions, the finding is sufficient: agent-derived scope cannot be assumed to converge on the minimum.
What is missing is a test that the authorized representation actually differs by principal where policy says it must, and enforcement positioned where the action becomes effective rather than where the request arrives.
Provenance integrity rides on the logs
The artifact is audit logs, traces, and run records. It was specified for debugging and incident response — something an engineer reads at 2 a.m. to find out what broke. It is now the evidence that a control operated.
A record built for engineers is not a record built for an auditor. Completeness was never a requirement, retention was set by storage cost, and nothing in the pipeline is signed. The question a trace must now answer is whether the definitions, context, tools, identity, policy, and execution path that produced a result can be demonstrated after the fact, to someone who was not there and does not trust you.
There is a second requirement inside the first, and it is what makes provenance a control rather than a filing obligation. The artifacts you must preserve as evidence — domain documents, tool descriptions, skill files, retrieval instructions — are themselves causal inputs into execution. Classifying their origin and authority is part of the control, not an adjacent security concern.
Those artifacts are also the effective injection channel. Evaluation across six benchmarks and five backbones finds web and local content injection largely ineffective against safety-aligned models, while skill files and tool-description metadata remain highly effective, because instructions embedded in a document’s intended semantics do not present as abrupt external content [12]. The model vendors state the same exposure from their side: an audited connector is not the same as audited data, since a connector can load a poisoned README straight into context despite passing malware checks [13]. An artifact that is legible, mandatorily retrieved, and unsigned is an input to execution whose provenance is often not recorded as part of the execution evidence.
What is missing is a record specified for a reader who was not present and does not trust you: complete by requirement rather than by accident, retained on a schedule set by obligation rather than storage cost, and covering the provenance of the context artifacts that steered the run, not only the run itself.
The challenge function
Everything specified so far is first-line work. Definitional gating, an eval suite, enforcement at the execution boundary, a signed and complete trace — the building function builds and operates these, and building them is architecture.
Independent challenge is a separate activity with a separate output. The first line builds and operates the control. Independent challenge establishes whether that control can support the assurance claim being made on it. Those are different questions answered by different parties, and the second cannot be answered by the first.
Each section above named what is missing for its own property. In every case, a better artifact alone is insufficient. What is also missing is a second party.
None of the four can be self-certified by the party that produced it, and at present the party that produced it is usually the only party looking. A data organization can build all of this and still employ nobody whose job is to disagree with it.
The chain runs artifact, assurance property, first-line control, independent challenge, named owner. Enterprises reliably own the first link in all four cases. The chain past that point is frequently incomplete, and where it breaks, it breaks because nobody was assigned the artifact in its new assurance capacity.
Treating a control-bearing artifact as architecture is what starts turning it into a control — specifying it, gating it, validating it, and assigning it. That is not a discovery. It is an assignment somebody has to make on purpose.
Run it against one metric
This is testable in an afternoon and it does not require a project. Pick one metric your executives read weekly. Ask the four questions against that single metric, and require a named person for each answer rather than a team.
Who owns the definition, and who approved the last change to it. What independent evidence establishes that the value is correct, and who produced that evidence. Whether the authorized representation differs by principal, what enforces the difference, and where that enforcement sits relative to the point of execution. Whether you could reconstruct afterward how the number was produced, and whether anyone outside the producing team could verify the reconstruction.
The first question is usually answered immediately and the rest stall. The stall is the finding, and it tells you where to inspect rather than what you will find there. Some of those gaps are ownership gaps, and assignment closes them at the cost of one decision. Others turn out to be implementation gaps: once an owner is named, they look for the control activity and the evidence, and discover that neither exists. Both outcomes are worth an afternoon. Only one of them is cheap.
