AI governance under the EU AI Act: why evidence shape matters more than documentation volume

The EU AI Act’s high-risk obligations are in force. Supervisors will not ask how many pages you wrote about your model. They will ask whether a specific action was allowed under a stated rule, at a stated time, with a record that still exists. Volume of documentation is not evidence. Shape is.

What the Act actually asks of high-risk systems

For high-risk AI, the Act requires a quality-management system, logging, human oversight, and technical documentation that lets a competent authority understand how the system behaves. Article 12 is the logging obligation: automatic recording of events over the system’s lifetime, sufficient to identify risks, substantial modifications, and — where relevant — to facilitate post-market monitoring and supervision.

That sounds like “keep logs.” It is not. A log that records model outputs, prompt IDs, and confidence scores is a forensic artefact. It tells you what the model said. It does not tell you whether the action that followed was permitted. Agentic systems book trades, release payments, update client records, and change supplier status. The consequential event is the state transition, not the token stream.

Article 14 (human oversight) is equally easy to misread. A person clicking “approve” on a screen they cannot validate in the time given is not oversight as the Act conceives it. If the human cannot check the rule against the facts at decision time, the click is ceremony. The Act wants effective oversight. Effective oversight needs a checkable rule and a record of whether it held — before the action ran.

Documentation volume is the wrong optimisation

Most AI governance programmes still optimise for artefacts designed around outputs: model cards, bias reports, DPIAs, vendor questionnaires, and a folder of policies. Those documents answer “what did we intend the model to do, in general?” They do not answer “why was this payment released at 14:23 on 3 March?”

When an examiner asks that second question, teams reconstruct. They pull CRM exports, ticket comments, chat logs, and a week of engineering time. The reconstruction may even be accurate. It is still a narrative. Narratives can be disputed, edited, and assembled under pressure. Supervisors in financial services already distinguish between contemporaneous records and post-hoc stories. The AI Act’s logging and documentation duties will be read the same way: if the only proof you have was written after the letter arrived, you did not have a control. You had a writing exercise.

More documentation makes the reconstruction longer. It does not make it contemporaneous. A 90-page model card and a 40-page risk file can coexist with a payment that no rule actually gated. The gap is architectural, not editorial.

Evidence shape: five fields, at the moment of decision

Examination-ready evidence has a shape. For each consequential action you should be able to produce, without archaeology:

• What was decided — allow, deny, or escalate.
• Which rule applied — a versioned policy or invariant, not a slogan.
• What the system knew — the facts the rule operated on at that instant.
• Who approved — if a human was required, which role, under which delegation.
• Proof the rule held — a tamper-evident record, and where needed a cryptographic demonstration, created before execution.

That is the same structure as a regulatory evidence trail. It maps onto Article 12’s demand for automatic recording, and onto the practical question every competent authority will ask once agents act inside high-risk workflows: show me the decision, not the brochure.

Shape also constrains disclosure. A raw log dump can contain client data you are not allowed to hand over. A record that names the rule, the outcome, and a verifiable proof can be shown without exporting the underlying portfolio or clinical file. Documentation volume usually increases disclosure risk. Evidence shape can reduce it.

Where model-centric governance fails the Act

Governance frameworks built for generative outputs assume a human will act on a recommendation. The Act’s high-risk annex includes systems that influence credit, insurance, employment, and access to essential services — domains where software already executes. Agentic AI removes the remaining gap: the model does not stop at a paragraph. It calls an API.

If your control plane is “the model was tested on a sample” plus “a human reviewed a dashboard,” you have neither Article 12 logging of the action nor Article 14 oversight of the action. You have sampling and theatre. Sampling cannot prove a rule held for the case under examination. Theatre cannot prove the human could have known.

The correct split is the one we argue throughout this site: the AI proposes; a validator decides, deterministically, against a specification you can show. The Act does not name that architecture. It names outcomes — logging, oversight, risk management — that this architecture actually produces.

What to do before the next information request

Pick the highest-risk workflow where an agent (or an agent-assisted system) can change state: a payment, an onboarding decision, a credit variation, a clinical authorisation. Write the rule in a form that can be checked, not only read. Put the check upstream of execution. Emit the five-field record at that gate. Retain it in a hash-chained log.

Then try the examination question internally: “Why was this action allowed?” If the answer requires a ticket search, you are still in documentation-volume mode. If the answer is a record that existed at 14:23, you have started to meet the Act as it will be enforced — as a demand for evidence of control, not a demand for more PDFs.