2026-07-14 · Compliance

What a defensible AI agent audit trail actually contains

Ask a team whether their agent is auditable and the answer is usually yes, we log everything. Then ask three follow-up questions. Can you prove nobody edited a record from eighteen months ago? Can you say which version of the policy was live during a specific conversation? Are the logs still there after the retention window your log platform applies by default?

The answers are typically no, no, and no. That is not negligence; application logs were designed for debugging, where mutability is a feature and thirty days is generous. An audit trail is a different artefact with different properties, and the gap becomes expensive at exactly the moment you cannot close it retroactively.

The expectation has also tightened. Per-record trails capturing the input, the version in force, the output, and the trigger have become the standard in regulated sectors, replacing aggregate statistics as adequate evidence.

The seven fields #

For every step an agent takes, not every conversation, a record needs:

What came in. The customer message or the trigger that started the step, with the session and, where relevant, a stable customer reference.
What the system did. The step type: a model call, a tool call, a guardrail evaluation, a handover. Naming the tool and the model matters; a conversation that called create_payment_link is materially different from one that only read a balance.
Which rules were applied and what they decided. Including the ones that passed. A record showing that eleven guardrails evaluated and one blocked is evidence; a record that only mentions blocks tells you nothing about coverage.
What went out. The reply as delivered, after any post-rules modified or blocked it.
Under which version. A hash of the agent definition, the instructions, and the policy in force at that moment.
When, precisely and in a stable timezone. Contact-window disputes turn entirely on this.
Its position in the chain. The hash of the previous entry, which is what makes the whole thing tamper-evident.

Note what is not on the list: model confidence scores, token counts, latency. Useful for operations, irrelevant to whether the record is defensible.

Hash chaining is the cheap part #

Each entry includes the hash of the one before it. Change an old entry and every subsequent hash is wrong, so tampering is detectable by recomputing the chain rather than by trusting anyone's access controls. The head hash is a single value that commits to the entire history, so you can record it externally, in a separate system or a periodic report, and later prove the whole history was already fixed at that time.

This is a well-understood construction and it costs a hash per write. What it buys is a change of claim: instead of "our access controls make edits unlikely" you can say "an edit would be detectable, and here is the verification". Those are different sentences in a supervisory conversation.

What it does not do is prevent deletion of the tail, or prove an entry was written when it claims. Anchoring the head hash somewhere you do not control closes the second gap; ordinary backups and append-only storage handle the first. Verify continuously rather than on request: zolva scorecard audit.db checks the chain and prints the operating picture, and running it on a schedule means a break surfaces in hours rather than during an audit.

The which-version question #

This is the field teams most often lack, and the one reviewers most reliably ask about. A conversation from March is being reviewed. The agent said something the current policy would have blocked. Did the policy not cover it then, or did the guardrail fail?

Without a config hash in the record, that is unanswerable, and unanswerable tends to be read unfavourably. With one, you resolve the hash to the exact revision in version control and read the policy that was actually live. It also makes a subtler question answerable: when did behaviour change, and which change caused it, which is the difference between a one-paragraph explanation and a week of archaeology.

This works because the agent is declared as data. YAML plus Markdown plus a policy file hashes to a stable value and resolves back to a commit. Behaviour spread across a Python graph, a prompt template, and a few environment variables does not.

Storage and retention #

The audit store should be pluggable, because a pilot and a production deployment have genuinely different needs. SQLite is right for a pilot: no infrastructure, a file you can copy, fine for a single process. Postgres is right once you have concurrent writers, real retention obligations, and a backup regime that a supervisor may ask about.

Two things to decide early rather than late. Retention, which for financial services is usually measured in years and may outlast the agent, the vendor, and the project team, so it belongs on infrastructure you control. And residency, because if the store is inside your perimeter, the answer to where customer conversations live is your existing data policy rather than a new one. The audit playbook covers both backends.

Redaction versus evidence #

These pull in opposite directions and the resolution is to apply them at different points. Redaction exists so PII does not reach a third-party model. Evidence exists so you can show what actually happened. Mask everything in the audit log and it stops being evidence; mask nothing before the model call and you have exported customer data to a vendor.

Mask on the way out to the provider, not on the way into the record. Card numbers, emails, phone numbers, and your own identifier formats are redacted before any text reaches the model, while sessions, the audit chain, and human handover keep the true transcript inside your perimeter, where it is subject to your access controls rather than someone else's. Data exports mask again, because a fine-tuning dataset or an analytics extract is a different trust boundary from the audit store. The redaction playbook has the mechanics.

Mapping to obligations #

The same artefacts answer several regimes at once, which is the practical argument for building them once and properly:

Record-keeping. The hash-chained log of every step, retained on your own infrastructure.
A traceable system definition. Config hashes tying each conversation to an exact revision of the agent, instructions, and policy.
Human oversight. Handover events recorded as fact, including the resolution the human gave, rather than asserted in a policy document.
Accuracy and ongoing monitoring. Eval gate results and the scorecard over time, which is what SR 11-7 has always meant by ongoing monitoring.

terminal

zolva compliance audit.sqlite --agents agents/ --eval-report eval.json --out pack.json --gate

That reads the artefacts back and emits one signed bundle mapped to named articles, with the chain head hash as the tamper-evidence anchor and bundle_sha256 self-sealing the report. --gate exits non-zero unless every control passes, so a missing control is a build failure rather than an audit finding. It is packaging of evidence you already produce, not a compliance guarantee; your own mapping and sign-off decide adequacy.

← All posts