8 AI Agent Audit Requirements for Compliance Teams

What Counts as an AI Agent in a Compliance Workflow

A model scores a transaction and hands the score to a person. An agent, on the other hand, scores the transaction, decides the score clears the threshold, closes the alert and moves to the next one.

Three properties turn models into ai agent platforms worth governing separately:

• It plans across multiple steps rather than answering one question.
• It calls tools that change state, such as writing to a case management system, sending a notice or closing an item.
• It operates on a schedule or a trigger rather than only on a human request.

Part of the assessment is establishing which of the three properties a given product actually has, because a retrieval assistant marketed as an agent needs a much lighter governance wrapper than a system with write access to your issues register.

Requirement 1: A Written Scope of Authority

Before anything else, the institution needs a document stating what the agent may do without a human, what it may propose, and what it may never touch. Writing that boundary by autonomy level works best, with the evidence obligation alongside it.

Write the boundary per workflow, as the same agent might operate at recommend level on fair lending testing and at retrieve and draft level on policy authoring.

Two limits belong in every scope document regardless of level:

• A monetary or volume ceiling above which the agent stops and escalates
• A list of actions it may never take autonomously

Requirement 2: An Immutable Log of Actions

Audit logs for an agent need to record what changed in a system. A usable action record carries seven things:

1. The trigger that started the run
2. The inputs the agent received
3. The sources it retrieved
4. The tools it called
5. The state changes it made
6. The human who approved the action where approval applied
7. A timestamp on each step

Requirement 3: Source Traceability

Traceability means a link or reference from each material statement back to the specific source. Three tests separate real traceability from a citation-shaped feature:

• Ask the agent a question whose answer depends on a recent amendment and check whether the cited text reflects the amendment
• Ask a question the source material does not answer and see whether the system declines or fabricates a citation
• Follow three citations manually and confirm each says what the output claims

Requirement 4: Version Records for the Model, Prompt, and Knowledge Base

Three things can change underneath an agent:

1. The underlying model gets replaced or updated by the model provider.
2. The prompt or instruction set gets revised by the vendor.
3. The knowledge base gets refreshed with new source material.

Every logged action should carry the identifier of the model, the instruction set and the knowledge base snapshot in force when it ran. It should also be possible to reproduce a past run against its original configuration.

Requirement 5: Human Approval Gates

Design the human approval gate around consequence. This would include any actions that create an external record, affect a customer, change a policy in force, or close a regulatory obligation need a named approver.

When the agent encounters a situation outside its scope, low confidence, or conflicting sources, the correct behaviour is to stop and route to a person, with the reason recorded. Ask the vendor what triggers escalation and if thresholds are configurable.

Requirement 6: Ongoing Monitoring and Outcomes Analysis

Agents degrade for reasons like the model provider updating the model, the regulatory corpus shifting, or the input distribution changing as the business changes.

Ongoing monitoring and outcomes analysis are the SR 11-7 terms, and a vendor selling into banking should recognise both. Outcomes analysis compares what the agent produced against what turned out to be correct.

This is the same discipline applied in continuous controls monitoring, turned on the agent itself. Set the sampling rate by consequence. A high-autonomy agent closing alerts warrants a higher review rate than a drafting assistant, and the rate should rise after any material change to the model, the instruction set or the knowledge base.

Three signals warrant a defined threshold and an alert:

• Escalation rate, which indicates whether the agent is meeting situations outside its competence
• Override rate, which indicates whether reviewers trust the output
• Disagreement between the agent and a second reviewer on sampled items

Requirement 7: Data Handling, Residency, and Training Use

Compliance work runs on the institution’s most sensitive material. Three questions establish whether that material is adequately controlled:

1. Does the institution’s data train a model used by anyone else?
2. Where does the data sit and process, and which subprocessors touch it?
3. What is retained, for how long, and who inside the vendor can read it?

Some platforms address this by keeping the model inside the institution’s boundary. For example, Ask Kaia is a self-hosted, purpose-trained solution that never shares client data with commercial AI models, and that is trained exclusively on federal banking regulations and related regulatory data.

Requirement 8: Change Notification, Exit, and Evidence Portability

The agent vendor is a third party, and the institution’s existing third-party risk management programme applies. Three additions are specific to AI agents:

• Model provider disclosure
• Advance notice of material model or instruction changes
• Portability of the evidence record

Write the evidence retention and export obligation into the contract at signature.

Frequently Asked Questions

The output of the assessment by an agent belongs in a documented risk analysis because this is what an examiner will ask to see when the agent appears in a workflow they are reviewing. The related explainer on conducting an AI risk analysis covers how to structure it.