Human oversight of AI agents that actually works
"Human in the loop" is easy to promise. How to design approval of AI agent actions that staff can sustain and auditors can trust.
Why every AI agent needs a control layer
The models are good enough. What decides whether an AI agent reaches production in a regulated organisation is everything around the model: who it acts for, what it may touch, who approves its work, and what you can prove afterwards.
Paul JonesFounder and CTO · · 4 min readMost conversations about AI agents start with capability. Can it answer the phone? Can it draft the case note? Can it triage the ticket? For a growing list of tasks the answer is yes, and it gets better every quarter.
In regulated organisations, that is not the question that stops projects. Gartner expects more than 40% of agentic AI projects to be cancelled by the end of 2027, citing rising costs, unclear value and weak risk controls. In our experience the last of these is the one that kills projects late, after the demo has impressed everyone and the assurance review begins.
Agents are a new kind of actor
A chatbot answers questions. An agent takes steps: it reads a record, drafts an update, routes a call, assigns a ticket. That makes it closer to a member of staff than to a search box, and it raises the same questions you would ask about a new starter:
- Identity. Who is this agent acting for, and in which organisation?
- Permission. What is it allowed to see and change, and for what purpose?
- Supervision. Which of its actions need a person to check them first?
- Evidence. If something goes wrong, can you show exactly what happened, in order, and that nobody has altered the record since?
Answer these separately for each agent and you get a different control story for every tool. Your CISO, SIRO and clinical safety officer then have to assess each one from scratch. That does not scale.
What a control layer is
A control layer is one place that every agent action passes through on its way to your systems. In Reimea Meridian it has five parts.
| Part | What it does |
|---|---|
| Identity | People sign in with the customer's Microsoft Entra ID, and sign-in is limited to that tenant. Nobody sets the platform up except a named bootstrap administrator. |
| Policy | Editions, roles and entitlements decide which agents and modules each person can use. |
| AI gateway | Every model call goes through one gateway, which applies the customer's policy. On Azure, providers that would process data outside the UK are refused. |
| Human approval | Agents draft captures, suggestions and answers. A named person accepts, edits or rejects them before they take effect. |
| Audit | Each step is written to an append-only, hash-chained trail, with a copy in the customer's Log Analytics and eight-year retention by default. |
None of this is exotic. It is the same discipline banks and health systems already apply to people and to software releases. The difference is applying it to agents, consistently, before the first one goes live.
An agent that cannot show its working cannot be signed off. An agent that runs through a control layer shows its working by default.
Why one layer beats many
It concentrates assurance. Your security and governance teams assess the control layer once, in depth. Each new agent then inherits those controls, so its own review covers only what is genuinely new about it.
It keeps data where it should be. If every model call goes through one gateway, data residency is a policy you set in one place. You do not have to check it in every vendor's small print.
It makes evidence routine. When the audit trail is part of the path rather than an add-on, you can answer an incident review or subject access request from records, not recollection.
It lets you widen autonomy safely. Today most agents in regulated settings should draft, and people should decide. Over time some action types will earn approval by exception. A control layer lets you make that change per action type, with the evidence to justify it.
Deploying it in your own tenant
For health and care and financial services, where the control layer runs matters as much as what it does. Meridian is deployed single-tenant: each customer gets its own Azure resource group, managed identity, Key Vault, keys, storage and database, in UK South or UK West. Records are sealed with AES-256-GCM, and the keys stay in the customer's Key Vault.
What to ask any agent vendor
- Where does the agent run, and whose keys protect the data?
- Which model providers can it call, and can you restrict them by region?
- Which actions does it take without a person's approval, and can you change that per action?
- Is the audit trail tamper-evident, and can you export it to your own monitoring?
- What is built today, and what is roadmap? Who signs the safety case?
If the answers are vague, the agent is not ready for regulated work, however good the demo.
Where Reimea is today. The control layer described here is built into Meridian release 23. Clinical modules are released after clinical safety sign-off, and independent penetration testing and Cyber Essentials Plus are in progress. The full list is on our assurance section.