Audacia recently featured in a Sky News report asking whether AI agents need safety regulation, in the same way cars, medicine and nuclear power do.
Sky's Science and Technology Editor Tom Clarke visited our Leeds office to speak with Managing Director Philip White about the risks of AI agents, the controls we build into the agents we deliver for enterprise clients and what government regulation of AI might look like.
The biggest AI risk may be accidental
Much of the conversation about AI safety focuses on bad actors: people deliberately using AI to cause harm. Philip White argued that a bigger, less-discussed risk comes from good intentions.
"One of the scariest things that probably isn't necessarily getting enough attention is the big bucket of accidental bad actors. Good actors that are over-empowering models with access to data, access to infrastructure without realising the potential harm they can do."
As frontier models and AI coding tools become available to everyone, almost anyone can build an AI agent, and not everyone building one is thinking about the guardrails.
What happens when you restrict an AI agent?
To show why guardrails need to be designed at the system level, Principal Software Engineer Edward Carr ran a live demonstration for Sky. He removed one agent's ability to access the internet but allowed it to communicate with a second agent that still had internet access. He then asked the first agent for the latest Sky News headlines.
The first agent worked out that the quickest route to the answer was to ask the second agent.
No rules were broken and nothing malicious happened, but the demo shows clearly that restricting one agent doesn't restrict the system it belongs to. When agents can talk to each other, access controls have to account for every path to a capability, not just the obvious one.
How we build guardrails into AI agents
Audacia builds, runs and supports agentic AI systems for large-scale enterprise and public sector organisations operating in complex environments. Lead Data Scientist, Chris Bentley showed Sky a working demo of an AI agent built for a client.
"When we're building these sorts of tools for the client, we take away some of that autonomous capability that it has. So, the agent can only do certain things. It can only access tools that we give it."
Additionally, every AI system we build and use is governed by six principles, set out in our AI policy and independently assessed through our ISO 42001 certification:
- Accountability: Every AI system has clear human ownership, and anyone using AI is responsible for checking its outputs.
- Fairness: We test and monitor for bias and take corrective action when unfair outcomes appear.
- Maintainability: We plan for ongoing performance monitoring, cost management and revalidation long after launch.
- Privacy: We build privacy in from the start and limit AI systems to only the data they need. Where decisions significantly affect people, there's human oversight and a route to challenge them.
- Security: Our security assessments cover AI-specific threats like prompt injection and data poisoning, alongside our ISO 27001 controls.
- Transparency: We're clear about where AI is being used, what an AI system can and can't do, what data it uses, and its potential for mistakes.
What could AI regulation look like?
Tom Clarke compared AI regulation to cars. There's one set of rules for manufacturers, covering brakes, seat belts and airbags, and another for drivers, such as speed limits and drink-driving laws.
For AI, the manufacturers are the big AI labs building frontier models. The drivers are the businesses and individuals deploying AI agents out in the world. Both may need their own rules, and both depend on AI labs letting regulators properly examine their models.
Putting our own AI governance to the test
While regulation is still being debated, we haven't waited for it. Earlier this year we achieved ISO 42001, the world's first international standard for AI management systems.
Our audit covered the whole business, not just software delivery. It included how teams like HR and marketing use AI in their day-to-day work. That means the way we govern AI, including the agents we build for clients, is independently assessed.
Thinking about deploying AI agents?
We help enterprise organisations design, build and govern AI agents that deliver real value without taking unnecessary risks. Speak to our team about where AI agents could work in your organisation, or find out more about our agentic AI services.
