Using AI to improve efficiencies in handling complex medical enquiries

Client

The Medicines and Healthcare products Regulatory Agency is an executive agency of the Department of Health and Social Care in the United Kingdom which is responsible for ensuring that medicines and medical devices work and are acceptably safe. The MHRA has several independent advisory committees which provide the UK Government with information and guidance on the regulation of medicines and medical devices. The agency employs more than 1,200 people in London, York and South Mimms, Hertfordshire.

Background

As part of it's role, the MHRA publishes the British Pharmacopoeia (BP), a collection of pharmaceutical standards used globally by healthcare professionals and manufacturers. The BP team faced a steep rise in regulatory enquiries, ranging from simple clarifications to highly technical assessments. Incoming queries were managed through a shared mailbox, making it difficult to search previous responses or ensure consistency across the team. MHRA needed a way to handle a growing volume of complex enquiries more efficiently, without compromising the accuracy and regulatory precision their role demands.

Solution

Working with MHRA’s internal Digital Technology Group, Audacia designed and delivered the BP AI Assistant, a secure, staff-facing tool built around Retrieval-Augmented Generation (RAG). During the proof of concept, Audacia and MHRA evaluated several closed and open-source models together to select the most effective solution. RAG was chosen in part because of the complex data transformations needed to reference the source material accurately. The assistant combines a large language model with MHRA’s own authoritative datasets, including historical query-response pairs and BP documentation, to produce fully referenced draft replies for human review.

At the core of the system is a secure AI assistant grounded in validated query responses and BP standards. It interprets each enquiry, retrieves the most relevant BP content and generates a referenced draft reply that reflects the BP’s regulatory tone, with every statement traceable back to validated sources. Audacia built an ETL pipeline that takes the BP’s XML data and transforms it into a consistent, machine-readable format for RAG workflows, normalising data structures and metadata for reliable retrieval across monographs, annexes and cross-referenced sources. The same pipeline brings updated BP content into the knowledge base. Retrieval precision was enhanced through a hybrid approach combining vector-based semantic search with traditional lexical methods, preserving sensitivity to critical regulatory terminology that a purely semantic approach could miss.

Audacia supported the full development lifecycle, from early prototypes and co-design with BP users through to production deployment, refining prompts to mirror BP’s tone and regulatory language. The design and architecture passed through MHRA’s Solution Design Board, the agency’s governance body for new systems, and the solution was penetration tested before go-live. The assistant is hosted within MHRA’s own Azure tenancy, with access controlled through Microsoft Entra single sign-on. Prompts and data are excluded from model training by the LLM provider.

Before go-live, MHRA’s subject matter experts used the assistant and assessed its answers. The central check is grounding, with every statement having to trace back to a retrieved source or to the guidance provided. Any change to prompts, model version or the knowledge base is tested end to end through its effect on answers, with the same cases run before and after to show whether performance is maintained or improved. A built-in upvote and downvote feature captures user feedback on every response, creating a dataset to guide future refinement.

Results

The BP AI Assistant has delivered measurable improvements in both speed and consistency. Response times have improved significantly, with 95% of queries now answered within 18 working days and 80% within 10 days, regularly exceeding MHRA’s key performance indicators. Staff now spend less time searching archives and drafting repetitive correspondence, freeing up time for higher-value scientific and regulatory work.

The success of the project has been recognised internationally, with MHRA sharing the approach with regulators in Australia, Qatar and Vietnam, and it has become a blueprint for reuse across other MHRA teams handling external queries, including the Customer Experience Centre. The project was also shortlisted for AI Innovation of the Year at the National Technology Awards 2026.