Summary

• The DBS has registered its Barring Referral AI Summary Tool as a public beta, supporting a team of approximately 16 caseworkers handling around 1,200 monthly referrals.
• The AWS-hosted system produces source-linked PDF summaries using foundation models, but cannot make barring decisions.
• Testing involved approximately 140 production cases, with safeguards addressing inaccurate summaries, bias, sensitive information and excessive reliance on AI.

The Disclosure and Barring Service has disclosed details of an AI system that summarises safeguarding referrals for caseworkers, introducing generative AI into a sensitive public service where inaccurate information could influence decisions about who is permitted to work with vulnerable people.

The Barring Referral AI Summary Tool, published on the government’s Algorithmic Transparency Recording Standard register on 14 October, is classified as public beta. It produces structured PDF summaries of referral evidence, with citations intended to help employees locate and verify information in the original documents.

Approximately 16 triage caseworkers are expected to use the system, supporting a team that receives around 1,200 referrals each month. Its initial application is restricted to cases submitted through the digital Barring Referral Service portal.

DBS has explicitly excluded automated decision-making from the tool’s responsibilities. Caseworkers must continue examining the underlying evidence, gathering additional information where necessary and making their own decisions through the established barring process.

The published record provides unusually detailed information about the technology’s architecture, model selection, testing and safeguards, offering a view of how generative AI is being introduced into government operations beyond the more familiar applications involving public enquiries or administrative correspondence.

Evidence remains the basis of every decision

Safeguarding referrals can include extensive collections of statements, forms, correspondence and other documents. Caseworkers must identify relevant facts, establish the chronology of events and determine what additional evidence is required before completing an assessment.

Previously, employees navigated this material manually across several systems. The new tool prepares a summary when an eligible case is opened, identifying information such as dates, addresses and relevant criteria, with references back to supporting documents.

The summary is intended to reduce the time needed to become familiar with a case while allowing staff to concentrate on assessing evidence and resolving outstanding questions.

However, producing a plausible account of documents is different from establishing that the account is complete and accurate. Generative AI can omit qualifications, misinterpret chronology or present unsupported conclusions with apparently convincing language.

A citation can help expose those errors by identifying the passage from which a statement was derived, but the presence of a citation does not guarantee that the original evidence supports the generated interpretation.

DBS therefore requires caseworkers to check summaries against source documents. AI output cannot form the sole basis of an assessment, recommendation or barring decision, and employees must report inaccuracies through established operational and quality assurance processes.

The agency also identifies automation bias as a risk. Employees may attach excessive weight to computer-generated material, particularly when it appears structured, comprehensive and consistent with their initial understanding of a case.

Training consequently covers verification, recognised limitations of generative AI and the importance of maintaining independent professional judgement. Existing appeal arrangements remain in place because decisions continue to be made by DBS personnel.

Several models support one casework process

The tool operates within Amazon Web Services and combines document processing, retrieval, language generation and PDF production.

Submitted documents are processed using AWS Lambda and Textract, including the extraction of information from forms and tables. Amazon OpenSearch stores document representations and supporting metadata so relevant evidence can be retrieved when the system prepares its summary.

The recorded models include OpenAI’s GPT-OSS-120B for text generation, Nvidia’s Nemotron Nano 12B for image descriptions and Amazon Titan Embedding v2 for retrieving relevant information.

All three are pretrained models accessed through AWS Bedrock. DBS says referral information is supplied as context for generating summaries and is not used to train or fine-tune the underlying foundation models.

Retrieval-augmented generation allows the system to locate relevant passages before asking the language model to prepare an answer. The implementation also uses contextual grounding and automated reasoning checks intended to reduce unsupported statements.

Those controls address particular forms of error, but they cannot eliminate the need for human verification. Retrieval systems can select incomplete evidence, while language models can still misunderstand information supplied in an otherwise relevant document.

The system processes personally identifiable information and special category data. DBS says access is controlled through restricted cloud storage, encryption, identity management, monitoring and audit logs, with processing governed by an approved Data Protection Impact Assessment.

Technology services company Reply developed the system and provides maintenance under its existing technical relationship with DBS.

The supplier does not have routine access to sensitive production records. Exceptional access requires explicit authorisation, time-limited permissions and an audit trail, according to the published arrangements.

These safeguards establish the intended data handling controls, although operational effectiveness depends on their implementation, monitoring and continued enforcement as the service develops.

Testing exposes what remains uncertain

DBS evaluated the system using fabricated cases, 15 anonymised training cases and approximately 140 production cases during its development and private beta phases.

The tests compared work completed with AI-generated summaries against work undertaken without them. The assessment covered case review time, accuracy of important fields, completeness, citation quality, potential bias and the cognitive demands placed on caseworkers.

DBS reports that summaries helped employees orient themselves within cases more quickly while retaining links to supporting evidence. However, the transparency record does not disclose a comprehensive numerical accuracy result or a quantified average reduction in processing time.

Consequently, the operational benefit cannot yet be assessed independently from the published information. Time saved during an initial review must also be considered alongside the additional effort required to verify summaries and correct inaccuracies.

The number and variety of cases used for evaluation influence how confidently the results can be applied to routine operations. Unusual referrals, incomplete documents and complex combinations of evidence may present problems that were not fully represented in the testing population.

DBS says its equality assessment identified no disproportionate adverse impact during private beta testing, while acknowledging the need for continuing evaluation as the system processes a wider range of cases.

The agency has established procedures for reviewing changes to the underlying models, including regression testing and governance approval before updates are introduced.

Such arrangements are important where public services depend on externally developed AI technology. Changes to model behaviour could affect the quality of summaries even when the original documents and operating process remain unchanged.

The Data Protection Impact Assessment was approved in May 2026 and is scheduled for annual review, alongside continuing checks of system performance and compliance.

The transparency record establishes the proposed controls and initial testing evidence rather than a completed independent assessment of long-term results. Wider operation will show whether the system consistently improves casework without encouraging employees to overlook relevant evidence or rely excessively on automated summaries.

DBS has preserved the division between machine-generated assistance and human statutory responsibility. The quality of that arrangement will depend on whether verification remains effective as the tool becomes a routine part of safeguarding work.