Summary

• The FCA has published examples of good practice and areas for improvement covering Consumer Duty, governance, senior management accountability and operational resilience in AI applications.
• Its observations include informal approvals, inadequate monitoring, ineffective human reviews and excessive reliance on technology suppliers.
• The publication introduces no new AI-specific requirements. Separately, an upcoming joint FCA and Bank of England survey indicates AI adoption among 86% of surveyed firms, although full methodology is not yet available.

Britain’s financial regulator has identified weaknesses in how firms govern artificial intelligence, finding that inadequate monitoring, poorly documented decisions and ineffective human oversight can undermine controls intended to protect customers and maintain operational resilience.

The Financial Conduct Authority (FCA) published examples of good practice and areas for improvement on 19 October, drawing on supervisory activity, engagement with firms and testing programmes including its AI Live Testing initiative and Supercharged Sandbox. The publication was updated on 20 October.

Its findings cover consumer outcomes, senior management accountability and the safeguards needed when organisations introduce AI into established business processes. Some firms had developed applications without formal approval arrangements, while others relied on employees to review automated recommendations without giving them enough information or authority to challenge the results.

The publication does not create new AI-specific regulatory obligations or establish that every practice criticised represents a proven breach. Instead, the FCA is explaining how firms can meet existing requirements as machine learning, generative AI and autonomous agents become more closely integrated with financial services.

Governance cannot follow deployment

Financial institutions already maintain procedures for approving technology changes, managing operational risks and determining which senior managers are responsible for important business activities. AI introduces different technical characteristics, but those arrangements still need to establish who owns the system and how its risks will be controlled.

The FCA found examples of firms introducing AI without sufficiently formal processes for testing, challenging and approving its intended use. Some organisations attempted to fit governance arrangements around applications only after important design decisions had been made, making it harder to explain why particular controls were selected.

Retrospective approval can leave gaps between technical development and business accountability. A model may have been tested by engineers without the relevant operational, legal or compliance teams examining the consequences of errors or unintended behaviour.

The regulator also encountered organisations whose governance frameworks did not distinguish adequately between applications with different potential impacts. A tool preparing an internal summary may require fewer safeguards than software influencing lending decisions or customer complaints, even where both rely on similar underlying models.

Better practice involved assessing risks at the beginning of development, involving the relevant business functions and maintaining records explaining why an application was approved. Some firms established dedicated AI governance committees, while others adapted existing arrangements to accommodate specific risks.

The FCA does not prescribe one organisational model. Smaller firms can use proportionate controls, provided responsibilities remain clear and the people involved have sufficient expertise to understand and challenge how systems operate.

Senior managers remain accountable

The Senior Managers and Certification Regime already requires individuals holding relevant responsibilities to take reasonable steps to manage risks within their areas of accountability.

Using AI does not remove those obligations, although senior managers are not expected to become specialists in model architecture or software engineering. They need appropriate information, access to expertise and evidence that important risks have been assessed.

The regulator identified cases where records of management decisions were incomplete, leaving firms unable to demonstrate the basis for approving an application or selecting particular safeguards.

Some senior managers delegated responsibility almost entirely to technical teams without sufficiently questioning whether the controls addressed the potential consequences for consumers or business operations.

Stronger examples involved documented escalation routes, agreed performance thresholds and regular information allowing management to assess whether deployed systems continued to operate as intended.

In one anonymised case, a firm required the relevant senior management functions and business areas to approve an incident playbook before an AI application was introduced. The document set out how staff should respond to service interruptions, unusual outputs and deteriorating model performance.

Agreed red, amber and green thresholds informed decisions about additional monitoring or temporary shutdown. The approach connected model behaviour with operational decisions rather than leaving staff to develop an improvised response after an incident occurred.

Human review can become ineffective

Financial firms frequently introduce human approval as a safeguard when automated systems make recommendations that could affect customers.

However, the FCA found that some reviewers gradually accepted AI-generated suggestions without sufficient scrutiny. Where the reviewing role was poorly defined, staff could become less likely to challenge an output even when there were indications of uncertainty or error.

Requiring an employee to click an approval button does not establish effective oversight if that person lacks the necessary information, expertise or authority to intervene.

The regulator also warns that human review can conceal weaknesses in an underlying model. An organisation might rely on employees correcting inaccurate recommendations while failing to investigate why the software continues generating them.

One firm testing an AI-assisted affordability assessment adopted a different approach. Employees made their decisions without seeing the model’s recommendation, allowing the organisation to compare independent human judgements with automated outputs.

Separating those decisions during testing helped the firm examine whether staff were placing excessive confidence in the technology. The method illustrates how oversight can produce evidence about performance rather than merely record approval.

For higher-risk applications, firms need clear criteria defining which decisions require review, how uncertainty should be handled and when a case must be escalated to someone with appropriate authority.

Consumer outcomes extend beyond accuracy

The FCA’s Consumer Duty requires relevant firms to consider whether their products and services deliver appropriate outcomes, including for people who may be vulnerable or need additional support.

AI systems can influence those outcomes through customer communications, eligibility assessments, complaint handling and the routes available for obtaining assistance.

The regulator found firms assuming that automated communications were sufficiently clear without testing whether different groups of customers understood them. Others supplied technical descriptions of models without explaining what an outcome meant for the individual or what they could do if they disagreed.

Some organisations also placed excessive confidence in language models identifying vulnerability from text alone. Temporary or less obvious circumstances may not be reliably detected through the information available in a particular conversation.

Better practice involved testing communications with consumers, providing meaningful routes to human support and reviewing whether automated interactions produced satisfactory results.

In another example, a firm used approved source information and restrictions on generated responses to reduce inappropriate chatbot behaviour. It maintained records of prompts, outputs, model versions and escalation decisions, allowing subsequent problems to be investigated.

Such records can help organisations identify recurring failures, but they must also consider whether the information being retained contains sensitive customer details requiring appropriate protection.

Monitoring cannot end when an application is first approved. Models may respond differently as customer behaviour, operating conditions or underlying software change, making continuing assessment necessary.

Supplier dependence creates operational risks

Many financial institutions obtain AI technology through external providers, including cloud services, commercial models and software embedded within other business applications.

The FCA found examples of firms relying on supplier assurances without testing whether those claims reflected the behaviour of their own deployed systems.

Other organisations had not developed adequate contingency arrangements if a supplier changed its model, withdrew a service or experienced a serious interruption.

Those dependencies can affect operational resilience because an AI application may become an essential part of a customer process even when the institution does not control the underlying technology.

Better approaches included contractual arrangements covering changes to models, monitoring of third-party services and the ability to move to another provider when necessary.

The regulator also highlights central inventories of AI agents, recording their owners, permissions and the systems they can access. Such registers may help firms identify where automated software has been given authority to perform actions across multiple applications.

However, maintaining an inventory is not sufficient unless permissions are enforced and changes to agent behaviour are assessed before they affect production systems.

Techopia previously examined the Financial Stability Board’s recommendations concerning human oversight of agentic AI. Those voluntary international recommendations are distinct from the FCA’s supervisory observations under existing British regulation.

The scale of adoption is also increasing. In a speech accompanying the publication, FCA chief data, information and intelligence officer Jessica Rusu said a forthcoming joint survey with the Bank of England found that 86% of participating firms were using AI.

That figure originates from an advance statement about research scheduled for publication the following week. The complete survey methodology was not available on 21 October, so the percentage should not be interpreted as a verified measure of every UK financial institution.

The FCA’s examples provide more immediate evidence of how implementation can fail, even where firms have established formal technology controls. As AI applications gain access to more information and operational functions, financial institutions will need to demonstrate that accountability, monitoring and intervention work in practice rather than relying on policy documents or nominal human approval.