Summary

• NatWest reports a 20% relative reduction in misheard words across the accents tested with Nvidia's Nemotron speech recognition technology.
• Ninety employees contributed approximately 3,000 natural-language banking questions, with further adaptation using anonymised contact centre transcripts.
• The work supports Spending Insights, which is being tested with 75% of Royal Bank of Scotland customers ahead of wider deployment.

NatWest Group has reduced errors in its AI speech recognition technology by 20% across a selection of British accents, working with Nvidia to improve how conversational banking software understands customers’ pronunciation, regional expressions and financial terminology.

The development supports Spending Insights, a service that allows customers to ask questions about their finances using speech or text. The feature is being tested with 75% of Royal Bank of Scotland customers before a planned wider rollout across NatWest Group.

NatWest combined Nvidia’s Nemotron automatic speech recognition technology with its own language models, which interpret transcribed requests and support the generation of responses about customers’ financial information.

Initial development involved 90 employees from different parts of the UK recording approximately 3,000 realistic banking questions. Participants represented accents including Glaswegian, Northern Irish, Welsh, Yorkshire, Mancunian, Midlands and Southern English, alongside non-British speech patterns.

Rather than requiring everyone to read fixed scripts, researchers encouraged employees to use the words and expressions they would naturally choose when discussing their money. The resulting recordings helped identify cases where existing speech recognition struggled with regional pronunciation or the meaning of a financial question.

When one misheard word changes the answer

Automatic speech recognition converts spoken language into text that can be processed by other software. Its accuracy depends on identifying words despite differences in pronunciation, pace, background conditions and sentence construction.

Banking questions present additional difficulties because customers frequently refer to precise amounts, dates, merchants and periods of time. An error involving one of those details can alter the information retrieved or the calculation performed.

During NatWest’s testing, a Scottish employee saying “last quarter” was transcribed as “last water”. Although the resulting phrase is recognisably inappropriate in a banking context, the example illustrates how a small recognition error can change the meaning of an otherwise straightforward request.

The researchers also identified merchant names, financial terminology and dates as recurring areas of difficulty. Names that rarely appear in the information used to develop a general speech model may be particularly challenging, while customers can describe the same transaction in several different ways.

NatWest says collaboration with Nvidia reduced the number of words misheard by 20% across the accents tested, improving the model’s word error rate.

Word error rate measures substitutions, insertions and deletions relative to a reference transcription. A 20% relative reduction in errors does not mean the system became 20 percentage points more accurate, nor does it establish the same performance for every speaker.

The bank has not disclosed the complete baseline error rate or separate results for each accent. Without those figures, it is not possible to determine whether recognition quality became consistent across the groups tested or whether some continue to encounter more errors than others.

The published result also concerns recognition of words rather than the accuracy of the financial information eventually supplied to a customer. A correctly transcribed question can still be misunderstood by the language model responsible for interpreting it.

Adapting the software to financial language

Following the employee recordings, NatWest expanded its development work using anonymised contact centre transcripts. These provided examples of language encountered in customer interactions, including financial expressions and the different ways people refer to their transactions.

The bank says the transcripts remained inside its secure environment, with no customer data shared with Nvidia. Model adaptations using NatWest information were also restricted to the bank.

This division allows Nvidia’s speech recognition technology to be used alongside institution-specific adaptations without transferring the underlying customer records to the technology supplier.

However, the effectiveness of the arrangement depends on appropriate access controls, secure processing and the quality of anonymisation. Financial conversations can contain distinctive combinations of transactions and personal circumstances, even when direct identifiers have been removed.

NatWest will also need to manage changes to the models as speech recognition technology develops. Improvements intended for one group of speakers or financial expressions should be evaluated against the wider customer population before deployment.

The system separates the task of recognising speech from the subsequent interpretation of a banking request. Nvidia’s Nemotron technology converts audio into text, while NatWest’s models work with that information to identify the customer’s question and prepare a relevant response.

Each stage introduces possible errors. An incorrectly recognised amount may cause the software to retrieve unsuitable records, while a correctly recognised request can still produce an inaccurate response if the interpretation or supporting information is wrong.

Testing the complete interaction therefore requires more than measuring transcription accuracy. The bank must also examine whether customers receive correct information, can identify mistakes and can recover when the software misunderstands a request.

Speech becomes another banking interface

Spending Insights is designed to let customers explore financial activity through conversation rather than relying entirely on fixed menus and search fields.

A customer could ask about spending over a particular period, request additional detail about a category and then change the question based on the information returned. NatWest plans to support those interactions with visual information, including charts, as well as spoken or written responses.

For the service to be useful, it must retain sufficient context between questions while continuing to respect the permissions governing customer information.

Speech may also improve accessibility for people who find conventional banking interfaces difficult to navigate. Those benefits will be limited if recognition performance is uneven across accents, speech characteristics or environmental conditions.

Providing text-based alternatives and straightforward ways to correct misunderstood requests remains important. Customers may also prefer not to discuss sensitive financial information aloud in public or shared spaces.

The service’s development raises broader operational questions about the cost of running speech recognition and language models for routine banking enquiries. Processing an open-ended conversation may involve more computing than a conventional transaction search or menu-based interaction.

NatWest will need to establish whether the resulting improvements in customer experience justify those operating costs, particularly if use expands significantly following the current trial.

The bank has not published figures for customer satisfaction, the cost per interaction or the proportion of enquiries completed successfully without additional assistance.

Its reported reduction in recognition errors nevertheless addresses a specific limitation of voice interfaces: the difficulty of making general-purpose speech technology work reliably with regional language and specialist vocabulary.

Wider deployment will provide a more demanding test involving different microphones, background noise, individual speech characteristics and questions not anticipated during development.

NatWest has demonstrated an improvement across its tested accents, while the reliability of the complete banking experience will depend on how accurately the system interprets requests, retrieves information and handles mistakes during everyday customer use.