Why financial firms need evidence their AI controls work

AI governance needs evidence of working controls

A financial firm can provide cybersecurity training and approved AI tools and still need evidence that employees handle sensitive information safely. For boards, compliance teams, security leaders and those responsible for AI governance, the practical test is whether staff follow information-sharing rules, check unusual requests independently and raise concerns during everyday work. Written controls establish what should happen; observed employee behaviour shows whether people carry them out.

Oz Alashe, Chief Executive Officer and Founder of CybSafe, recommends assessing controls through those everyday actions. CybSafe has a commercial stake in demand for the security support and behaviour measurement its experts recommend. “Financial firms need evidence that security controls work during an ordinary working day. That means understanding how people handle sensitive information and what happens when a convincing request arrives. Approved processes need to fit the job, and firms need to measure whether people can use them effectively,” Alashe said.

Training coverage and AI use measure different things

Joint research by CybSafe and the National Cybersecurity Alliance shows why everyday behaviour needs examination. Among finance and insurance workers who use AI at work, 42 per cent reported sharing sensitive information with AI tools without their employer’s knowledge. The corresponding figure for workplace AI users overall was 32 per cent. For financial firms, the higher sector rate makes employees’ information-sharing decisions a direct governance concern.

Interpreting these figures requires care about whom they describe. The survey covered 5,000 adults across the UK, US, India, Brazil and Singapore, including 308 employed respondents in finance and insurance. Within that sector group, the sensitive-information result applies to respondents who use AI at work. It describes those surveyed users’ reported behaviour, so applying it to every finance and insurance employee would change the population being measured.

AI use was common within the surveyed sector group: 72 per cent of finance and insurance respondents said they used AI at work. As generative AI tools, which produce content from user instructions, spread through office work, employees regularly need to decide what information they can enter. For financial institutions, sensitive categories include customer data, market-sensitive information and internal financial records. Sharing this information without employers’ knowledge raises concerns about both control over the data and visibility into its use.

Training access offers a separate measure of how employers support those decisions. In the same survey, 75 per cent of finance and insurance respondents reported access to cybersecurity training, compared with 60 per cent of employed respondents across all sectors. The sector therefore combines relatively broad training access with a higher reported sensitive-data sharing rate among workplace AI users. But these measures describe provision and behaviour in different respondent groups, so comparing them cannot establish training’s effect on individual decisions.

Even within a single workforce, access, completion and effective use of training are distinct questions. Governance teams need to establish whether employees receive training, understand the rules and apply them when choosing what to enter into AI tools. Speed and convenience are possible pressures to investigate, along with whether approved processes fit the job. Examining those pressures leaves the cause of behaviour open while giving firms a concrete way to test whether their support works.

Clear ownership gives employees a route to raise concerns

Applying an information-sharing rule may require help from another team, which makes ownership part of the control. The Financial Conduct Authority, or FCA, has highlighted cases in which firms introduced AI before establishing clear governance and escalation arrangements: the responsibilities and routes for raising a problem. Employees could therefore gain access to the technology before the firm had settled who should answer questions about its use. A workable deployment needs an identifiable destination for concerns.

Responsibility also needs to remain clear after deployment. In FCA examples, divided or uncertain ownership left staff without a clear way to escalate concerns or challenge decisions. Oversight helps an employee when it identifies whom to contact and how to obtain a response. Firms need to test whether staff can follow that route during ordinary work.

The FCA’s emphasis on outcomes alongside policy documentation makes this test relevant to governance reviews. Its examples concern responsibility and escalation, while the survey measures reported AI use and information sharing; each addresses a different part of employees’ working conditions. CybSafe’s recommendations add the company’s commercial perspective on assessing and improving behaviour. Alashe and the other CybSafe experts quoted here speak from within the same organisation.

Convincing requests make independent verification essential

A clear route for help also matters when an employee receives a persuasive request. In the CybSafe and National Cybersecurity Alliance survey, 22 per cent of UK respondents reported receiving a scam phone or video call in which the caller closely resembled someone they recognised through voice or appearance. This measure covers UK respondents across sectors and concerns reported impersonation through a recognisable voice or appearance. It is separate from the finance-sector findings about AI use.

The consequences reported by people receiving those calls make verification a material security issue. Among those UK respondents, 36 per cent said they or their employer lost money or data as a result. That percentage applies to respondents who reported the specified calls. Such requests involve social engineering: using urgency, pressure or familiarity to influence someone into taking an action that benefits an attacker.

Those pressures shape whether an employee can follow a verification rule. Dr Suzie Dobrontei, Behavioural Scientist at CybSafe, recommends examining what makes checking difficult and testing practical support. “A convincing request can arrive when someone is busy and trying to help. Employers need to understand what makes checking difficult in that moment. Give people a practical way to verify the request independently and support them when they pause or raise a concern. Then test whether that support changes what happens,” Dobrontei said.

Putting Dobrontei’s recommendation into practice requires a checking method independent of the incoming request, along with support when checking delays a response. Employers can then examine whether staff use that method when work demands their attention. The impersonation and AI-sharing findings concern different actions and populations, so each needs its own assessment. For verification, the question is whether an employee can pause, check independently and obtain help when needed.

Test changes against observed behaviour

Assessing those actions starts with a baseline: an initial measure of employee behaviour before support arrangements change. For AI use, firms should establish whether approved systems are available, whether staff understand which information they may enter and whether they follow those limits under pressure. Each of these three conditions needs examination. Recording them separately helps a firm identify where a change in support could help.

The same approach applies to verification and escalation. Firms should examine whether staff can independently check an unusual request and find a usable route for raising concerns or challenging a decision. Following the action through to its outcome shows what happens after someone recognises a problem. It also reveals whether the next step depends on help that employees can actually obtain.

Professor Jason Nurse, Director of Science and Research at CybSafe, recommends comparing later behaviour with the starting measure: “Organisations should establish a baseline, understand the barriers people encounter and test whether changes to their support improve those behaviours over time. Steps like providing safe, approved AI tools and ensuring training sessions clearly outline the reasoning behind rules are good, but it is just as important to measure exactly how much effect these changes are having. That is how you go from piecemeal improvements to building a robust security culture.”

Following Nurse’s recommendation means investigating barriers before choosing a response, then measuring behaviour again after making changes. Safe, approved AI tools are one proposed way to help employees complete work within permitted boundaries. Training that explains the reasons for information-sharing rules is another, intended to help staff apply those rules to the decisions they face. Measuring subsequent behaviour tests how much each change helps.

The choice of change should also reflect the work employees need to complete. Alashe’s emphasis on processes fitting the job gives firms a reason to investigate whether an approved process is difficult to use in a particular role. Dobrontei’s focus on checking raises a related question: whether verification or escalation remains usable when someone is busy and trying to help. Firms can examine these as design questions and adjust support where they find barriers, while keeping that work separate from assumptions about why survey respondents shared information.

Even after a change improves one action, firms need to measure what follows it. Nurse cautions that one sound security habit does not establish whether a person completes the next step. Recognising that a request needs checking, for example, leaves the separate task of carrying out verification. Measuring recognition and completed verification separately shows where employees still encounter difficulty and which support changes help them finish the protective action.

Key takeaways for leaders

• Test controls in everyday work: AI governance reviews need evidence of how employees handle sensitive information, verify unusual requests and raise concerns. Compliance and security teams should test whether approved processes remain usable under workplace pressure.
• Separate training access from safe AI use: Among surveyed finance and insurance workers using AI at work, 42 per cent reported sharing sensitive information without their employer’s knowledge. Training owners should assess access, understanding and application separately to identify where support needs improvement.
• Assign clear escalation ownership: Financial firms need named owners for AI questions and concerns. Governance teams should test whether employees know whom to contact and can obtain a response when challenging a decision.
• Make independent verification practical: Among UK respondents who reported receiving convincing impersonation calls, 36 per cent said they or their employer lost money or data. Security teams should provide independent checking methods and support employees when verification delays a response.
• Measure behaviour before and after changes: Security and AI governance teams should establish a baseline, identify barriers and reassess behaviour after introducing support. Track recognition and completed protective actions separately to see where employees still encounter difficulty.