UK FCA Exec Calls for Disclosure of Significant AI Failures

UK Financial Conduct Authority (FCA) Chief Data Officer Jessica Rusu said on Monday, in a speech to a Bloomberg UK Policy Series audience, that UK firms should be required to report "significant AI failures and near-misses," adding to growing calls for regulatory control over enterprise AI deployments. Rusu said existing rules already require firms to report major incidents and that this must extend to significant AI failures and near-misses, calling near-miss reporting for agentic AI "a national security priority" and "a regulatory priority too."

Rusu also said firms must invest more in cyber and operational resilience and that agentic systems require "continuous, conditional assurance," because a system safe at deployment may behave differently at scale, under stress, or as underlying models change. She said the FCA will work with industry, fellow regulators and frontier labs on a "trusted framework for agentic finance," covering agent identity, delegated authority, accountability and execution. Questions to the FCA asking what "significant AI failures and near-misses" precisely means had not been answered by publication time.

Pieter Arntz, a malware intelligence researcher at Malwarebytes, said a significant AI failure is one with serious consequences, such as an AI system authorizing substantial fraudulent payments, exposing sensitive customer data, denying access to essential financial services, or triggering transactions that threaten market integrity. He described a near-miss as an event that could realistically have caused serious harm but was caught beforehand, such as an unauthorized transfer blocked by a separate approval control. He warned the approach could become a "paper tiger" if compliance amounts to filing a report and absorbing a predictable fine.

Gartner VP analyst Nader Henein, based in the UK, was skeptical of the proposal's significance, noting that UK enterprises already face EU rules requiring similar disclosures and that the Bank of England, the UAE Central Bank and the Monetary Authority of Singapore have issued warnings. He said that months earlier European central banks and the European Systemic Risk Board raised the cyber risk assessment level for frontier models to "severe," and that without clear laws, oversight and fines, such statements are unlikely to change behavior in finance.

Noah Kenney, principal consultant at Digital 520, warned that increased visibility requirements could inadvertently incentivize the wrong behavior: a bank with good monitoring may report many attempted agent overreach incidents while one with poor monitoring reports zero, making the safer bank look worse. Frank Dickson, principal analyst at Dickson Research, said the near-miss idea is "the most important thing in the speech" and the part financial firms are least ready for, and noted Rusu's warning about reliance on a single model, including outages at one AI supplier. Dickson also said a model can pass testing in March and misbehave in May if the provider changes the model without notice, so model version changes should receive the same scrutiny as other production changes.

UK Financial Conduct Authority (FCA) Chief Data Officer Jessica Rusu said on Monday, in a speech to a Bloomberg UK Policy Series audience, that UK firms should be required to report "significant AI failures and near-misses," adding to growing calls for regulatory control over enterprise AI deployments. Rusu said existing rules already require firms to report major incidents and that this must extend to significant AI failures and near-misses, calling near-miss reporting for agentic AI "a national security priority" and "a regulatory priority too."

Rusu also said firms must invest more in cyber and operational resilience and that agentic systems require "continuous, conditional assurance," because a system safe at deployment may behave differently at scale, under stress, or as underlying models change. She said the FCA will work with industry, fellow regulators and frontier labs on a "trusted framework for agentic finance," covering agent identity, delegated authority, accountability and execution. Questions to the FCA asking what "significant AI failures and near-misses" precisely means had not been answered by publication time.

Pieter Arntz, a malware intelligence researcher at Malwarebytes, said a significant AI failure is one with serious consequences, such as an AI system authorizing substantial fraudulent payments, exposing sensitive customer data, denying access to essential financial services, or triggering transactions that threaten market integrity. He described a near-miss as an event that could realistically have caused serious harm but was caught beforehand, such as an unauthorized transfer blocked by a separate approval control. He warned the approach could become a "paper tiger" if compliance amounts to filing a report and absorbing a predictable fine.

Gartner VP analyst Nader Henein, based in the UK, was skeptical of the proposal's significance, noting that UK enterprises already face EU rules requiring similar disclosures and that the Bank of England, the UAE Central Bank and the Monetary Authority of Singapore have issued warnings. He said that months earlier European central banks and the European Systemic Risk Board raised the cyber risk assessment level for frontier models to "severe," and that without clear laws, oversight and fines, such statements are unlikely to change behavior in finance.

Noah Kenney, principal consultant at Digital 520, warned that increased visibility requirements could inadvertently incentivize the wrong behavior: a bank with good monitoring may report many attempted agent overreach incidents while one with poor monitoring reports zero, making the safer bank look worse. Frank Dickson, principal analyst at Dickson Research, said the near-miss idea is "the most important thing in the speech" and the part financial firms are least ready for, and noted Rusu's warning about reliance on a single model, including outages at one AI supplier. Dickson also said a model can pass testing in March and misbehave in May if the provider changes the model without notice, so model version changes should receive the same scrutiny as other production changes.