UK AI chatbots misstate financial queries in Saturn study

Traders Union
All News
Financial News
AI chatbots misstate financial queries

Highlights

Saturn research finds AI models from ChatGPT, Claude, Copilot, Grok, and Gemini answer financial queries incorrectly 57 percent of the time on average.
AI models make mistakes in 88 percent of complex financial questions, with some responses potentially costing savers up to 17,500 pounds in tax charges.
Financial Conduct Authority data shows one in five UK adults is open to AI decision-making on debt, pensions, and investments, driving calls for urgent regulation.

Saturn testing highlights accuracy gaps

As reported by Financial Times, research from technology firm Saturn says popular AI models from ChatGPT, Claude, Copilot, Grok and Gemini answer financial queries incorrectly 57 per cent of the time on average. When faced with more complex questions involving more than one calculation, the models make mistakes in 88 per cent of cases on average, and some models give wrong answers in 99 per cent of those harder prompts.

The study uses more than 100 money-related questions across free and paid-for tools, with prompts repeated up to five times. In total, more than 10,000 questions are put to 18 AI models, and Saturn says the responses include calculation errors, missed upcoming tax changes and hallucinated rules.

Amal Jolly, Saturn chief executive, says millions of people are trusting AI models for money advice but are receiving answers that can cost them money. The research says one mistaken response on pension tax rules from Claude Haiku 4.5 could leave a saver facing a 17,500 pound charge from HMRC, while another answer on student loans invents a rule suggesting a graduate moving abroad could stop repayments.

Consumer and regulatory pressure grows

Paid-for and newer models perform better than cheaper and older versions, but the strongest result in Saturn's testing still shows a high error rate. Claude Opus 5 in reasoning mode is the best-performing model overall, yet it still makes mistakes in 39 per cent of answers.

Sarah Coles, head of personal finance at investment platform AJ Bell, says advisers are sometimes approached after clients follow baffling AI suggestions and need to check whether action has already been taken. She adds that people needing support should seek proper advice, though chatbots can still be useful for budgeting and background research such as identifying possible overspending from income and outgoings.

The findings come as the Financial Conduct Authority says one in five UK adults is open to AI making decisions for them, with demand strongest in complex areas including debt, pensions and investments. FCA research published last month also shows younger investors are more likely to trust AI than financial influencers or TV shows, while two-thirds expect to increase their AI use over the next year.

Jolly says AI financial advice remains unregulated and leaves consumers without the protections, including compensation, available through a human adviser. He says the FCA has started considering the issue but needs to move quickly to protect consumers.

In our earlier article on Weiss Ratings’ AI-driven Ultimate Portfolio, we explained how the firm combines stock ratings, economic forecasting and market-timing models into a real-money strategy aimed at long-term, retirement-focused investors. We noted that Weiss has promoted performance claims around its integrated approach and later added an “AI Performance Booster,” illustrating how AI is increasingly positioned as a decision-making aid in investment research and portfolio management.

This material may contain third-party opinions, none of the data and information on this webpage constitutes investment advice according to our Disclaimer. While we adhere to strict Editorial Integrity, this post may contain references to products from our partners.