human robot

Navigating the Hallucination: GenAI’s Imperfection in FIs

“How is GenAI being used by financial services?” said Robert Antoniades. Co-Founder and General Partner of Information Venture Partners. “The simple answer is it’s not being used. Certainly not broadly.”

Given the amount of airtime GenAI has been getting in the conversations around innovations in financial services, this answer may be a surprise. However, Antoniades explained while AI is used all over the financial system, “The specific use cases of our interpretation of Gen AI, which is Chat GPT, I would struggle to find many use cases for that.”

Artificial Intelligence (AI) has been used increasingly in the financial services sector since the sixties. A recent report published by Bain Capital Ventures (BCV) stated that over 75% of companies of financial services companies now use AI for their operations. 

ai use in FI

Generative AI and Large Language Models (LLMs) are seen by many to have extensive potential, too, targeting areas where traditional AI has had its shortcomings.

“As sand can fill the open space in a jar of stones, generative AI can fill the gaps within financial services organizations left unfulfilled by traditional AI,” reads the BCV report. “These gaps today represent the unfulfilled potential of financial services organizations to offer better, higher quality, less expensive experiences for customers, more rewarding and fulfilling experiences for employees, and better earnings for shareholders.”

Throughout the fintech community, companies are announcing their exploration and application of GenAI. Areas that could benefit from the technology span financial services, from personalized customer service to improved underwriting. Where traditional AI has been applied, innovators are now striving to extract more, turning to the application of GenAI. But, while there is potential, many feel current applications fall short of the mark. 

“If ChatGPT is the iPhone, we’re seeing a lot of calculator apps,” said Christina Melas-Kyriazi, a partner at Bain Capital Ventures to the Financial Times earlier this year. “We’re looking for Uber.” 

A reason for this, in many cases, is its accuracy. 

Generative AI’s Imperfection

“The concern with Generative AI is its hallucinations or errors,” said Sam Bobley, CEO of Ocrulus.

The GenAI “hallucination” refers to instances when AI models produce answers or outcomes that are deemed “nonsensical” or “false”. While AI experts aren’t sure why exactly it occurs, possible explanations turn to the complexity of language and insufficient data. In the more publicized instances, lawyers have created arguments based on made-up precedents with non-existent citations, and journalists have published articles filled with fake news

These occurrences, while troublesome for the aforementioned lawyers and journalists, could be catastrophic if applied in finance. 

“If you’re applying for a small business loan or a mortgage, and the AI makes a mistake, and the wrong person is getting approved, or the lenders lose a ton of money, it’s a massive problem,” Bobley continued.

A particular area financial innovators have been exploring for GenAI application is financial advice. GenAI chatbots could be trained to become hyper-personalized financial advisors, able to communicate directly with consumers and tailoring responses to meet their specific needs. Consequently, this application could reduce barriers to accessing financial services and improve financial inclusion. 

RELATED: Why GenAI is More than Hype

However, due to possible hallucinations, applying pure GenAI to financial advisory services is still out of reach. “In financial services, if it’s anything important, it has to be 100% accurate,” said Antoniades.

“When you make a deposit in your bank account, you want to know the money’s there. It’s not that it can be there 99.9% of the time. It’s there. When they give you advice, they really should be 100% accurate, not 90% accurate…There’s no room for errors.”

Human in the Loop

While an answer to the hallucination issue could be restricting GenAI’s usage to areas that don’t need 100% accuracy, other solutions exist. Many are turning to a symbiosis of GenAI and human interaction. 

Antoniades explained that in many cases, GenAI could be used as a tool to generate an initial response that is then used by employees as a starting point for further exploration. “When you think about LLMs, they’re actually horizontal by nature. They scour all the information out there and then present it to you. And I think there’s a place for that,” he said. By doing so, GenAI can make workflows more efficient, allowing humans to only spend time focusing on specific areas and automating more manual tasks. 

Due to GenAI’s ability to evolve based on interaction, “feedback loops” can be created to help the tool reach 100% accuracy. Humans can check outcomes generated by the AI and feed in corrections that the AI can use for future answers. “Prompt engineering” is also a possible solution proposed by experts, allowing humans insight into the AI’s thought process and identifying issues. While still in need of human oversight, this process of troubleshooting, leading to increased accuracy, can improve the efficiency of automation.

“Because humans are focusing on fewer issues, and there’s automation in the process, you shorten the duration of which these processes take place. They’re thereby generating revenue for the institution much faster,” he continued. 

While a future where financial services are powered purely by AI is, in many cases, still only theoretical, the inclusion of it into a workflow including humans could still make its mark.

RELATED: The Fintech Coffee Break – Joe Robinson, Hummingbird

  • Isabelle Castro Margaroli

    Isabelle is a journalist for Fintech Nexus News and leads the Fintech Coffee Break podcast.

    Isabelle's interest in fintech comes from a yearning to understand society's rapid digitalization and its potential, a topic she has often addressed during her academic pursuits and journalistic career.