Bank launches generative AI strategy with conversational chatbot for frontline employees
In every industry, technology executives are getting the AI “shoulder tap” – questioned by boards on what their generative AI plan is. Rather than providing its board with a knee-jerk use case, the technology team at one North American bank was more forward-looking. They had a business need in mind, but they knew that for AI to be applied in a meaningful and successful way, they had to anchor any use cases in a broader AI strategy.
Inspiring change
The bank’s contact centre operational costs were going up. The culprit? An internal search tool with low accuracy was forcing frontline employees (tellers, financial advisors and more) to call an internal helpline. This led to response delays, increased costs, spotty response reliability and the redundancy of multiple people seeking the same information. It was also undermining the market-leading customer experience that the bank strives for with every interaction.
As a visionary institution committed to being digital innovators, the bank was ready to be an early adopter of generative AI technology. The team thought a conversational chatbot would not only address response accuracy, operational efficiency and user experience issues, but also improve training and knowledge management, tighten up regulatory compliance and strengthen its security posture.
The bank wanted a partner who was experienced in solving complex technical challenges and would bring best practises and lessons learnt from deploying generative AI – a technology in its infancy. They sought a partner who would help harness ambitions into a strategic AI roadmap built for scaling and with attention to the right metrics to demonstrate business value. And they wanted a partner who was a good cultural match – who would deliver a programme “with” them, not “to” them..
Driving innovation
The bank turned to Avanade, who formed a cross-disciplinary team from advisory, data, analytics and AI to design a chatbot solution that moved rapidly from proof-of-concept to minimum viable product to deployment. It incorporates:
- The Microsoft AI tech stack, leveraging key capabilities in the Microsoft Azure AI suite, including the Azure AI Search and Azure OpenAI, hosted on Azure Application Services for scalable backend deployment, with Azure Security Centre and Azure Active Directory providing advanced security and user authentication.
- Customised prompts through advanced prompt engineering techniques, such as chain of thought and custom functions, to enhance chatbot performance.
- A novel architecture design that includes a multi-prompt bot and a distillation process, enabling high-accuracy targets and optimised responses.
We also understood the need for a framework for observability. As the bank rolls out the chatbot to more users, eventually reaching tens of thousands of employees in thousands of branches, it will need to be monitored for continued response accuracy. Avanade constructed a LLM operations framework with:
- A bespoke solution to measure chatbot accuracy in production using custom metrics, including LLM-as-judge and answer similarity.
- Automated regression testing and reporting for performance validation to ensure continuous improvement.
- Customised prompts for monitoring and regression, the design and development of FAST API endpoints, as well as testing and integration into a CI/CD pipeline.
Achieving what matters
The results for the bank are compelling:
- The conversational bot is processing thousands of documents and managing hundreds of thousands of interactions weekly.
- It has achieved high-90s response accuracy using a more cost-effective model like GPT-3.5.
- Automating regression testing in the LLM operating model has reduced manual efforts by more than 90%.
- Frontline employees are serving more customers yet spending less time in their interactions, improving operational efficiency.
- Compliance has improved, with workers accessing the correct procedures and responding to customers with confidence.
- Customer engagement and satisfaction has increased.
- Growing AI literacy across the enterprise is expected to drive the organisational culture toward digital transformation and innovation.
Most significantly, the bank’s technology team has an answer for their board, with alignment around a generative AI strategy, a governance framework, and real dollars and people powering the program. The team is experimenting with many more use cases and are imagining what they can do with AI to make a genuine human impact in other areas.
When everyone and everything is connected by data, Avanade can help turn your data, analytics and AI ecosystem into your biggest advantage.
Want to talk to an expert?
Headquarters
North America
1191 Second Avenue
Suite 100
Seattle, WA 98101
Growth Markets
Avanade Asia Pte Ltd
250 North Bridge Road
#30-03 Raffles City Tower
Singapore 179101