Chatbot.
Digital assistant.
AI concierge.
We’re all familiar with these terms. Commonly used within our digital products. Often fronting similar interfaces. Occasionally describing VERY different capabilities.
Internally, the naming helps to cement the vision. It signals ambition. It can shape how teams think about future capability.
Externally, it can be central to the proposition & desired positioning.
For the user, it sets an expectation.
But let’s be honest. Some providers are more aspirational, a little punchy even, when it comes to these claims. So when it comes to understanding what these experiences actually deliver, you need to go beyond the hype & uncover the real experience.
Setting the expectation
At a high level, the language suggests a range of capability.
- Chatbot is a term used as shorthand for a wide variety of digital chat capabilities
- Digital assistant might suggest a level of awareness and the ability to guide
- AI concierge implies something more proactive, perhaps using agentic and acting on behalf of the customer
However, in practice, the experience is not always aligned to the label.
Two providers may both describe their solution as a “digital assistant”. One might signpost users to an article, whilst another completes a task end-to-end. One might be more proactive, the other more contained.
But let’s remember. Most users are simply looking for help.
To find something.
To fix something.
To understand something.
To achieve something.
Whilst hopefully avoiding frustration.
So in the user’s eyes, who cares if it’s really agentic. It’s the experience itself that determines customer satisfaction & loyalty. If this falls short, so will your retention.
80% would rather do business with a competitor after a poor customer experience – IBM
Narrowing the context
Within many apps, a conversational experience can be accessed on demand. For simplicity, let’s call this interface the chatbot.
A small icon. Usually on the homepage. Sometimes labelled “Help” or “Support”. Sometimes branded with a dedicated name or icon. The names are often suggestive of a character. A personal approach that’s carefully crafted to build trust & engagement with your bank / retailer / telco / utility provider.
Tap it, and a chat interface opens. This is where we focus our analysis within our Chatbot Experience Benchmark.
What are the best-in-class examples of personal relevance or task completion? Which ones understand intent and offer good chat?
When comparing experiences it can certainly help to narrow down the context and also the sector. Within our benchmark reports, this allows the analysis to present a controlled, unbiased & evidence-led comparison.
Why banks?
When creating a benchmark report, we’re careful to select sectors that offer range and representation.
Within our Chatbot Experience Benchmark, we focus on 20 global banking apps. These providers are early adopters & big investors, representing both fintechs and legacy giants.
From Monzo & Lloyds to Bunq & Bank of America, these providers are:
- Balancing the emergence of AI capabilities with local compliance
- Responsible for a vast amount of personal data
- Each setting the precedent for others in Financial Services & beyond
With a collective user base of >600M, they are setting conventions and shaping the expectations of a vast user base.
Yet. To date, there hasn’t been a rulebook for these experiences. A framework for assessing how they compare behind-the-login. A blueprint for what good looks like.
Until now.
An invitation for digital product teams
For digital product teams, the upcoming Chatbot Experience Benchmark offers an invitation.
- Cut through assumptions and discover how the leading chatbots really compare.
- Explore a showcase of Best-In-Class conversational experiences.
- Learn vital lessons and discover where there’s space to take the lead.
The report is now available for pre-order.
