Usability Testing of a New Loyalty Service for the Bank

About client

JSC CB “PrivatBank” is the leading state-owned bank with a customer base of 17 million. It offers highly developed digital banking services for both B2C and B2B customers through its mobile SuperApp Privat24, and web portals.

Challenge

The new loyalty program “Privit” introduced a revolutionary concept for the Bank, not only through its mechanics and personalization engine but also with a fresh communication strategy Craft Innovations faced the challenge of combining usability testing for both consumer and business target audiences as well as assessing visual communication and naming perceptions and evaluating the effectiveness of future TV commercials designed to promote this program.

Project goal

Gather customer perceptions and insights about the loyalty program concept to inform the bank’s product, marketing, and design teams.

Process explained

1. To accomplish the project goals, we selected two primary research methods: moderated usability testing and in-depth interviews combined with focus groups for obtaining relevant feedback on the TV commercial concept.Our research targeted two major audiences: consumers and merchants. The consumer group was divided into two subgroups based on their experience with loyalty programs and specific characteristics important to the project’s goals. The B2B audience included marketing and customer experience executives from various retail and service industries.
   For each target audience, the Craft Innovations research team developed a list of 15 user tasks to simulate different use cases. Based on these scenarios, the bank’s product design teams created clickable prototypes of the new service. To derive actionable outcomes from the usability tests, we employed the Single Usability Metric (SUM) methodology and the SUM Score metric. This metric aggregates various parameters, such as task completion rate, execution time, number of errors, and Customer Effort Score, providing a comprehensive understanding of the UX/UI performance.
2. In this project, each usability test session, combined with an in-depth interview, lasted 90-120 minutes. These sessions provided us with an extensive understanding of users’ mental models, concept perceptions, barriers, and opportunities for service improvement.
3. Based on the research results, PrivatBank’s product design and marketing teams made significant improvements to the service and successfully launched it in Q2 2024.

Methods we applied

• B2C / B2B recruiting
• Moderate Usability testing
• In-depth interview
• Focus groups