The Challenge of Measuring User Experience in Financial Platforms

George is a UX Analyst at Ferratum and HCi researcher with 4+ years of expertise in conducting meticulous user research, usability tests, and data analysis to drive innovation.

Online loan applications are quite complicated processes and improving their user experience is our Product Excellence team’s and George’s bread and butter. To achieve this, they constantly evaluate the ease of use of our products. A significant part of the evaluation involves gathering feedback from our customers about their perceived experience.

Customer centricity is a word that we see on almost every product description, and not by accident. We live in a fast-paced and highly competitive landscape in which the concept of customer centricity has become the measure of success for almost all institutions, especially those operating in the realm of FinTech.

Customer centricity is not just a buzzword though; it represents a mindset shift in how financial platforms approach their operations. It essentially describes the prioritisation of the needs, preferences, and overall experiences of customers.

User Experience and Usability

Okay… customer centricity… that should be connected with user experience, right?
User Experience (UX) is a multifaceted concept that encompasses a user's interactions with a product and is the core of customer centricity. UX deals with the emotional, cognitive, and behavioural aspects of a user's journey —in short, it tries to answer the question “How is the experience of the person who is using our product?”.

Zooming in, to provide a great user experience one must provide a product that is not only easy to use, but also efficient to use. In other words, a product that is highly usable. In the context of financial platforms, a superior user experience means more than just a cool website, nice animations, or a straightforward navigation system; it implies a deep understanding of the customer’s financial goals, emotions, and the ability to offer seamless, secure, and personalised solutions.

Why It Is Important to Measure Usability

What we cannot confidently measure, we cannot confidently improve. Measuring usability is a critical aspect of improving user experience, and it serves several vital purposes for almost all applications alike. Let’s look on it from a financial institution’s point of view:

Enhanced Customer Satisfaction: Usability evaluations, such as user testing and feedback collection, help identify pain points and bottlenecks within a product. The results of these evaluations can be given to a UX designer, who is going to make changes that increase usability.

Improved conversion rate: Making a product easy to use and efficient, aids the person who is using it to complete their goal easier, faster, with less hiccups. Which in turn decreases the percentage of people who abandon the product.

Competitive Advantage: In all applications, especially in the FinTech industry, the competition is fierce. A user-friendly and efficient platform is a significant differentiator.

The Challenge of Measuring Usability in Financial Platforms

While the importance of measuring usability in financial platforms is evident, it comes with its own set of challenges. We have noticed that in financial platforms usability measurement is not as straightforward as in some other domains. There are 3 main challenges we have encountered:

Feedback sessions can only be roleplayed: Financial platforms handle sensitive and confidential information, customers are required to submit information like their ID number, or IBAN. A common way of usability testing entails users completing a process while speaking their thoughts out loud (aka moderated and unmoderated testing). In situations where the process to be tested is related to some financial task (e.g. applying for a loan) the possibility to use real sensitive information is diminished. Roleplaying can be used instead where each participant is given a crafted, fake identity.

Complexity of Financial Transactions: Financial transactions involve multiple steps, data inputs, and decision-making processes where a lot of them happen offline. For example, in testing the usability of a credit card, one must consider situations where the purchase happens in-shop, or in some foreign currency country. Measuring usability in this context requires a deep understanding of the intricacies of financial operations and regulatory requirements.

Bias in Usability Assessment: A quite unique challenge that we noticed in some financial applications and published this year. People’s perceptions of usability can be influenced by the outcomes they receive from the product they are using. In the context of financial platforms, where critical decisions like loan approvals are made, user bias may affect usability assessments.

Why care about the bias on usability assessment?

User bias towards the usability of a system is normally offset by proper user sampling. However, we noticed in systems that administer critical feedback, users tend to be biassed based on the system’s response. In financial terms: customers who got their loan accepted rated our applications of high usability, whereas customers who got their loan rejected rated our applications of low usability. We systematically analysed this behaviour and published our results in the International Conference for Human-Computer interaction.

This bias can significantly affect the usability measurement, and should be taken into account. Nonetheless, after discovering this bias we still need to validate it (Does it appear in other domains? Is it an attribute of the system or of the users?) and propose a method to mitigate it. We're currently having some fruitful conversations and evaluating suggestions from the fellow researchers in HCII 2023 to mitigate this problem. In the future we plan to use this learning to overcome the challenges in accurately measuring system usability.

Disclaimer: The information provided in this article is intended for general informational purposes only. It is not intended to be, and should not be taken as, professional or financial advice.