Reverse engineering user research from the UK’s 7th largest bank
Monzo was one of the first challenger banks back in 2015.
It was a breath of fresh air to the ‘old ways’ of doing things: calling your bank to make transfers, going to the branch to cash-in a cheque, or paying back your friends in coins.
Not with Monzo.
Digital-first with easy-to-use UI and down-to-earth branding.
Monzo is now the UK’s 7th largest bank with over 7.5 million users.
Whilst users grew 24% last year, revenue was up at an even more impressive rate at 130% — reaching £355 million in 2023.
What’s cool about Monzo, is they’ve largely grown through Product-Led Growth (PLG).
Famously, PLG came in the form of their Willy-Wonka-style Golden Ticket referral scheme and the bright orange debit card that quickly spread through London and other major cities.
The Scarcity Principle of the limited Goldren Tickets per user meant they were hard to come by; prospects had to join a wait list or find someone who had a spare ticket, spreading the brand name through word of mouth.
Whilst the early PLG loops have largely died off, Monzo continues to grow, reporting that 66% of new users joined through word of mouth in 2023.
When a product has Product Market Fit, growth comes easier. Users are more likely to spread the word and tell their friends. Plus, the Network Effect of the frictionless transfers between Monzo users means that Monzo is better when more people are on it. You’re incentivised to share with your friends.
I’ve had Monzo for over five years now. I’ve fallen out of love with it at times, and fiercely relied on it at others.
But recently, I received an interesting email: user research on mortgages.
‘Mortgages? What does Monzo do with mortgages’ I thought.
So, let’s dive into the intriguing user research, the methods Monzo use to test assumptions and we’ll have a go at reverse engineering what the Fintech could be planning next.
We’ll look at:
The UX research emailThe question contentThe question typesThe prototype
And round off with some research best practices and learnings from Monzo.
Let’s go.
A well-crafted UX Research Email
First up, I like this email. It’s concise, pretty and personal.
This email is great for 5 key reasons:
It’s in an attractive template, meaning that — due to the Aesthetic Usability Effect — the experience feels easier to use.The sender introduces themselves in the first line, making the email more personalThey get to the point quickly — in the first paragraph of body text we know why she’s reaching outIt is specific — we’re clear on who this research is targeting i.e. people with mortgagesThe survey CTA is clearly visible in a bold, primary colour
However there are four things I’d improve:
Use my preferred name (not even my mum calls me Rosemary…)Make sure there’s also a link (as another way to access the survey)Make sure the CTA is top of foldTell me how long the survey is going to take
And perhaps offer an incentive, although this can bias the results. As you’ll find out, it was a preeeetty long survey.
I’d have like an amazon voucher for my time as it felt almost cheeky towards the end.
In any case, I imagine they’ve tested this format to find what works best for their users.
An Extensive Question Set…
Tapping through to the questions themselves, I see a Monzo-branded web survey with near 35 survey questions, that aim to find out:
If I own a property (or multiple)If I have a mortgageIf I am the decision makerIf I used a mortgage broker when I got the mortgageWhen I took out my first mortgageIf I used a mortgage brokerIf I have remortgagedIf I used a mortgage broker for the remortgageIf I moved to a new lender or new deal, and whenMy feedback on the process and what I considered importantMy thoughts on a mock upWhether I agree with different statements about said mock up
From this list, we know that Monzo are researching mortgages, and perhaps more specifically mortgage brokers or remortgaging. Perhaps the research questions is something to the effect of:
What place do mortgage brokers have, if any, in the lending and remortgaging process?How do Monzo users feel about their mortgage brokers and the mortgage deals they secured?What is important to users in the mortgage lending and remortgaging process?How might we provide a better experience to users through the lending process?
Its likely that the recent high in interest rates drove this research, and the general chaos in the mortgage markets in the UK over the past year or so.
In any case, despite the research being mortgage-centric, there’s a few interesting learnings we can take away from this question set about how to run good surveys:
1) It’s important to check your respondents are the right persona
Are they the decision maker? Are they a first-time user or a second-time user?
These sort of breakdowns make sure that the person you’re talking to is the right persona and you can be precise with your analysis.
It lets the researchers break down answers by new/returning mortgage-rs i.e. whether they’re a newbie or an established player in the field in question.
This not only leads to more insights, but sifting the wrong people out means you increase the validity and accuracy of your results.
It’s tempting to get as many answers as possible and analyse all the results, but if some people aren’t right, you’re harming your analysis, outcomes and decision-making power.
2) It’s crucial to dig slowly, layer by layer
Each of these questions is asked separately. In a way, it feels like Progressive Disclosure.
What’s crucial about this is that each question is easy to understand and doesn’t try to do too much at once.
Many people fall into the trap of asking two questions in one, to try and reduce the number of overall questions. Don’t do this — as you can confuse users.
If someone doesn’t understand your question — because you’ve crammed too much in — you’ll harm the validity of your results.
This is especially true with surveys ,where you have one shot to collect the data. You can’t just rephrase the question…
Next, onto how these questions were asked.
Creative Question Types
Within the survey, I came across a huge range of question types:
Multiple choice (most used)Single choice (less frequent)Free text entry (very few)5-point scale (sometimes used)Respond to prototypes (one prototype) — more on this in the next sectionLikert Scale (based on Prototype)
One of the coolest things about this survey was how the questions changed depending on your inputs.
It felt like there was a range of journeys I could have taken within this survey thanks to the responsive questions.
Looking at the URL, it seems Monzo are using Qualtrics, an experience management platform.
Given the level of customisation needed, as well as the creative use of some of the question sets, Google Form, Survey Monkey or Typeform would perhaps be more limited here.
My one pet peeve is this question:
19 options. NINETEEN!
I scanned this and picked the first one that resonated.
There’s a fine line between being specific, as we covered earlier, and being overly exhaustive in your question set. Completion rates also matter.
Here’s another: this question one isn’t so long, but could have put ‘if any’ in the question to avoid leading the respondent:
With some of these processes, I feel like some people just go with the flow and don’t have super strong opinions or even know what to care about. It is such a minefield out there.
Next, onto my favourite and final part: the prototype hidden at the end of the survey.
Prototype Test
Now, at this point I’m near 30 questions in. Which meant that this was a welcome sight: some colour that differs from the black & quite question set.
According to Teresa Torres in her book Continuous Discovery Habits, the crucial rule for assumption testing with prototype tests and other methods is to replicate a real life scenario as much as possible.
Meaning you need to give:
Context (Imagine you receive this email)Mock up (As close to real life as possible)Actions (As close to real areas to interact with the prototype as possible)
And you need to measure:
Whether users do the thing you expect, or not.
With this in light, the Monzo prototype does well, with the context and email mock up. I think it could have been more realistic with some email inbox UI.
Ideally, you watch if your users actually tap the CTA. However this is a survey. So the next best thing is the question
How likely or unlikely are you to tap CTA?
Then a 5-part Likert scale.
The next question is a free text entry asking if there’s anything they like or dislike:
I personally didn’t have any strong feeling about the email, so found this part hard. Perhaps a question like:
Please add more detail about your response.
Would have been less hot and cold.
Next, we’re asked the same 5-part Likert scale for four statements that relate to the image:
After seeing this page, how much do you agree or disagree with the following statements?Monzo and partners will help make the remortgaging process hassle free
We were correct in the assumption that they’re testing how they can make the process better (in this case, less hassle) than current solutions (i.e. mortgage brokers).
I would trust Monzo and partners to handle matters relating to my mortgage
Interesting assumption around trust here.
Monzo and partners will help me make sure I don’t pay more than I have to on my mortgage
Score! Another near-correct assumption that they’re looking at how good of a deal users feel they get, and if Monzo can position themselves better.
Monzo and partners will help me save time on my remortgage
Interesting that the second pain after hassle was time in their view.
So now we know they care about remortgaging, and whether they can provide a hassle-free, time-saving and trustworthy solution by using partners.
Veeeery interesting.
Watch this space.
And in the meantime, let’s summarise some key learnings about survey science and assumption testing.
Lesson & takeaways
Five Key learnings about how to run good user research:
Make sure you’re sampling the right users: ask sufficient questions in order to sift out the unqualified personas. Go for quality over quantity (as long as you hit the minimum sample size for your research)Don’t try too much in one question: ask your questions gradually to make sure you don’t over-complicate and invalidate your data.Avoid leading questions: ensure to write ‘if any’ or give users an exit from questions, and ‘none of the above’. Again, this will make sure you gather results you can rely on.Try to replicate a real-life scenario to test assumptions: give a context, give a mock up, and ask about real behaviour. Even better, watch them do it.Most importantly: work out what you’re trying to test ahead of time. Write out your research plan and assumptions. What do you think is going to happen? Make bets with people in the team to make it fun and ensure a culture of learning, growing and customer-centricity.
Any more learnings you’ve taken here?
Drop them in the comments 🍃
Till next week 👋
How Monzo does assumption testing was originally published in UX Collective on Medium, where people are continuing the conversation by highlighting and responding to this story.
