Whether you’re running moderated usability testing or remote unmoderated IA testing, knowing how many participants to recruit is crucial for getting the most out of your research

Why does the number of participants matter?

If you have too few participants, you might miss valuable insights or usability issues that could affect a broader range of users. You may also risk the findings not being taken seriously by stakeholders who question your small sample size and don’t believe your findings.

On the flip side, testing with too many participants can be costly both in terms of paying recruitment and incentive costs as well as additional resourcing time. Each additional test session requires time for recruitment, preparation, moderating and data collation, all of which add up very quickly.

In a budget constrained environment that most of us work in, the goal of usability testing is to find and address the issues that are causing the most pain for your users as cheaply and efficiently as possible.

Understand the purpose of your research

Before you can decide how many participants you need, ask yourself the purpose of your testing and what kind of insights you’re looking for. Is this a qualitative exploration to understand user behaviour in-depth? Or is it a quantitative study to get more data on user pathways and the effectiveness of your information architecture (IA)?

Qualitative research – moderated testing

Moderated usability testing is about uncovering common, high-impact problems that affect many users.  In qualitative studies, the goal is to uncover deep insights, understand user behaviour and users’ mental models rather than gathering statistically significant data. Therefore, you can often work with a smaller group of participants—enough to identify key pain points and usability issues. 

If you’ve been in the world of UX research for a while, you’ve likely heard about Jakob Nielsen’s famous ‘rule’: for usability testing, 5 participants are often enough to uncover the majority of usability issues. Why is five the magic number?

When running usability tests, you’ll typically find that the first few participants encounter most of the major usability issues. With each additional participant, you uncover fewer new problems. By the time you reach 5 participants, you’ll have likely identified around 85% of usability issues.

If you have budget for than 5 users, consider conducting a few rounds of testing each with 5 participants, and iterate your design between rounds.

 

The power of the “magic number 5” in usability testing.

There are situations when 5 participants won’t cut it.

  • Complex tasks or specialised users: When your user base is highly specialised – think medical professionals, financial analysts, or technical engineers – or if your product involves complex tasks, workflows, or multi-step interactions, the standard "magic number" rule of might not apply. Testing with a group of 5 participants in such cases may not capture the full breadth of needs or behaviours because unique audiences often have unique contexts, needs, workflows, and pain points that a small sample might not fully represent. The likelihood of encountering varied usability issues in these instances is higher, so a broader sample size will give you a more comprehensive understanding of where users struggle. Consider increasing the participant count to 8-12.
  • Diverse audience: You might also need more participants if your product or site has several personas and your target audience groups include people with diverse abilities, needs, behaviours, or knowledge. In this instance, it might be a good idea to test with 5 users from each major user group, rather than 5 in total. When testing with more participants, you can be confident that your findings apply broadly and that you’re not missing out on insights only applicable to specific user groups.

Quantitative research – remote, unmoderated testing

Remote, unmoderated testing can often be done quickly and cheaply and provide quantitative numerical data about patterns and behaviours to inform design decisions that can be generalised to a larger population.  

We often use remote quantitative testing in conjunction with moderated testing:

  • to quickly and cheaply identify the big issues that affect many users before we conduct moderated testing to explain the findings and understand the “why”; or
  • after moderated testing to gather more quantitative test data to quickly and cheaply validate your qualitative test findings. This will provide more empirical evidence. which in turn will help convince stakeholders.

Calculating the number of participants you need when conducting remote testing such as Information Architecture (IA) tree testing or click-testing can be particularly confusing. It depends on your budget and these three factors:

 

1. Number of tasks: The more tasks you want to test, the larger your participant pool should be. Testing a wide variety of tasks with too few users can lead to incomplete data, making it harder to spot issues. If you have a larger number of tasks, you’ll want to increase the participant count to ensure that each task gets adequate coverage.

 

2. Tasks per participant: Tree and click testing often involve participants completing several tasks within a session. However, if participants are assigned too many tasks, they may experience fatigue, leading to lower quality data. Additionally, participants tend to become familiar with the IA or designs being tested, relying on their memory rather than interacting with it naturally. To reduce the learnability effect, limit the number of tasks each participant completes to no more than 10-12 and increase the participant count to distribute the tasks across a larger pool. This ensures that each task is tested enough times to identify issues without overwhelming individual participants.

 

3. Number of times (sample size) each task is tested: If you aim to test each task multiple times to gather more robust data, you will need a larger participant pool. We generally aim for a sample size of at least 10-15 for each task to ensure you capture patterns. This approach enhances the reliability of your findings and ensures that your design decisions are based on a comprehensive understanding of user interactions.

How to calculate how many participants or tasks

So, if you want to test 30 tasks, have at least 10 results per task (n=10), and limit the number of tasks each participant attempts to 12, how many participants do you need to recruit? The answer is 25 participants.

As not all UX folks are also mathematicians, we have created a little tool to help you easily calculate the number of participants you need for your next study. You can also play around with this tool if you have a set number of participants and want to work out how many tasks to write.

Download our free participant calculator to simplify your planning and ensure your testing is effective.