*This article has been updated since it was originally published in 2009*
This is number 2 in a 3-part series about card sorting, written by David Humphreys and Tania Lang. You can find the other 2 articles here:
Running moderated card sorts
To run an in-person card sorting session, you’ll need one or two facilitators to run the sessions and collate the data and participants who represent the users on the site.
The facilitator
The facilitator's role is to:
- welcome the participants
- explain what they are going to do in the session
- tell them the rules
- help them when they get stuck or confused by individual cards, and
- make sure the participants' writing is clear enough to read and that their groups make sense.
It's useful to have a second person in the room to help with participant questions and if possible, observe and learn from the process (and to help stick down cards at the end when they all finish at the same time).
The participants
One of my past students told me they did a card sort activity and the users came up with the same groupings as their current public website. When I asked who she did card sorting with, she told me “staff”. Can you see the problem? After a while, employees learn your website’s structure and it becomes their mental model.
As such, the participants should be representative of the users of the site – not staff! Unless staff are actual users of the site (such as for an intranet), we don’t recommend using internal staff for card sorting as they are not representative of your external user population and often have a different mental model (i.e. think differently) about how the information should be sorted. If you want to provide staff with an opportunity to be involved, conduct remote online card sorts using tools such as OptimalSort.
Depending on time and budget you can run a different card sorting session for different user groups on the site (such as prospective students and current students on a university website) to get a good picture of how their different mental models work. Again, using remote online card sorting tools, such as OptimalSort make it easier to isolate results. However if time and money are tight, just getting a good mix of users in the room that are representative of the people using the site is a great start.
In a moderated session we typically limit to 10 participants. Each session should have representative users of one user group although that is not always possible.
10 participants can usually generate enough content to work with however Jakob Nielsen recommends 15 users per group.
The aim is to generate enough results that you have a good understanding of the patterns that emerge in the results and are comfortable with their repeatability.
Recruitment and venue
Recruitment can be a huge time-sink and a major headache. We partner with Market Research firms because they have huge databases of potential participants and have the resources to recruit quickly and efficiently. However, recruitment costs can be expensive, and if you are using external participants you will often need to pay them a small incentive just to show up. Only the most dedicated (or rabid!) customers are going to turn up for free.
Recruitment firms usually require about a week's lead time and if you are doing the recruitment yourself you should allow a similar time frame for a dedicated resource. Make sure you ring and confirm with all participants the day before the session.
Environment
You'll need a large room with a large amount of table space – enough for 10 large sheets of paper to be spread out on without crowding (we have had very unhappy participants in crowded table spaces). Ideally each person should have about a metre to spread out in.
The room layout for card sorting we recommend is a large conference room with tables in a U-shape or classroom style with plenty of space and little opportunity to see what other teams or individuals are doing. This is the ideal scenario but please be mindful of your participant's comfort. They will produce better results for you if they are happy. From our experience, board rooms with a long thin table down the middle are not great unless they are a really big organisation with a really big boardroom.
Running remote online card sorts
Online card sorting tools
We like to use a suite of online UX testing tools from Optimal Workshop. They have a online remote card sorting tool, OptimalSort, which also assists with analysing the data once your participants have completed the card sort.
Another option is Miro. While it doesn’t have a dedicated tool, it does have card sorting templates that can be used instead but it won’t give you the really useful data visualisations and analysis that OptimalSort provides.
Advantages of unmoderated, remote card sorting
- Cost effective – There’s no need to find a venue and participant incentives tend to be lower for online tasks rather than in-person sessions.
- Time – Participants can complete the card sort in their own time, which means more participants are likely to be able to complete it. Also takes a lot less of your time as you don’t have to prepare all the cards and run the session.
- Recruitment – Given the activity is remote and often takes less time, it is generally much easier to recruit participants that are geographically dispersed or difficult to access. Using recruitment tools such as Askable, you can get a larger number of participants to undertake a short online activity in a short timeframe. In 2024, we did an online card sort for a government mental health site and it was completed by 30 participants within 48 hours.
- Analysis – Tools like OptimalSort provide analysis tools too with good visualisations of different groupings. You can also include questions which allow you then to view data for different cohorts e.g. separate results from employees versus customer. You can also delete or filter results.
Disadvantages of unmoderated, remote card sorting
- Language – There’s no way to know if the words you’ve used on your cards are being misinterpreted.
- Commitment – For remote online activities, some participants often don’t make much effort to think about what they are doing. Often you get a few ‘rubbish’ results that are pretty useless e.g. 3 groups: 1) ‘Information’ 2) ‘FAQs’ and 3) ‘General’.
In our next article, we will talk about some common tips and challenges for card sorting.
This is number 2 in a 3-part series about card sorting, written by David Humphreys and Tania Lang. You can find the other 2 articles here: