Note: This is the fourth in a series of posts about the Nonprofit Web Design Process. See the end of this post for a linked index of other posts in the series.

Purposes

Card sorts are a form of usability testing that help us understand how users group and label content. Simply put, it’s a matter of creating a “card” for each piece of content you plan to offer on your website and asking folks to stack the cards into groups and then apply a label to each group. We have done this in-person as part of stakeholder discovery and have also done them remotely, with end-users, as a user research activity.

Some indicators that a card sort test is a good activity for a design project include:

  • A website that is structured according to the org chart so that each department has their own section. This is known as “showing your corporate underpants” 🙂
  • Internal staff have trouble finding content on the website or deciding where to put new content
  • A navigation structure that requires 3+ clicks to get to any substantial content

Our purposes for Card Sort tests are:

  1. To learn how users or stakeholders might organize the content on your website
  2. To guide the sitemap revision process and introduce new ways of thinking about content relationships and groups
  3. To identify content that’s difficult to sort so we can provide users with several pathways to access that content

Methodology

In-Person or Remote? Moderated or Unmoderated?

Deciding what kind of card sort test to do is usually a matter of time and budget. If we’re already planning an on-site visit with the organization, doing an in-person card sort makes perfect sense. An in-person test would be great for your organization to conduct on your own as well.

If we’re looking at using the card sort for user research, it usually makes more sense to do a remote test so we can get feedback from a variety of geographic locations. OptimalSort, seen in the screenshot below, is a great tool we use for remote card sorting. If we have the budget available, it’s always best to moderate at least a few of the card sort tests, meaning we are on the phone with the constituent while they’re sorting the cards to ask questions along the way and solicit more information on why they create the groups they do.

Screenshot from OptimalSort Card Sort Test

Note: Closed card sorting is also an option, though not one I’ve ever used. In a closed card sort, you create the groups and ask users to put cards into each one.

Recruiting

If we decide to go the route of end-user card sort tests, recruiting the participants is step one. We typically aim for a participant pool of 15-50 constituents. 15 is the minimum to make sure we have enough data to observe patterns and any more than 50 responses tend to get unwieldy to analyze. Just like with User Interviews, you want to make each of your audience groups is represented in your participant pool. So if your audience groups are volunteers, donors and alumni, you want to make sure you include a few participants from each group.

To find these participants, like with Interviews, you can rely on your internal stakeholders to identify folks that might want to participate and fit the profile. You can also start with a Survey and ask for volunteers that way. I’ve had success with an online recruiting tool called Ethnio that serves your website visitors a pop-up with a few screener questions. Those that meet your criteria are then asked for their contact information and you can follow-up with them for the card sort.

Incentives are a nice-to-have for card sorts and we usually go the route of doing a drawing for an Amazon gift card instead of offering one to each participant.

The Cards

There are all sorts of articles online about choosing the right cards for the card sort test. Some “tricks of the trade” I follow are:

  • Create 30-100 cards. This is a wide range but any fewer than 30 and it’s difficult to form groups; any more than 100, then the groups start to become difficult to manage.
  • Make sure the cards are all relatively on the same level of your content hierarchy. So you wouldn’t want one card to be “News” and another to be “News Article about XYZ”. Typically, I’ll use the lower tier of information (i.e. “News Article about XYZ”) to get to the right number of cards.
  • Be creative with your card labels. Avoid using the same words between cards, especially at the beginning of the card since this can result in people automatically associating those cards with one another. Keep in mind that your card labels don’t necessarily need to reflect your website’s navigation labels.

For more tips about selecting cards and the entire process, see this great article on Boxes and Arrows.

Deliverables

An initial deliverable for a Card Sort test would be a series of slides that summarize the findings; however, analyzing the results of a card sort can get a bit complicated. See, each participant (or group of participants) will have different groupings of the cards. What’s helpful is to observe patterns across different participants’ sorts to influence the creation of the Sitemap.

If we do a test via OptimalSort, there are some great analysis tools built-in to the software that help us identify these patterns. If we’re not using OptimalSort, there are some spreadsheet analysis templates that have proven useful. The ultimate deliverable from a card sort test is a new sitemap that reflects the thought patterns of the card sort participants. We’ll discuss the Sitemap in more detail in a later post.

What’s Next

There are two additional user research techniques that we’ll cover in the next couple of posts: Usability Testing and Personas. As I mentioned previously, a card sort is actually a type of usability test. The other type of usability test that we’ll cover next is a task-based test where we observe users interacting with your website.

Other Posts in this Series

  1. Introduction
  2. Stakeholder Discovery
  3. User Research
  4. Content Strategy
  5. Information Architecture
    • Sitemap
    • Wireframes
  6. Visual Design
  7. Solution Design
ABOUT THE AUTHOR

Lacey Kruger, principal information architect for Blackbaud, works with nonprofit clients to design online properties that work. Whether a full-scale website, a campaign site or a mobile app, Lacey guides clients through a research-based and user-centered approach to design. In her 10+ years at Blackbaud, she has developed a deep understanding of nonprofit web presences. That knowledge, along with her years of experience in information design, have established her as an industry expert.

Lacey has written a Blackbaud eBook, “A Guide to the Nonprofit Web Design Process” and her article, “Designing Nonprofit Experiences: Building a UX Toolkit” was published in User Experience magazine. She has presented at industry conferences including bbcon, IA Summit and BIG Design. When she’s not working, Lacey loves to cook and also enjoys yoga, watching movies and catching alligators (really!).

Get nonprofit articles, best practice advice, fundraising ideas and invaluable industry reports and webinars delivered for free!