Note: This is the third in a series of posts about the Nonprofit Web Design Process. See the end of this post for a linked index of other posts in the series.
Surveys and Interviews are useful tools to help us learn and/or validate what constituents want from a nonprofit’s website. Many times, the internal stakeholders at an organization have some idea of what their users want and they’ll share that knowledge with us during Stakeholder Discovery. We can also get a feel for what constituents want by reviewing the Analytics Data and seeing what content is most popular. The best way though, to find out what constituents want, is to ask them directly.
Our purposes for Surveys and Interviews are:
- To collect opinions and feedback about various aspects of the website
- To learn what users find most valuable about the website
- To find out what new content users might be interested in seeing on the site
- To establish a profile for each major audience group
Survey or Interviews?
Surveys provide us with more quantitative data about what users want versus Interviews that provide qualitative data. We can send a Survey to an entire housefile but we typically only will interview 6-8 constituents who are representative of each of the organization’s audience groups. Remember our goal of triangulating data? (see graph on the right) The other user research activities we plan to conduct for a project will help us determine if Surveys or Interviews will be most useful. If we have another activity that will provide quantitative data, we may do Interviews to balance out our research. If our other activities are more qualitative in nature, we may do a Survey to provide some statistically significant data.
If you have the time and inclination, you could also do both! Start with your Survey and ask users to provide their contact information if they’re interested in participating in future research. Based on their Survey responses, you can then hand-select participants for your Interviews who are representative of each of your audience groups.
Recruiting is pretty straightforward for Surveys since the goal is to cast your net as widely as possible. I typically recommend sending an email invitation to the entire housefile and also promoting the Survey on your website and via your social networking sites.
For Interviews, recruiting is a bit more challenging. As I’ve mentioned a few times, you want to make each of your audience groups is represented in your participant pool. So if your audience groups are volunteers, donors and alumni, you want to make sure you interview at least one of each.
To find these participants, you can rely on your internal stakeholders to identify folks that might want to participate and fit the profile. You can also start with a Survey and ask for volunteers that way. I’ve had success with an online recruiting tool called Ethnio that serves your website visitors a pop-up with a few screener questions. Those that meet your criteria are then asked for their contact information and you can follow-up with them for the interviews.
Interviews are typically more time-consuming for the participants so you may want to offer an incentive to thank your constituents for their time. A $20 Amazon gift card will usually do the trick.
Typically the questions in a Survey will mirror those asked during an Interview with the obvious difference that the Survey will often have a multiple choice listing of responses. I like to do rating scales with Surveys so I’ll list out features or content areas and ask the user to rate how interested they are in each item. Also, I tend to lead Surveys with the most important questions first so that if a user quits the Survey, we still have some valuable data.
Examples of the questions we might include are:
- What is your relationship with XYZ organization?
- How often do you visit XYZ site?
- What areas of the site do you typically visit?
- What new content/features would you like to see on the site?
- What do you particularly like about the site?
- What might you like to change on the site?
- What other websites do you visit that are like XYZ site? What do you like or dislike about those sites?
- Do you have any additional comments or feedback?
I rarely ask demographic information in a Survey but some clients ask for it. I think it would be weird to ask demographic questions in an Interview (how old are you?!) so I don’t typically include them there either. Usually the internal stakeholders have enough knowledge about the demographic make-up of their constituency to get me what I need for design. Knowing that the population is older may affect the design somewhat but knowing male vs. female or income ranges simply aren’t necessary to me so I opt to leave them out.
A deliverable for a Survey or Interviews would be a series of slides that summarize the findings. For Interviews, I will also provide transcripts or recordings of the Interviews though I’m not sure they’re ever reviewed!
As I proceed with the User Research, I like to compile findings from each activity into a single series of slides so that I can compare findings from one activity to the next and point out trends in the data. The deliverable would always culminate with a series of hard and fast recommendations about how the research will affect the next phases of the project.
Once you’ve determined what your constituents want from your website, the next major task is to define the structure of the site (a.k.a. the Information Architecture) so users can find what they’re looking for. Card Sort tests can provide insight into how constituents sort and organize the content you plan to offer on your website.