Usability testing kit

A collection of guidelines and templates to help you check the usability of election materials

You’ve actually been conducting usability testing on your materials for years. It happens every time someone uses a form, reads a social media post, or marks a ballot.

The materials provided in this kit can help you systematize the process, produce reliable findings, and generally expand your ability to test thoroughly with real users — before you launch it out into the world.

Usability testing is a way to learn about how easy or difficult it is for people to use something by observing them actually using it. It produces much different findings than just asking them if they like it. Usability testing can be quick and easy, and you can do it any time.

What you'll need

This kit covers 3 different approaches to usability testing. Different approaches require slightly different materials, but these are the basics that will get you started:

  • Someone to run the test
  • Someone to help observe and take note
  • Someone who might use what you’re testing
  • Something you want to test (a newly designed ballot, a revamped election website, new voting equipment, etc.)
  • A quiet place to conduct the test

Table of contents

  1. Getting familiar with usability testing
  2. Planning your usability test
  3. Holding a practice test

Getting familiar with usability testing

You can think of usability testing as a different way of doing community review. In a usability test, you test your materials with individuals, one at a time. You ask each person to use the materials. While they do, you watch and listen without teaching or training.

That’s different from focus groups, surveys, and advisory groups, where you might show the materials to groups of people and ask them what they think or how they feel.

Why do usability testing

Through usability testing, you can learn about what works (and what doesn’t) for users, and you can confirm or challenge assumptions. You won’t have to put something into operation just hoping it will work. You’ll know.

When you have data from usability testing, you can update your election materials, design a ballot, or launch a new outreach campaign with confidence.

It will help you:

  • Identify problems in the design of election materials that could lead to problems for users or for election administrators
  • Make it more likely that interactions people have with your office are efficient, effective, and pleasant
  • Make it easier for users to use your materials – whether the users are candidates, voters, poll workers, election staff, advocates, media, or other people your office interacts with

When to do usability testing

Depending on what you’re working on, you can do usability testing almost any time.

But it’s most useful:

  • Before you start redesigning something (so you can identify the problems you want to fix and address them in ways that won’t frustrate users)
  • While you’re still working on a draft (so that you can check your work early)
  • When a significant event happens that may cause major changes or that may require staff or volunteers to be retrained
  • When something about the voting situation has changed since the last election (such as new machines, new laws or rules, or a new step in a process)

Planning your usability test

What does a usability test look like?

A usability test is made up of one-on-one sessions with one moderator and one user or participant. The moderator (usually you) will be near the participant to conduct the session and ask follow-up questions (but not to teach or help). A note taker may support the moderator by logging observations and discussion.

During the session, the moderator and note taker collect observations (what you see and hear). Observations give you data that you can use to make judgments about what’s frustrating to users and how you might eliminate those frustrations.

A usability tester speaks with 3 participants while another tester takes notes

Usability session with moderator, note taker, and 3 participants

Each participant uses a copy of the same materials. The moderator guides participants through the session, observes their behavior, and takes notes (if possible, which is why you have a note taker). Remember: do not help participants do the tasks.

After the testing part is done, if there’s time at the end of the session, you may help participants learn how to use the materials. They often appreciate knowing what their mistakes were. Correcting participants will help you understand what to fix in the materials, too.

Usability test sessions are usually short – about 15 or 20 minutes – but can go longer.

Consider inviting other people on your team to join in by observing the sessions (but silently).

How many participants do I need?

Some rules of thumb (we’ll give additional details about participants in the rundowns below, too):

  • Try to test at least 4 people who normally might use the material you’re testing. If you think there might be different behaviors for different types of users, include 4 of each type.
  • Make sure your participants represent a diverse group of users. This means including people with disabilities, different ethnic backgrounds, and differing English language proficiencies.
  • Don’t worry about demographics like age, education, or income. Think more about what experience participants might have doing the tasks you want to observe, and simply recruit a range. For example, you probably want some participants who are first-timers as well as people who have been through the tasks before.
  • You don’t need a large sample of participants. You can easily learn what’s frustrating to users and how you can make improvements with just a few participants. Don’t worry about “statistical significance.”

The goal is to learn enough in your usability test to make informed design decisions, so you can take the next steps (and, maybe, test again).

How should I find participants?

There are lots of ways to find participants!

First, you have to decide if you’re going to schedule people in advance, or if you’re going to use impromptu recruiting by going to where likely participants might be.

If you’re scheduling appointments ahead of time, tell everyone: colleagues, staff, friends, family, advisory groups or other groups that would fit your user profile. Use every channel you’ve got. Use social media like Twitter or Facebook, or put an ad on Craigslist to invite participants to your study. Post a notice on your website that says you’re looking for people to volunteer to help you make things better.

Usability test moderator discusses materials with 2 participants

Usability testing at a community center

If you’re doing quick testing in the field, go to places like community centers, county offices, shopping areas, farmers markets, and libraries. You want to be in a place with a lot of foot traffic that will give you the opportunity to recruit participants right there, on the spot.

Looking for some recruitment tips? Check out this article on how to recruit subjects for your usability tests.

Often, participants can be recruited as unpaid volunteers. Other times, you might think about paying participants for their time or reimbursing them for things like parking or travel.

No matter what approach you take to recruit your participants, make sure they fit your criteria of likely users. That means that they currently do the behavior you want to observe, such as voting, running a campaign, working the polls, and so on.

What should I look for during a usability test?

You’re looking for both what worked (successes) and what didn’t work (failures). For example:

  • Any errors or mistakes – even if the participant corrects them
  • Failure to complete or submit
  • Places where the participant didn’t follow instructions (on a form, for example)
  • Any signs that the participant doesn’t understand the information or actions needed
  • Any errors specific to the material being tested (filling out a form, tallying totals, packing up materials incorrectly, transcribing from source to form, etc.)

Behaviors you might see and want to take note of:

  • Questions participants ask
  • Requests for assistance
  • Spoken comments or non-verbal reactions
  • Expressions of emotion (confusion, frustration, anger, disgust, delight, satisfaction)
  • Any behavior that is adaptive (such as taking out reading glasses or moving in closer to the machine)
  • Incidents that would be documented and recorded
  • Behaviors or comments that surprise you
  • Things that surprise the participants
  • Distractions (glare, noise, movement)

Holding a practice test

Before you test with real volunteers, you might want to do a practice session with office colleagues or other trusted friends. This exercise will also help you practice note taking while doing a usability test.

Since you might not already have materials you want to test, we’re providing you with a sample: instructions for how to fold an origami rooster.

Your goal with the test? To measure the effectiveness of the origami instructions provided.

What you’ll need

  • Moderator
  • 3 participants
  • Paper supplies to make an origami
  • Test materials (see below)

Duration of test

Not more than 15-20 minutes per participant

How to do the test

Find a quiet spot to conduct 3 usability tests — one participant at a time. Share the instruction sheet with them along with the paper supplies to make the origami.

With 12 graphics and illustrations, the 12 basic steps needed to fold an origami rooster

Practice test instructions

Follow the moderator’s guide provided to introduce the participant to the project and guide them through the process. While they’re testing, jot down observations.

After you’ve completed all 3 sessions of your usability test, use the note taking guide provided to record your top 5 observations and insights about the usability of the origami instructions, focusing specifically on what you heard and what you saw. In the “additional comments” section, include points of analysis on the closing questions:

  • What worked well?
  • What needs improvement?

Test materials

To run this practice usability test, you’ll need these test materials:

  • Moderator’s guide
  • Origami instructions
  • Note taking guide

Download the practice test materials

The test materials are contained in a zip folder. When you click the link above, your download should begin automatically, and the folder will be saved to your default download location.

Once you’ve downloaded the folder, you’ll need to extract the files.

  • To extract the files on a PC, right-click on the zip file and select Extract All. You’ll be prompted to choose a location. Do so, and then press Extract. Your computer will extract the files and create a new folder.
  • To extract the files on a Mac, you can simply double-click on the zip file, and your computer will extract the files into a new folder.

Now that the files are extracted, you can view the folder’s contents and use the materials. The moderator’s guide and note taking guide are Microsoft Word .docx files, while the origami instructions are a .jpg image.


Once you’ve practiced with this test session, you’ll be ready to look at the 3 approaches to usability testing that we cover here and think about which one best suits your situation and your needs.

Table of contents

  1. Overview: 3 approaches to usability testing
  2. Testing in the office: quick usability feedback from colleagues
  3. Testing out of the office: catch real users in the field
  4. Testing in the conference room: thorough, structured testing with real users

Overview: 3 approaches to usability testing

It’s common for teams in the private sector to use usability testing to help them develop the right products the best way. In this tool, we present simplified and streamlined versions of usability testing that should make it easy for you to get started.

There are lots of ways to conduct a usability test. The approach you’ll take will depend on what you want to learn and the amount of time and resources available to plan a test and carry it out.

We highlight 3 possible approaches to usability testing that you can use individually, or in combination, to test your materials. The more you test, the more confidence you’ll have about how easily and successfully users will use your materials.

The 3 approaches are:

  • Testing in the office: quick usability feedback from colleagues. If you’re in the early stages of designing or changing your materials, this approach is a fast and easy way to get a second opinion or quick feedback on something specific.
  • Testing out of the office: catch real users in the field. If you have drafts or prototypes that are close to complete, you can use this approach to observe people interacting with the materials and test how successfully people use them.
  • Testing in the conference room: thorough, structured testing with real users. If you have solid, revised materials that you want to test, this approach  allows you to run a more formal, structured test with real users who you’ve scheduled ahead of time.

Now that you know the basics of the 3 approaches, you can find additional details, guidelines, and materials for the 3 approaches below.

Testing in the office: Quick usability feedback from colleagues

Getting quick usability feedback from colleagues is an informal way to get immediate reactions to the ease-of-use of your materials.

The idea is to walk down the hall or to the next department, find someone who isn’t too busy at that moment, ask for 5 minutes of help, put your design in front of them, and ask them to use it. Then, watch and listen while they work with it.

A test moderator speaks with a participant about their impressions of materials

Informal usability testing in the office

If your organization works closely with specific advisory groups, you can also recruit participants from this group.

Input from advisory groups is a great resource to supplement usability findings. In other words, you should not rely solely on advisory groups to get input on usability but rather use them as additional participants if you have access to them.

When to use it

Use this approach any time you want quick feedback on something you’re developing. Of the 3 approaches, it requires the least time and resources and usually works best in the early stages of designing new materials or updating old ones. It’s also good exercise if you’re new to usability testing.

Who to test with

Bring together 2-4 colleagues – but make sure they’re people who know nothing about the material you’re testing. Although your colleagues need not represent your target audience, you can ask them to use personas to think about a user’s perspective.

If you’d like some tips on how to use personas, check out this article about different ways to approach a form.

What you’ll need

  • A moderator
  • A rough draft (even a sketch) of the new material you’re developing
  • Test materials (see below)

Where to do it

The test can be scheduled informally and be conducted at your desk, a participant’s desk, or in a conference room.

How to do it

  1. Recruiting – Walk down the hall or to the next department and find someone in your office who’s not involved in the design/discussion of the material. Ask the person if they have 15 minutes to spare to do a quick review of the material with you.
  2. Setting up – If they agree, find a quiet spot to conduct the test and get comfortable. Take out your moderator’s guide provided in the folder below as well as the materials to be tested. Using just a notepad or the note taking guide provided, jot down observations and comments.
  3. Testing – Hand them the sketch or draft design and ask them to use it as a typical user might. Pay attention as they work their way through the material. Observe their behavior and jot down those glaring usability issues that came to the surface during the testing.
  4. After the test – Thank them for their time.
  5. Reporting – You do not need to compile your findings into a formal report but rather use your rough notes on usability issues to revise the materials or start developing a prototype.

Test materials

To run usability testing in the office, you’ll need these test materials:

  • Moderator’s guide
  • Note taking guide

Download the Testing in the Office test materials

The test materials are contained in a zip folder. When you click the link above, your download should begin automatically, and the folder will be saved to your default download location.

Once you’ve downloaded the folder, you’ll need to extract the files.

  • To extract the files on a PC, right-click on the zip file and select Extract All. You’ll be prompted to choose a location. Do so, and then press Extract. Your computer will extract the files and create a new folder.
  • To extract the files on a Mac, you can simply double-click on the zip file, and your computer will extract the files into a new folder.

Now that the files are extracted, you can view the folder’s contents and use the materials. All documents are Microsoft Word .docx files.

Testing out of the office: catch real users in the field

Go to your public with a friendly invitation to help you design something by trying it out, right then and there.

You’re catching passers-by who are real (potential) users of your materials. You can get this done in anywhere from an hour to a half day, depending on your prep time and how heavily trafficked your location is.

When to use it

Use this approach if you’re looking for quick input on prototypes that people can try out. This test is a little more structured than just testing in the office, but it’s quicker and easier than the testing in the conference room approach.

Who to test with

Your participants will come from the public – and they’re more like your real users (than your colleagues usually are). For example, if you’re testing something for voters, you can ask people if they’re registered to vote and go from there.

Depending on the time you have available for each session, you can work with anywhere between 4 and 8 participants for this type of testing. You don’t need to pay your participants, but because you’re asking strangers to give you their time, you might consider giving a small token of appreciation — like a promotional item provided by your election office.

What you’ll need

  • A moderator
  • A prototype/early version of the new material you’re developing
  • Your name badge
  • Test materials (see below)

Where to do it

The idea here is to get out of your office and get into a place where there are a lot of people. That might be the foyer in your municipal or county building, a public library, a community center, or a street corner.

A moderator takes notes while discussing materials with a test participant

Usability testing “in the field” at a community event

Go to where you’re likely to find participants easily. If you have some flexibility with location, find a place where you’re likely to find people who are like your real users (for example, if you’re testing Spanish translations on a ballot, go to a public library in a Latino neighborhood).

How to do it

  1. Setting up – Set up/find a table at the location. Be sure to use signs and badges to tell people what you’re doing (“Help us improve our election materials!”).
  2. Recruiting – If you have an identification badge, this is the time to put it on, along with a big smile! Walk up to someone and ask if they could give you a few minutes of help. Ideally, you want to find participants who would typically use the materials you’re testing. Use the same testing materials with each participant.
  3. Testing – When they’ve agreed to help you out, have each participant complete the consent form and demographic questionnaire provided. Then, begin the usability test. Pay attention as each participant works their way through the material. Observe their behavior and make note of their comments as they goes along. When they’re finished with the tasks, ask them to tell you 2 good things about the materials and 2 things that could be improved.
  4. After the test – Review your notes and make sure you haven’t missed anything you wanted to observe or follow up on. Thank your participant for their time and give them a copy of the “thank you” card on the way out.
  5. Reporting – The format of the report depends on the purpose and stakeholders involved in the process. You could compile a short report on the test conducted or simply use your notes to iterate your design further.

Test materials

To run usability testing out of the office, you’ll need these test materials:

  • Moderator’s guide
  • Demographic questionnaire
  • Consent form template
  • Note taking guide
  • Table sign template
  • Thank you template

Download the Testing Out of the Office test materials

The test materials are contained in a zip folder. When you click the link above, your download should begin automatically, and the folder will be saved to your default download location.

Once you’ve downloaded the folder, you’ll need to extract the files.

  • To extract the files on a PC, right-click on the zip file and select Extract All. You’ll be prompted to choose a location. Do so, and then press Extract. Your computer will extract the files and create a new folder.
  • To extract the files on a Mac, you can simply double-click on the zip file, and your computer will extract the files into a new folder.

Now that the files are extracted, you can view the folder’s contents and use the materials. All documents are Microsoft Word .docx files.

Testing in the conference room: thorough, structured testing with real users

With just a little more planning and rigor, you can design a formal usability test to control for factors that you can’t control for if you’re testing in the field.

When you bring screened, scheduled participants into your conference room, you can also set up ways for your teammates and stakeholders to observe. Using this approach, testing will take a couple of days, with a day or two of planning and a day or two of synthesis and reporting.

When to use it

Use this approach if you want to conduct a more structured test with participants who are your actual users — not just your colleagues or people on the street. A full usability test can happen at any stage in the process, from testing the original version, testing drafts, or testing the almost-final version. This approach may or may not require a formal report on what you learned.

Who to test with

For this test, select participants specifically because they represent the intended audience for the material being tested. Screen them through phone conversations and formally schedule their sessions.

You should test with at least 4 participants per user group who will interact with your materials (e.g., 4 regular voters, 4 new voters, etc.). Full usability tests can be conducted with anywhere between 8 and 100 participants. Most of these usability tests include 8-20 participants in individual sessions that last between 30 and 90 minutes. Most will be about an hour long.

A note taker making observations and notes at a table

In-depth usability testing in the conference room setting

Depending on your resources, you may choose to compensate your participants for their time, especially if each test is a long session of 30 minutes or more.

What you’ll need

  • A moderator
  • A helper/note taker
  • A complete copy of the material to be tested
  • Your name badge
  • Test materials (see below)

In some instances, additional observers may also be included in the testing room. Or, you can easily set up a webcam to have stakeholders observe from another room.

Where to do it

Find a quiet place for testing (like a conference room in a library or your office) that will allow you to have substantial, uninterrupted conversations with each participant. If your test includes observers, make sure you’re testing in a room that’s big enough to comfortably accommodate everyone. Alternatively, you can also schedule sessions in a usability lab (two rooms divided by a glass panel to keep participants and observers separate).

How to do it

  1. Recruiting – You should recruit your participants in advance. Schedule each session for between 30 and 45 minutes to give you enough time to conduct the test, jot down notes, and get ready for the next participant in line. Make sure you have some time between each session. We recommend at least a half hour between each session.
  2. Setting up – Get to your testing location early so that you can set up the testing area. Using the test materials in the folder below, you’ll have a number of different forms to keep track of. Having them laid out prior to testing will ensure a smooth session. Make sure to use the moderator’s guide and note taking guide provided so that questions and answers can be systematically recorded.
  3. Testing – First, have each participant complete the consent form provided. You can have them fill out the demographic questionnaire provided before or after the test. Following your moderator’s guide, carefully observe how each participant interacts with the testing material as well as any comments/gestures they make during the process.
  4. After the test – Thank them for their time and give them a copy of the “thank you” card on the way out.
  5. Reporting – Using the report template provided, compile your results into a report, highlighting all stages of the usability test — i.e., participant data, testing locations, material tested, findings, and recommendations.

Test materials

To run usability testing in the conference room, you’ll need these test materials:

  • Moderator’s guide
  • Demographic questionnaire
  • Consent form template
  • Note taking guide
  • Participant receipt sheet
  • Table sign template
  • Thank you template
  • Report template

Download the Testing in the Conference Room test materials

The test materials are contained in a zip folder. When you click the link above, your download should begin automatically, and the folder will be saved to your default download location.

Once you’ve downloaded the folder, you’ll need to extract the files.

  • To extract the files on a PC, right-click on the zip file and select Extract All. You’ll be prompted to choose a location. Do so, and then press Extract. Your computer will extract the files and create a new folder.
  • To extract the files on a Mac, you can simply double-click on the zip file, and your computer will extract the files into a new folder.

Now that the files are extracted, you can view the folder’s contents and use the materials. All documents are Microsoft Word .docx files.


No matter which approach you take, a usability test will give you important information about how real users interact with your materials, helping you to make informed decisions about how to update or fix issues before you launch the materials out into the world.

Whether you’re new to usability testing or have some experience, we encourage you to make use of the resources and templates provided in this kit and give usability testing a try.

Think of it as a stepping stone to developing an intuitive voting experience!