4.3: What Is User Testing?

Last updated

Aug 12, 2020
Save as PDF
- 4.2: What Is Project Management?
- 4.4: Why and How Do You Use a Team Contract?

Chelsea Milbourne, Anne Regan, Morgan Livingston, & Sadie Johann
California Polytechnic State University-San Luis Obispo

$\newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} }$

$\newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}}$

$\newcommand{\id}{\mathrm{id}}$ $\newcommand{\Span}{\mathrm{span}}$

( \newcommand{\kernel}{\mathrm{null}\,}\) $\newcommand{\range}{\mathrm{range}\,}$

$\newcommand{\RealPart}{\mathrm{Re}}$ $\newcommand{\ImaginaryPart}{\mathrm{Im}}$

$\newcommand{\Argument}{\mathrm{Arg}}$ $\newcommand{\norm}[1]{\| #1 \|}$

$\newcommand{\inner}[2]{\langle #1, #2 \rangle}$

$\newcommand{\Span}{\mathrm{span}}$

$\newcommand{\id}{\mathrm{id}}$

$\newcommand{\Span}{\mathrm{span}}$

$\newcommand{\kernel}{\mathrm{null}\,}$

$\newcommand{\range}{\mathrm{range}\,}$

$\newcommand{\RealPart}{\mathrm{Re}}$

$\newcommand{\ImaginaryPart}{\mathrm{Im}}$

$\newcommand{\Argument}{\mathrm{Arg}}$

$\newcommand{\norm}[1]{\| #1 \|}$

$\newcommand{\inner}[2]{\langle #1, #2 \rangle}$

$\newcommand{\Span}{\mathrm{span}}$ $\newcommand{\AA}{\unicode[.8,0]{x212B}}$

$\newcommand{\vectorA}[1]{\vec{#1}} % arrow$

$\newcommand{\vectorAt}[1]{\vec{\text{#1}}} % arrow$

$\newcommand{\vectorB}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} }$

$\newcommand{\vectorC}[1]{\textbf{#1}}$

$\newcommand{\vectorD}[1]{\overrightarrow{#1}}$

$\newcommand{\vectorDt}[1]{\overrightarrow{\text{#1}}}$

$\newcommand{\vectE}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash{\mathbf {#1}}}}$

$\newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} }$

$\newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}}$

$\newcommand{\avec}{\mathbf a}$

$\newcommand{\bvec}{\mathbf b}$

$\newcommand{\cvec}{\mathbf c}$

$\newcommand{\dvec}{\mathbf d}$

$\newcommand{\dtil}{\widetilde{\mathbf d}}$

$\newcommand{\evec}{\mathbf e}$

$\newcommand{\fvec}{\mathbf f}$

$\newcommand{\nvec}{\mathbf n}$

$\newcommand{\pvec}{\mathbf p}$

$\newcommand{\qvec}{\mathbf q}$

$\newcommand{\svec}{\mathbf s}$

$\newcommand{\tvec}{\mathbf t}$

$\newcommand{\uvec}{\mathbf u}$

$\newcommand{\vvec}{\mathbf v}$

$\newcommand{\wvec}{\mathbf w}$

$\newcommand{\xvec}{\mathbf x}$

$\newcommand{\yvec}{\mathbf y}$

$\newcommand{\zvec}{\mathbf z}$

$\newcommand{\rvec}{\mathbf r}$

$\newcommand{\mvec}{\mathbf m}$

$\newcommand{\zerovec}{\mathbf 0}$

$\newcommand{\onevec}{\mathbf 1}$

$\newcommand{\real}{\mathbb R}$

$\newcommand{\twovec}[2]{\left[\begin{array}{r}#1 \\ #2 \end{array}\right]}$

$\newcommand{\ctwovec}[2]{\left[\begin{array}{c}#1 \\ #2 \end{array}\right]}$

$\newcommand{\threevec}[3]{\left[\begin{array}{r}#1 \\ #2 \\ #3 \end{array}\right]}$

$\newcommand{\cthreevec}[3]{\left[\begin{array}{c}#1 \\ #2 \\ #3 \end{array}\right]}$

$\newcommand{\fourvec}[4]{\left[\begin{array}{r}#1 \\ #2 \\ #3 \\ #4 \end{array}\right]}$

$\newcommand{\cfourvec}[4]{\left[\begin{array}{c}#1 \\ #2 \\ #3 \\ #4 \end{array}\right]}$

$\newcommand{\fivevec}[5]{\left[\begin{array}{r}#1 \\ #2 \\ #3 \\ #4 \\ #5 \\ \end{array}\right]}$

$\newcommand{\cfivevec}[5]{\left[\begin{array}{c}#1 \\ #2 \\ #3 \\ #4 \\ #5 \\ \end{array}\right]}$

$\newcommand{\mattwo}[4]{\left[\begin{array}{rr}#1 \amp #2 \\ #3 \amp #4 \\ \end{array}\right]}$

$\newcommand{\laspan}[1]{\text{Span}\{#1\}}$

$\newcommand{\bcal}{\cal B}$

$\newcommand{\ccal}{\cal C}$

$\newcommand{\scal}{\cal S}$

$\newcommand{\wcal}{\cal W}$

$\newcommand{\ecal}{\cal E}$

$\newcommand{\coords}[2]{\left\{#1\right\}_{#2}}$

$\newcommand{\gray}[1]{\color{gray}{#1}}$

$\newcommand{\lgray}[1]{\color{lightgray}{#1}}$

$\newcommand{\rank}{\operatorname{rank}}$

$\newcommand{\row}{\text{Row}}$

$\newcommand{\col}{\text{Col}}$

$\renewcommand{\row}{\text{Row}}$

$\newcommand{\nul}{\text{Nul}}$

$\newcommand{\var}{\text{Var}}$

$\newcommand{\corr}{\text{corr}}$

$\newcommand{\len}[1]{\left|#1\right|}$

$\newcommand{\bbar}{\overline{\bvec}}$

$\newcommand{\bhat}{\widehat{\bvec}}$

$\newcommand{\bperp}{\bvec^\perp}$

$\newcommand{\xhat}{\widehat{\xvec}}$

$\newcommand{\vhat}{\widehat{\vvec}}$

$\newcommand{\uhat}{\widehat{\uvec}}$

$\newcommand{\what}{\widehat{\wvec}}$

$\newcommand{\Sighat}{\widehat{\Sigma}}$

$\newcommand{\lt}{<}$

$\newcommand{\gt}{>}$

$\newcommand{\amp}{&}$

$\definecolor{fillinmathshade}{gray}{0.9}$

USER TESTING AND RESEARCH BASICS

User testing and research focus on understanding user behaviors, needs, and motivations through observation techniques, task analysis, and other feedback methodologies. Mike Kuniaysky, a specialist in User Experience, notes that it is “the process of understanding the impact of design on an audience.”

The types of user research you can or should perform will depend on the type of site, system, or app you are developing, your timeline, and your environment.

WHEN TO PERFORM USER RESEARCH METHODS

Below are examples of the types of research you could perform at each phase of your project. This information is guided by the user-centered design (UCD) process. A methodology may appear in one or in several phases.

Table $\PageIndex{1}$
Methods	Know Your User	Content	Design	Test & Refine
Card Sorting - This method helps ensure that the site structure matches the way users think by allowing users to group your site’s information.	YES	YES	NO	YES
Contextual Interviews - A method that enables you to observe users in their natural environment, giving you a better understanding of the way users work.	YES	NO	NO	NO
First Click Testing - A testing method focused on navigation, which can be performed on a functioning website, a prototype, or a wireframe.	NO	YES	YES	YES
Focus Groups - A moderated discussion with a group of users that allows you to learn about user attitudes, ideas, and desires.	YES	YES	YES	NO
Heuristic Evaluation/Expert Review - A group of usability experts evaluating your website against a list of established guidelines.	YES	NO	NO	YES
Individual Interviews - One-on-one discussions with users show you how a particular user works. These discussions enable you to get detailed information about a user's attitudes, desires, and experiences.	YES	YES	YES	YES
Parallel Design - A design methodology that involves several designers pursuing the same effort simultaneously, but independently, with the intention to combine the best aspects of each for the ultimate solution.	NO	NO	YES	NO
Personas - The creation of a representative user based on available data and user interviews. Though the personal details of the persona may be fiction, the information used to create the user type is not.	YES	NO	NO	NO
Prototyping - Creation of a mock-up of the site, which allows the design team to explore ideas before implementing them. A prototype can range from a paper mock-up to interactive HTML pages.	NO	NO	YES	YES
Surveys - A series of questions asked to multiple users of your website, which helps you learn about the people who visit your site.	YES	YES	YES	YES
System Usability Scale (SUS) - SUS is a technology-independent ten-item scale for subjective evaluation of the usability.	NO	NO	NO	YES
Task Analysis - A method where you learn about user goals, including what users want to do on your website; this method helps you understand the tasks that users will perform on your site.	YES	NO	NO	NO
Usability Testing - One-on-one sessions where a "real-life" user performs tasks on your site. This method identifies user frustrations and problems with your site.	YES	YES	YES	YES
Use Cases - A description of how users use a particular feature of your website. This method provides a detailed look at how users interact with the site, including the steps users take to accomplish each task.	NO	YES	YES	NO

BEST PRACTICES

During Project Planning you should do the following:

Consult the UCD Guide for a step-by-step visual map to guide you through the user-centered design process.
Consider when testing can be incorporated in your schedule
- Test early and often
Define the target audience(s) for the site. This will assist the UX team in the following:
- Creating a screener
- Recruiting participants
- Defining Personas
- Identifying tasks for testing

RECRUITING PARTICIPANTS & THE LEGEND OF “THE GENERAL PUBLIC”

If you have ever tried to put together any user experience testing, especially for the government, at one point or another your client may have said that their target audience is the “general public.” Your conversations may have sounded something like this:

Question: “Who is the target audience for this site?”

Answer: “The General Public”

Question: “Who would you like us to bring in for usability testing?”

Answer: “The General Public”

Question: “Who should be involved in this card sort?

Answer: “The General Public”

Unfortunately, many believe the “general public” is well suited for any test, survey, or focus group. The thing is that using the non-descript “general public” isn’t helpful because it doesn’t exist. So your critical task in recruiting for tests – and also in designing – is to get more specific and identify what your team really means when they say the “general public.”

Knowing who your target audiences are and identifying their top tasks is a critical part of designing your site and forming an effective content strategy. This audience analysis can be done through various market research, user research, and web analytics techniques. When it comes to testing, being able to draw on those established audience groups and recruiting representative participants is fundamental to yielding useful results and the acceptance of your insights and recommendations.

IDENTIFYING YOUR PARTICIPANTS

When identifying test participants, create a list of the traits, characteristics, or experience you will need in a participant based on metrics or discussions with your client or team. This information will help you develop the recruiting criteria and screener you will use to recruit participants.

At a high level, here are some of the things your recruiting criteria and screener should cover:

Table $\PageIndex{2}$
Age Range	What are the age ranges? How many do you need in each range?
Gender	Gender type? How many of each gender?
Ethnicity	Which ethnicities? How many of each ethnicity?
Education	Level(s) (Examples: high school or GED degree, business or trade school, associates degree, bachelor's degree, master's or doctorate) Distribution at each education level
Language	Primary or target language
Familiarity	Level of the desired familiarity with the service, subject, or technology
Previous Usage	Previous usage of the service, subject, or technology
Technology qualifications	The use of specific hardware, software, or devices
Internet	Level/frequency of internet use
Special qualifications	Job history Health history (including past tobacco use) Personal history Living environment Family environment Children Marital Status

Here are a few example templates available for you to adapt for your own use:

KNOWING WHERE TO LOOK

Once you identify who you are testing, you must address where you can find your participants. In many cases, the list you generate will dictate the location for your search and, in some cases, your test methodology as well.

USING AN AFFILIATION TO FIND PARTICIPANTS

Having an affiliation with an organization can help you find participants that meet your criteria. For example, if your recruiting criterion includes retired service personnel who assist returning veterans blinded during service, it may look something like:

Level of the desired familiarity with the service, subject or technology
- They must be familiar with the site
Previous usage of the service, subject or technology
- Weekly visits to the site in question
Technical qualifications
- Must use a screen reader

There is a likelihood that these participants are affiliated with the VA, are not centrally located geographically, and are visually impaired. Therefore, these circumstances would encourage the following:

Recruiting through the VA
Testing remotely to eliminate the need for the cost of travel
Allowing participants to use their own equipment

LICENSES AND ATTRIBUTIONS

“User Testing” by Usability.gov (U.S. Dept. of Health and Human Services). The Research-Based Web Design & Usability Guidelines, Enlarged/Expanded edition. Washington: U.S. Government Printing Office, 2006. (Unless otherwise noted, text, documents, and images on the Usability.gov website are in the public domain, are not copyrighted, and therefore may be copied and distributed at no cost. https://www.usability.gov/about-us/index.html)
Reference: Observing the User Experience: A Practitioner's Guide for User Research by Mike Kuniavsky

Search

Text Color

Text Size

Margin Size

Font Type

USER TESTING AND RESEARCH BASICS

WHEN TO PERFORM USER RESEARCH METHODS

BEST PRACTICES

RECRUITING PARTICIPANTS & THE LEGEND OF “THE GENERAL PUBLIC”

IDENTIFYING YOUR PARTICIPANTS

KNOWING WHERE TO LOOK

USING AN AFFILIATION TO FIND PARTICIPANTS

LICENSES AND ATTRIBUTIONS

Support Center

How can we help?