4.3: What Is User Testing?
- Page ID
- 47374
\( \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)
\( \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} \)
\( \newcommand{\id}{\mathrm{id}}\) \( \newcommand{\Span}{\mathrm{span}}\)
( \newcommand{\kernel}{\mathrm{null}\,}\) \( \newcommand{\range}{\mathrm{range}\,}\)
\( \newcommand{\RealPart}{\mathrm{Re}}\) \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\)
\( \newcommand{\Argument}{\mathrm{Arg}}\) \( \newcommand{\norm}[1]{\| #1 \|}\)
\( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\)
\( \newcommand{\Span}{\mathrm{span}}\)
\( \newcommand{\id}{\mathrm{id}}\)
\( \newcommand{\Span}{\mathrm{span}}\)
\( \newcommand{\kernel}{\mathrm{null}\,}\)
\( \newcommand{\range}{\mathrm{range}\,}\)
\( \newcommand{\RealPart}{\mathrm{Re}}\)
\( \newcommand{\ImaginaryPart}{\mathrm{Im}}\)
\( \newcommand{\Argument}{\mathrm{Arg}}\)
\( \newcommand{\norm}[1]{\| #1 \|}\)
\( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\)
\( \newcommand{\Span}{\mathrm{span}}\) \( \newcommand{\AA}{\unicode[.8,0]{x212B}}\)
\( \newcommand{\vectorA}[1]{\vec{#1}} % arrow\)
\( \newcommand{\vectorAt}[1]{\vec{\text{#1}}} % arrow\)
\( \newcommand{\vectorB}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)
\( \newcommand{\vectorC}[1]{\textbf{#1}} \)
\( \newcommand{\vectorD}[1]{\overrightarrow{#1}} \)
\( \newcommand{\vectorDt}[1]{\overrightarrow{\text{#1}}} \)
\( \newcommand{\vectE}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash{\mathbf {#1}}}} \)
\( \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)
\( \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} \)
\(\newcommand{\avec}{\mathbf a}\) \(\newcommand{\bvec}{\mathbf b}\) \(\newcommand{\cvec}{\mathbf c}\) \(\newcommand{\dvec}{\mathbf d}\) \(\newcommand{\dtil}{\widetilde{\mathbf d}}\) \(\newcommand{\evec}{\mathbf e}\) \(\newcommand{\fvec}{\mathbf f}\) \(\newcommand{\nvec}{\mathbf n}\) \(\newcommand{\pvec}{\mathbf p}\) \(\newcommand{\qvec}{\mathbf q}\) \(\newcommand{\svec}{\mathbf s}\) \(\newcommand{\tvec}{\mathbf t}\) \(\newcommand{\uvec}{\mathbf u}\) \(\newcommand{\vvec}{\mathbf v}\) \(\newcommand{\wvec}{\mathbf w}\) \(\newcommand{\xvec}{\mathbf x}\) \(\newcommand{\yvec}{\mathbf y}\) \(\newcommand{\zvec}{\mathbf z}\) \(\newcommand{\rvec}{\mathbf r}\) \(\newcommand{\mvec}{\mathbf m}\) \(\newcommand{\zerovec}{\mathbf 0}\) \(\newcommand{\onevec}{\mathbf 1}\) \(\newcommand{\real}{\mathbb R}\) \(\newcommand{\twovec}[2]{\left[\begin{array}{r}#1 \\ #2 \end{array}\right]}\) \(\newcommand{\ctwovec}[2]{\left[\begin{array}{c}#1 \\ #2 \end{array}\right]}\) \(\newcommand{\threevec}[3]{\left[\begin{array}{r}#1 \\ #2 \\ #3 \end{array}\right]}\) \(\newcommand{\cthreevec}[3]{\left[\begin{array}{c}#1 \\ #2 \\ #3 \end{array}\right]}\) \(\newcommand{\fourvec}[4]{\left[\begin{array}{r}#1 \\ #2 \\ #3 \\ #4 \end{array}\right]}\) \(\newcommand{\cfourvec}[4]{\left[\begin{array}{c}#1 \\ #2 \\ #3 \\ #4 \end{array}\right]}\) \(\newcommand{\fivevec}[5]{\left[\begin{array}{r}#1 \\ #2 \\ #3 \\ #4 \\ #5 \\ \end{array}\right]}\) \(\newcommand{\cfivevec}[5]{\left[\begin{array}{c}#1 \\ #2 \\ #3 \\ #4 \\ #5 \\ \end{array}\right]}\) \(\newcommand{\mattwo}[4]{\left[\begin{array}{rr}#1 \amp #2 \\ #3 \amp #4 \\ \end{array}\right]}\) \(\newcommand{\laspan}[1]{\text{Span}\{#1\}}\) \(\newcommand{\bcal}{\cal B}\) \(\newcommand{\ccal}{\cal C}\) \(\newcommand{\scal}{\cal S}\) \(\newcommand{\wcal}{\cal W}\) \(\newcommand{\ecal}{\cal E}\) \(\newcommand{\coords}[2]{\left\{#1\right\}_{#2}}\) \(\newcommand{\gray}[1]{\color{gray}{#1}}\) \(\newcommand{\lgray}[1]{\color{lightgray}{#1}}\) \(\newcommand{\rank}{\operatorname{rank}}\) \(\newcommand{\row}{\text{Row}}\) \(\newcommand{\col}{\text{Col}}\) \(\renewcommand{\row}{\text{Row}}\) \(\newcommand{\nul}{\text{Nul}}\) \(\newcommand{\var}{\text{Var}}\) \(\newcommand{\corr}{\text{corr}}\) \(\newcommand{\len}[1]{\left|#1\right|}\) \(\newcommand{\bbar}{\overline{\bvec}}\) \(\newcommand{\bhat}{\widehat{\bvec}}\) \(\newcommand{\bperp}{\bvec^\perp}\) \(\newcommand{\xhat}{\widehat{\xvec}}\) \(\newcommand{\vhat}{\widehat{\vvec}}\) \(\newcommand{\uhat}{\widehat{\uvec}}\) \(\newcommand{\what}{\widehat{\wvec}}\) \(\newcommand{\Sighat}{\widehat{\Sigma}}\) \(\newcommand{\lt}{<}\) \(\newcommand{\gt}{>}\) \(\newcommand{\amp}{&}\) \(\definecolor{fillinmathshade}{gray}{0.9}\)USER TESTING AND RESEARCH BASICS
User testing and research focus on understanding user behaviors, needs, and motivations through observation techniques, task analysis, and other feedback methodologies. Mike Kuniaysky, a specialist in User Experience, notes that it is “the process of understanding the impact of design on an audience.”
The types of user research you can or should perform will depend on the type of site, system, or app you are developing, your timeline, and your environment.
WHEN TO PERFORM USER RESEARCH METHODS
Below are examples of the types of research you could perform at each phase of your project. This information is guided by the user-centered design (UCD) process. A methodology may appear in one or in several phases.
| Methods | Know Your User | Content | Design | Test & Refine |
| Card Sorting - This method helps ensure that the site structure matches the way users think by allowing users to group your site’s information. |
YES |
YES |
NO |
YES |
| Contextual Interviews - A method that enables you to observe users in their natural environment, giving you a better understanding of the way users work. |
YES |
NO |
NO |
NO |
| First Click Testing - A testing method focused on navigation, which can be performed on a functioning website, a prototype, or a wireframe. |
NO |
YES |
YES |
YES |
| Focus Groups - A moderated discussion with a group of users that allows you to learn about user attitudes, ideas, and desires. |
YES |
YES |
YES |
NO |
| Heuristic Evaluation/Expert Review - A group of usability experts evaluating your website against a list of established guidelines. |
YES |
NO |
NO |
YES |
| Individual Interviews - One-on-one discussions with users show you how a particular user works. These discussions enable you to get detailed information about a user's attitudes, desires, and experiences. |
YES |
YES |
YES |
YES |
| Parallel Design - A design methodology that involves several designers pursuing the same effort simultaneously, but independently, with the intention to combine the best aspects of each for the ultimate solution. |
NO |
NO |
YES |
NO |
| Personas - The creation of a representative user based on available data and user interviews. Though the personal details of the persona may be fiction, the information used to create the user type is not. |
YES |
NO |
NO |
NO |
| Prototyping - Creation of a mock-up of the site, which allows the design team to explore ideas before implementing them. A prototype can range from a paper mock-up to interactive HTML pages. |
NO |
NO |
YES |
YES |
| Surveys - A series of questions asked to multiple users of your website, which helps you learn about the people who visit your site. |
YES |
YES |
YES |
YES |
| System Usability Scale (SUS) - SUS is a technology-independent ten-item scale for subjective evaluation of the usability. |
NO |
NO |
NO |
YES |
| Task Analysis - A method where you learn about user goals, including what users want to do on your website; this method helps you understand the tasks that users will perform on your site. |
YES |
NO |
NO |
NO |
| Usability Testing - One-on-one sessions where a "real-life" user performs tasks on your site. This method identifies user frustrations and problems with your site. |
YES |
YES |
YES |
YES |
| Use Cases - A description of how users use a particular feature of your website. This method provides a detailed look at how users interact with the site, including the steps users take to accomplish each task. |
NO |
YES |
YES |
NO |
BEST PRACTICES
During Project Planning you should do the following:
- Consult the UCD Guide for a step-by-step visual map to guide you through the user-centered design process.
- Consider when testing can be incorporated in your schedule
- Test early and often
- Define the target audience(s) for the site. This will assist the UX team in the following:
RECRUITING PARTICIPANTS & THE LEGEND OF “THE GENERAL PUBLIC”
If you have ever tried to put together any user experience testing, especially for the government, at one point or another your client may have said that their target audience is the “general public.” Your conversations may have sounded something like this:
Question: “Who is the target audience for this site?”
Answer: “The General Public”
Question: “Who would you like us to bring in for usability testing?”
Answer: “The General Public”
Question: “Who should be involved in this card sort?
Answer: “The General Public”
Unfortunately, many believe the “general public” is well suited for any test, survey, or focus group. The thing is that using the non-descript “general public” isn’t helpful because it doesn’t exist. So your critical task in recruiting for tests – and also in designing – is to get more specific and identify what your team really means when they say the “general public.”
Knowing who your target audiences are and identifying their top tasks is a critical part of designing your site and forming an effective content strategy. This audience analysis can be done through various market research, user research, and web analytics techniques. When it comes to testing, being able to draw on those established audience groups and recruiting representative participants is fundamental to yielding useful results and the acceptance of your insights and recommendations.
IDENTIFYING YOUR PARTICIPANTS
When identifying test participants, create a list of the traits, characteristics, or experience you will need in a participant based on metrics or discussions with your client or team. This information will help you develop the recruiting criteria and screener you will use to recruit participants.
At a high level, here are some of the things your recruiting criteria and screener should cover:
|
Age Range |
|
|
Gender |
|
|
Ethnicity |
|
|
Education |
|
|
Language |
|
|
Familiarity |
|
|
Previous Usage |
|
|
Technology qualifications |
|
|
Internet |
|
| Special qualifications |
|
Here are a few example templates available for you to adapt for your own use:
KNOWING WHERE TO LOOK
Once you identify who you are testing, you must address where you can find your participants. In many cases, the list you generate will dictate the location for your search and, in some cases, your test methodology as well.
USING AN AFFILIATION TO FIND PARTICIPANTS
Having an affiliation with an organization can help you find participants that meet your criteria. For example, if your recruiting criterion includes retired service personnel who assist returning veterans blinded during service, it may look something like:
- Level of the desired familiarity with the service, subject or technology
- They must be familiar with the site
- Previous usage of the service, subject or technology
- Weekly visits to the site in question
- Technical qualifications
- Must use a screen reader
There is a likelihood that these participants are affiliated with the VA, are not centrally located geographically, and are visually impaired. Therefore, these circumstances would encourage the following:
- Recruiting through the VA
- Testing remotely to eliminate the need for the cost of travel
- Allowing participants to use their own equipment
LICENSES AND ATTRIBUTIONS
- “User Testing” by Usability.gov (U.S. Dept. of Health and Human Services). The Research-Based Web Design & Usability Guidelines, Enlarged/Expanded edition. Washington: U.S. Government Printing Office, 2006. (Unless otherwise noted, text, documents, and images on the Usability.gov website are in the public domain, are not copyrighted, and therefore may be copied and distributed at no cost. https://www.usability.gov/about-us/index.html)
Reference: Observing the User Experience: A Practitioner's Guide for User Research by Mike Kuniavsky