9Methods in the Making:
A Perspective on the State of
Human Research in Design
Bruce Hanington
Introduction
This article was developed from experiences in human-centered
design, both within field research and as a design researcher and
educator. Several of the observations, insights, and examples offer-
ed here have been inspired, or at least clarified, by a current proj-
ect being conducted by the School of Design at Carnegie Mellon
University for the United States Postal Service (USPS). The USPS
project entails the transformation of complex informational docu-
ments into accessible language and visualizations in a new set of
documents for use by postal employees, the public, and business
customers. My role in this project has been to advise the research and
design team on user research and product testing, given a mandate
of user-centered design.
This project is noteworthy from several perspectives relevant
to this article, and design research in general. First, there is the
unique aspect of application focused on the design of an informa-
tional document. Although certainly arguable as an interface, there
is a perceived difference between this product and more traditional
interfaces housed in three-dimensional and digital artifacts. Related
to this is the recognized paucity of user-centered design and testing
within communication (graphic) design,1 particularly in comparison
to the more established history of industrial design. Fundamental
to my own background in human factors and industrial design has
been the realization that although one can identify these differences,
they become relatively mute in the process of research and design.
That is, the issues that emerge, with respect to both content and
methodology, are relatively similar in practice, and in fact should be
mutually informing across disciplines and products.
The information shared here, culled from the USPS project
and others, should serve to reinforce the need and demand for user-
centered approaches in design, and offer some clarity in the methods
that can best serve this cause.
The Language of Human-Centered Design
The very phrase user-centered design is worth contemplating at the
outset, noteworthy at least for the absence of the word “research.”
User-centered design describes a process, one that is at once both
1 See, for example, the argument put forth
by Strickler regarding suspect reliance
by graphic designers on “specialist”
design intuition. Zoe Strickler, “Elicitation
Methods in Experimental Design
Research,” Design Issues 15:2 (Summer,
1999): 28.
© Copyright 2003 Massachusettes Institute of Technology
Design Issues: Volume 19, Number 4 Autumn 2003
03 Hanington 9/15/03, 9:21 PM9

Design Issues: Volume 19, Number 4 Autumn 200310 Design Issues: Volume 19, Number 4 Autumn 2003 11
human- and design-centric. Research, in this case, is implicit, yet is
addressed within the context of design. Design, in turn, is recognized
as an activity inherently tied to human needs and concerns. For this
reason, I would argue for further clarity and humanizing of the
phrase by calling it human-centered design.
I offer in contrast the more traditional terminology of user test-
ing, and its counterpart, usability. There is a growing body of design
literature critical of the limited connotations of these terms, both in
definition and practice.2 On one level, user testing may be miscon-
strued as implying a test of the user, certainly something we strive to
de-emphasize to participants in human factors research! In response,
a more accurate descriptive term would be product testing.
Furthermore, if we examine the activities of research at any
given time in the life of a project, the term user testing is, in fact,
a misnomer. The phrase implies that a product (or artifact, be it a
prototype, manufactured object, or document) has an informational
set to be matched (tested) against user (human) interpretation. In
many stages of a design project, user research offers a more appropri-
ate description of the activities actually taking place. For example,
when we are collecting information from people to inform our
baseline knowledge of their needs, desires, or thought processes, we
are engaged in user research. User research may entail interviews,
conversations, business or facility tours, the examination of currently
used documents or products, and work observations, as well as
documentation through writing, sketching, and photography.
Sometimes, it is also relevant to distinguish between users and
tasks.3 Whereas user research reveals aspects of people as described
above, tasks often are isolated for research in terms of how goals are
accomplished, pathways of experience, milestones and roadblocks
to achievement. Eventually, aspects of users and tasks are mapped
together.
Finally, user testing and usability often too narrowly define
the range of human concerns of interest to design. This too is increas-
ingly documented in current design research literature, with clear
trends identifying the need to address aspects of product desirability,
pleasurable interactions, and emotional resonance, in addition to the
more established elements of product design centered around what
is useful and usable.4
Project Life Cycles and Research
Past models of user testing and usability consulted users in late-
stage product development, primarily for evaluating prototypes or
finished products. There is a growing argument to include people
in the very early stages of design, including pre-ideation phases.5
In agreement with this, I advocate that, in the life of longer-term
projects, a roster of stakeholders be built with agreement for partici-
pation at various stages throughout product development. This part-
nership results in an ongoing relationship, whereby relevant people
2 Bruce Hanington, “Innovation and
Method in Design Research” in Silvia
Pizzocaro, Amilton Arruda, and Dijon
De Moraes, eds., Proceedings of the
Politecnico di Milano Conference, Design
(plus) Research (May, 18-20, 2000): 64–
69. See also Patrick Jordan, Designing
Pleasurable Products: An Introduction to
the New Human Factors (London: Taylor
& Francis, 2000).
3 JoAnn T. Hackos and Janice C. Redish,
User and Task Analysis for Interface
Design (New York: John Wiley & Sons,
1998).
4 This argument currently is being
promoted primarily in conference
forums and accompanying proceed-
ings. For example, Martin Helander,
Halimahtun Khalid, and Tham Ming Po,
eds., Proceedings of the International
Conference on Affective Human Factors
Design (CAHD), Singapore, June 27–29,
2001 (Asean Academic Press), and the
Third International Conference on Design
and Emotion, Loughborough, England,
July 13, 2002 (proceedings forthcoming).
5 This view is supported by Liz Sanders of
SonicRim, among others (in presentation,
Carnegie Mellon University School of
Design, February 12, 2001).
03 Hanington 9/15/03, 9:21 PM10

Design Issues: Volume 19, Number 4 Autumn 200310 Design Issues: Volume 19, Number 4 Autumn 2003 11
may be called upon to assist in both the generation and evaluation
of concepts and solutions, while concurrently becoming invested in
the project.
Particularly at the beginning of a project, when the user
group and its tasks are unknown to the design team, it is critical for
members to immerse themselves in the user’s world to develop a
functional literacy of the material with which they will be working.
User research, as contrasted to user testing above, is appropriate
here.
Initially, speculative scenarios may be used to test ideas of
product engagement and use. These are hypothetical scenarios of
use determined by the design research team, to pilot-test possible
issues in interpretation or navigation, while simultaneously provid-
ing a check of research protocol. Once detail is collected through
user research, actual scenarios may evolve for more specific product
testing.
During early development, prototype reviews may be con-
ducted with users or experts to probe for confirmation of design
directions established from earlier research. This should not be mis-
construed as user testing. In transforming the USPS manuals, for
example, document reviews were used with a second prototype as
probing confirmation of appropriate content, topics, and sequence
of information. The prototype had enough fidelity to present it to
users, yet it was premature to test specific content. Reviews were
conducted both with business customers, and “experts” within the
USPS. A typical protocol for this research would involve members
of the design team asking questions on common information needs
and scenarios of use, presenting the document and its general struc-
ture, and then asking for feedback on the prototype based on typ-
ical experiences of the user. The table of contents is put under par-
ticular scrutiny for logic of information flow, and the index is exam-
ined and supplemented by users for the comprehensive inclusion
of key terms. While these sessions are conducted with design team
members in person, in some cases, we may leave the document pro-
totype with users for longer periods of time and conduct follow-up
sessions for feedback.
At later stages of prototype development, more traditional
product testing provides critical information. In the USPS project,
document testing is carried out to evaluate successful elements and
trouble spots in document content and navigation. These tests are
slightly more formal than earlier phases of research, and involve the
users going through the document using actual scenarios, thinking
out loud to pinpoint decision-making issues, annotating the docu-
ment with color-coded dots and written comments, and answer-
ing probing questions. These sessions typically are videotaped to
provide a transcript of the session and to identify key observations
of behavior. For convenience, we may conduct these tests in our
own facilities; however, it is valuable to collect feedback in the actual
03 Hanington 9/15/03, 9:21 PM11

Design Issues: Volume 19, Number 4 Autumn 200312 Design Issues: Volume 19, Number 4 Autumn 2003 13
work setting, under real circumstances of use, wherever possible.
For complex and lengthy documents, it will be necessary to conduct
some tests of individual components or sections of the document
only, and to conduct other tests where we again leave the prototype
with users for extended periods of time, with follow-up sessions to
elaborate on feedback.
The process used in the USPS project is thus described
in terms of human-centered design, and includes the following
elements of research, some of them iterative:
• User Research—early, baseline collection of information
• Speculative Scenarios—preliminary scenarios of use built
from baseline information
• Pilot Testing—in-house testing of content, and research
protocol
• Product (Document) Reviews—expert and user reviews of
document
• Product (Document) Testing—testing of prototypes with
users and experts.
Research in the early phases of a design project often is referred
to as generative, formative, or discovery research, and generally
is contrasted to evaluative research, typically positioned as an
end-stage component of research. User participation in generative
research can provide critical information in understanding users,
and their needs and desires, but also can be invaluable in developing
ideas for product features and forms. There often is a false distinction
made between methods reserved for generative research, and those
for evaluative research. While purposes may be different, there can
be significant crossover in the application of methods and, in fact,
multiple iterations of form (concept) generation and evaluation
should be cyclical and mutually informing.
Method and Purpose
It is clear that there is a vast inventory of research methods from
which to choose. The key challenge lies in making an appropriate,
purposive connection to goals in the selection of methods used at any
given time in the design and research process.
Consider the array of methods offered in Table 1.
03 Hanington 9/15/03, 9:21 PM12

Design Issues: Volume 19, Number 4 Autumn 200312 Design Issues: Volume 19, Number 4 Autumn 2003 13
Table 1
A Nomenclature of Research Methods for Human-Centered Design
Traditional Adapted Innovative
Market research
Focus groups
Surveys
Questionnaires
Interviews
Unobtrusive measures
Archival methods
Trace measures
Experiments
Observational research
Participant observation
Still, video documentation
Ethnographic methods
Video ethnography
Beeper studies
Experiential sampling
Cultural inventory
Artifact analysis
HCI
Thinkaloud protocol
Heuristic evaluation
Cognitive walkthrough
Creative/Participatory
Design workshops
Collage
Card sorting
Cognitive mapping
Velcro modeling
Visual diaries
Camera studies
Document annotations
Interpretation and analysis tends toward:
Counts
Statistics
Spreadsheets
Graphing
Verbal + numerical information
Content analysis
Categories
Patterns, Themes
Affinities, Clusters
Visual + verbal information
Traditional Methods
There are many traditional research methods that serve their
purpose well, with little need to reinvent them for each intended
use. Surveys, interviews, questionnaires, and focus groups—the
traditional purview of market research—provide an efficient means
to reach large numbers of people. If structured effectively, data
collected, particularly from surveys and questionnaires, may be
easily compiled, analyzed, and visualized. However, the methods
are open to criticism, particularly for their reliance on what people
say to be true, often subject to the influence of self-report bias or the
natural tendency to make oneself appear “good.” 6 Focus groups
must be well facilitated to avoid bias introduced through peer pres-
sure unwittingly exerted by other participants or, in some cases,
by the researchers themselves. These methods tend to be better at
confirming known entities, yet are less critical in determining as-yet-
undiscovered information.
Archival and “trace” measures similarly rely on interpre-
tations of existing artifacts, yet still are valuable for their original
purpose of unobtrusiveness, intended to reduce researcher bias
and the reactivity of research participants. Archival research may
range from library records to historical files to documented process
work; traces are those measures made evident through accretion
or erosion.7 For example, a document that has been sectioned, re-
6 “As Agnew and Pyke (1982) put it, ‘On
a questionnaire, we only have to move
the pencil a few inches to shift our
scores from being a bigot to being a
humanitarian...,’” in Colin Robson, Real
World Research: A Resource for Social
Scientists and Practitioner-Researchers,
2nd ed. (Oxford: Blackwell, 2002), 310.
7 The landmark source on unobtrusive
measures remains the classic by Eugene
Webb, Donald T. Campbell, et. al.,
Unobtrusive Measures: Nonreactive
Research in the Social Sciences (revised
edition by Corwin Press, Sage Classics
1999; original publication by Rand
McNally, 1966).
03 Hanington 9/15/03, 9:21 PM13

Design Issues: Volume 19, Number 4 Autumn 200314 Design Issues: Volume 19, Number 4 Autumn 2003 15
sequenced, and flagged in key places by the user offers substantial
information to the designer during research.
The experiment as a research strategy rarely is used by
designers, yet several intentions behind it serve to provide lessons
of good practice for all research. For instance, the experiment draws
attention to tradeoffs made between control and realism, and argues
for rigor in research protocol. Lab research and protocols developed
to isolate variables for manipulation and measurement provide the
assurance of control, yet what field research lacks in control it may
gain in realism, which is an advantage over many laboratory studies.
Within my own teaching and consulting, the experiment is studied
as a foundational tool for critical insights in both planning and
evaluating research, assessing when and why control over variables
is necessary and appropriate, and determining a suitable balance
between rigor and relevance.
Adapted Methods
It makes sense that we would borrow established methods from
disciplines engaged in human research, since design is fundamen-
tally a human-centered activity. However, research professions often
have purposes and goals that differ from those of design. For this
reason, methods borrowed often must be adapted to better suit the
needs of design.8
Observation methods have previously been borrowed from
psychology by the human factors community and subsequently
used by design, thereby giving them a laboratory model of scien-
tific application. The growing consensus that the use of designed
artifacts occurs in natural settings of work, home, and play has
convinced many that human behavior therefore should be studied in
context. This has forged an increasingly greater connection with the
philosophy and methods of anthropology and ethnography, fields
acknowledged for their sensitivity to the study of human communi-
ties, while maintaining an awareness of the dangers of subjectivity,
researcher bias, and influence.
Methods borrowed may be appropriate for our needs in
design, yet it is equally important to recognize that we have adapted
them for our own purposes. For example, ethnographic methods
in anthropology may demand months or even years on behalf of
the researcher, who will spend time in a community with varying
levels of participation during their observations. Adapted methods
commonly used in design include so-called “beeper studies,” or
Experiential Sampling Methods (ESM), whereby people are paged at
various times of the day to record their behavior, product use, and/
or feelings, and video ethnography, where continuous video monitor-
ing is edited, or collected in samples initiated by user movement or
timers. These adapted methods serve to condense the extraordinary
time devoted by formal ethnographers into more manageable and
ultimately more relevant samples of information for the design
8 An excellent reference for sources of
adapted research methods and others
is contained in a special issue of Visible
Language 36:2 (2002): “An Annotated
Design Research Bibliography: By and For
the Design Community.” See pp. 161–168
for relevant discussion and sources of
adapted methods.
03 Hanington 9/15/03, 9:21 PM14

Design Issues: Volume 19, Number 4 Autumn 200314 Design Issues: Volume 19, Number 4 Autumn 2003 15
researcher. Likewise, while cultural inventories or artifact analyses
may not be as in-depth as those carried out by anthropologists when
examining other cultures, a modest version of the methods may
serve design purposes extremely well.9
While often scientific in approach, methods from human
computer interaction similarly may be useful to design research.
Depending on the particular needs, these methods may introduce
a degree of rigor appropriate for some studies. Typically centered
around issues within interface design, “thinkaloud” protocol has
participants think out loud as they navigate problems or use prod-
ucts, to help the researcher identify key decision points, both posi-
tively and negatively encountered. Heuristic evaluation provides an
expert evaluation of a product or interface against an established set
of principles or guidelines. In “cognitive walkthroughs,” an analyst
assesses the opportunities for appropriate actions that might be taken
by users in task sequences.10
Innovative Methods
Designers are fundamentally involved in creative, visual activity,
and the research methods they use should provide corresponding
opportunities. Fortunately, there are a number of design methods
now established and continuing to emerge that represent cred-
ible ways of collecting user information through creative means.
The benefits of working visually in research may be self-evident
to designers, who respond intuitively to the language and find a
more natural transition to design decisions from visual information.
Additionally, when participants are invited to assist in research by
engaging in a creative activity, the response is likely to be more
favorable than when faced with a request to fill out a survey or take
part in an interview. Creative methods are particularly appropriate
during generative research, often referred to as projective because of
their success in uncovering needs and desires that may be unknown
even to the user, and that are difficult to articulate when probed for
using traditional methods.11
Innovative methods typically are identified by their participa-
tory nature, creative engagement and outcome, and their relatively
specific application to design research. Examples include design
workshops and other creative sessions in which participants (users)
are invited to engage in the generation or manipulation of visual arti-
facts to communicate their thoughts or ideas. Completed as group
or individual activities, emerging artifacts might include collages
detailing preferences and feelings, cognitive maps or other diagrams
indicating sequences of activities, actions, or thoughts, or models
configured to represent desired product features and forms. Diaries
may be formed using photographs and text generated by users over
periods of days or weeks to provide insights into experiences and
feelings. Existing visuals and documents may be annotated using
colored Post-its®, highlighter pens, and handwritten notes.
9 A good range of anthropology-based
methods for design is presented in a
special issue of Innovation (Summer
1996): “Anthropology: A Research
Resource.” See also Tony Salvador,
Genevieve Bell, and Ken Anderson,
“Design Ethnography,” Design
Management Journal (Fall 1995): 35–41.
10 Several references are available for more
in-depth discussion of HCI methods:
Jakob Nielsen, “Heuristic Evaluation,”
in Usability Inspection Methods, Jakob
Nielsen and Robert L. Mack, eds. (New
York: John Wiley & Sons, 1994), 25-62;
Clayton Lewis and Cathleen Whatnot,
“Cognitive Walkthroughs,” in Handbook
of Human-Computer Interaction, 2nd
revised edition, M. Hollander, T.K. Lender,
P. Parch, eds. (Elsevier: North-Holland,
1997), 717–732.
11 Uday Dandavate, Elizabeth B.-N.
Sanders, and Susan Stuart, “Emotions
Matter: User Empathy in the Product
Development Process,” Proceedings
of the Human Factors and Ergonomics
Society 40th Annual Meeting (1996):
417. See also www.sonicrim.com for
reinforcement of this argument.
03 Hanington 9/15/03, 9:21 PM15

Design Issues: Volume 19, Number 4 Autumn 200316 Design Issues: Volume 19, Number 4 Autumn 2003 17
While these examples serve to illustrate the intent of innova-
tive design methods, they are in no measure a complete list. The
whole purpose of innovative methods is to allow for creativity in
designing methods appropriate to the situation. For example, I
had a student who was conducting a human factors design project
on public restrooms on the university campus. Naturally, she was
concerned about protocol when surveying people in context on
such a private matter. We invented a method of “graffiti walls,”
whereby she papered the walls of several restrooms with a headline
asking for input on experiences and needs. These then were photo
documented each day, and collected from the walls at the end of
the study. Needless to say, she received a wealth of rich and useful
information for her project.
Interpretation and Analysis
Whether collected using traditional, adapted, or innovative methods,
the interpretation and analysis of information by design researchers
often will result in formats that may appear unconventional. These
formats may include quantitative summaries and text reports, but
will likely be complemented with visual information in the form of
sketches, diagrams and maps, models, photographic records, and
videotape. Prototypes such as documents that have been annotated
and color-coded by users may be compiled into single documents
that are visually analyzed for key problem areas and points of
success. Research results commonly are presented in a team forum,
in which they are discussed at length to extract fundamental mean-
ings, and moved forward into possible design outcomes for further
iterations of debate, development, and testing. Meaning typically
is extracted through the search for emerging themes, patterns, or
clusters of affinitive information.
The framework of methods presented here is not a compre-
hensive list, but an attempt to provide a convenient classification of
method types. The framework hopefully provides enough structure
and key examples to see where other methods might naturally be
placed, as they are encountered or developed. It cannot hope to
adequately represent the myriad of techniques that may permeate
the life cycle of a typical research and design process, to say nothing
of discrepancies in names given to similar or same methods. This
flexibility, while contributing to some confusion at times, also can be
a positive opportunity. Design research should be a creative activ-
ity, benefiting from many of the same characteristics as the design
process. An integrated approach to design and research that includes
designers as researchers will contribute to an enhanced understand-
ing of project variables, and add value to both process and results.
Designers as Researchers
Vast resources often are spent on user research and testing, while
ultimately not making any evident connection to design outcomes.
03 Hanington 9/15/03, 9:21 PM16

Design Issues: Volume 19, Number 4 Autumn 200316 Design Issues: Volume 19, Number 4 Autumn 2003 17
These failures often can be attributed to the inherent difficulty in
translating results from other research disciplines into an adequate
language for application within the design process. Practitioners
from other fields, including human research and management,
may lack a critical aesthetic “filter.” Again, the term “user-centered
design” argues for a process with implicit human concerns, yet
places the activity of research within the context of design. While
designers cannot typically claim the same level of expertise as profes-
sional researchers from other disciplines (e.g., human factors, social
sciences, marketing, and anthropology), their active participation in
the research process serves at least two key purposes.
Firstly, knowledge of design allows the interpretation of
research information in context. Whether that information is a pref-
erence expressed by an individual user, or a pattern witnessed across
users, these results can be balanced against the creative possibilities
(and limitations) of design. The anecdotal is weighed appropriately
in the context of more widespread opinions, yet the interpretation
requires more sophistication than a strict adherence to favoring the
highest number of responses, so often seen in quantitative analy-
sis. For example, several users suggesting that an illuminated red
button be used in an interface does not necessarily argue for its direct
physical representation in a product, yet may suggest the need for a
readily identifiable design element that offers appropriate feedback.
The exact manifestation of those criteria will be a creative design deci-
sion.
Secondly, immersion in the research process and direct
engagement with users forges a sense of empathy between designer
and user. In direct conversations in which users have described
upsetting and costly experiences owing to inadequate information,
it is difficult for the designer not to feel a sense of responsibility.
Similarly, when observing users who express a tangible sense of
frustration when navigating an interface, the evident impact of
design decisions and need for improvements are driven home. Such
exercises in research tend to expand the notion of usability beyond
function, and to reinforce the necessary emotional component of
human-design interaction.
Conclusion
Human-centered design currently is under scrutiny, both for the posi-
tive aspects it has to offer, and in the critique it faces as it emerges
into a research discipline in its own right. While few would argue
against the merits of consulting users in the process of responsible
design, the debate about how this form of research is best conducted,
in sequence and method, continues. The tendency toward integrat-
ing a scientific approach into the activities of design, only to justify
the discipline to professions established in the history of science,
should be waning by now. This is not to say we are not respon-
sible for the appropriate rigors of research, but only suggests that
03 Hanington 9/15/03, 9:21 PM17

Design Issues: Volume 19, Number 4 Autumn 200318
models of research adapted from other human-centric fields such as
anthropology and ethnography, and those developed through our
own innovation, correspond more adequately to the requirements
of design both as a creative process and in holistic content inclusive
of emotive human concerns. As the field of human-centered design
matures and earns credibility on its own merits, we can look forward
not only to the development of methods that satisfy the needs of
research, but to an increasing array of rewarding products that
emerge from responsible practice.
03 Hanington 9/15/03, 9:21 PM18