Jeffrey Bardzell
Indiana University
(version: August 2018)

[Background: This is a handout for MS HCI/d students that I’ve been iteratively working on for a decade or so. Given the recent interest in this topic–see, e.g., howtocrit.com–I’ve decided to make it publicly available.]

Introduction

As both design practitioners and everyday citizens, we all critique. We do it automatically, easily, and at appropriate times. At the end of a movie, we ask each other, “what did you think?” and whatever follows is some form of critique. When we recommend a book to a friend, we often do so by making critical arguments. If you’ve ever pointed to a form of social injustice, chances are you also invoked critique.

Therefore, the purpose of this guide is not to teach you something you don’t already know. Rather, it is to help you build on a practice with which you already have competence to try to take it to another level, so that it can serve as a tool in your professional design practice. Have you ever felt strongly about a novel, but been unable to tell others just why it was so special? More to the point: have you ever felt strongly that a design should be framed or positioned this way, rather than that way, but been unable to get others to see with you? In such situations, our need for critique outweighs our abilities.

What is Critique?

Most people associate critique with criticizing something. We often think “to critique a design” usually means to challenge it, point out its shortcomings, identify its flaws. But this is a very limited and limiting notion of critique. When we read film, software, or device reviews, we don’t want to hear only about the flaws and the problems. We want to decide, in this instance, whether we should go to this film, or buy that software or device. We want to understand what it was trying to achieve, how it set about doing so, and in which ways it succeeded. We want to appreciate it for what it is, and then form a judgment about whether that fits with our desires and/or needs.

Speaking generally, critique can be understood as a sustained, interpretative evaluation of an object (such as a novel or software application), experience (such as going on a cruise or a videogame experience), or phenomenon (such as a “smart home”). Often, they are interlinked: when critiquing Airbnb, we might talk about it as an app, a kind of user experience, an alternative paradigm of traveling or relating to others, or as an instance of “the sharing economy.”

Forms of critique include movie and software reviews, scholarly works on artistic movements (e.g., a book analyzing Impressionist painting), activist analyses that expose hidden forms of oppression (e.g., in feminism and postcolonialism), and studio design crits. These are very different in scope, discursive form, and purpose. But common to them all is a systematic attempt to get a handle on something significant or meaningful but maybe a little elusive, often about which we must (or should) make decisions.

Why Critique Designs?

Again, virtually everyone critiques designs all the time as a part of everyday living. But there are some special roles for critique as a part of professional design practice.

Above all, critique can improve our design practices. Critique can support a particular given design process by collecting, analyzing relevant exemplars for the following: it is key to competitive analysis, creative inspiration, and forecasting/futuring. It can support collaboration in design, such as studio design “crits”, providing formative feedback. Critique can facilitate building on prior design achievements, through the discovery of design patterns and their use. And critique helps us theorize about designs, design genres, etc.; heighten our awareness of humankind’s best design achievements (and why they are achievements); it builds design literacy and domain literacy.

More broadly, critique improves our abilities of discernment and judgment in specified domains of life. It trains us to locate and understand meaning and significance; to cultivate appreciation, to inhabit a design (design situation, design experience, etc.) more fully. It helps us become (more) alive to phenomena and experiences of interest. It undergirds activities of self-transformation, epiphany, reaching out to the divine (e.g., when we read a transformative novel over and over again and reflect on its deepest meanings). All of this sharpens our awareness of how human needs are activated, served, or denied in concrete real life situations—the kinds of situations that we want to intervene upon.

Critique also supports communication, collaboration, and instruction. Critique is a staple in design education. One of the goals of critique has always been to help others see a work (or performance, or whatever) the way that a critic is able to see it—think about what a docent does in a museum, or what happens in an art history course. For designers, it is often important to get non-designer stakeholders and colleagues to see a problem or a proposal in a designerly way, and the practice of critique is a powerful way to do so.

Common situations that call for design critique are (a) when improving a design project in progress and (b) when curating design collections, used for inspiration, competitive analysis, and so forth. We might also add (c) when conducting design research in a more academic sense. These are clearly different purposes and the following guide might apply to them in different ways.

How to Critique Designs

Critique doesn’t have a set method; it is often highly contingent on the object of critique, the person(s) doing it, and its purposes. Instead, I offer a metadescription that characterizes the practice in a general way, and then I follow that with a more fine-grained account of common questions and critical tasks.

A Metadescription of the Tasks of Critique

At a high or meta level, design critique typically involves three related moves:

1. Offer a description. The goal here is to render the work as an object of analysis. What is the system? What does it do? Who uses it? What is it made of? The answers to these questions by and large should not be controversial or difficult to defend: any appropriate person should more or less agree with this aspect of the account.

2. Pinpoint the purpose of the critique by identifying a puzzle or hard question of interest. Common design critique questions include the following:
   • How is a certain experience achieved?
   • What justifies/explains my sense that this is good or bad design/direction/strategy/opportunity?
   • Why was that particular design significant/successful, when all these similar designs might have been but weren’t?
   • This design excels at X. How exactly does it do so? Can that be replicated?
   • In what ways is this design typical or paradigmatic of {a type of design, a style, a movement, etc.}?
   • What creative process could have possibly led to {this framing, style, use of design patterns, etc.}?
   • Why does/doesn’t this feel like it fits in our product portfolio, brand strategy, etc.?

3. Deploy common strategies to find “ways in” to better understand the problem or puzzle. Many specific questions are listed in the section below, but at a high level, there are a few common strategies.
   • Elucidating contexts. What counts as “context” is itself a subjective judgment—whatever seems relevant to the critique. What is the material ecology that the design is a part of? What is the historical situation out of which it emerged? How did people in that time tend to think about {that kind of design, the practice the design was intended to support, etc.}.
   • Forming explanatory hypotheses. A common critical move is to notice a pattern in a work and then to propose an explanation for it. Imagine, for instance, that a character in a movie acts in a puzzling way: we might consider that odd behavior in relation to her or his prior actions, and then say, “well, perhaps it’s because of X.” Then the critic can look for other evidence that comports with (or contradicts) X. Likewise, something anomalous might appear in a user study. Or something that seems like a really poor design catches on. What gives?
   • Moving back and forth between generals and particulars. Critical thinking about a work is often a kind of dialogue, where the critic contemplates an idea and looks for manifestations of that idea in the work, comparing and contrasting them, tracking how they develop. In the case of design, generals would include such things as the design’s gestalt (e.g., its overall purpose, rationality, vibe); general understandings of expected user experiences; themes, styles, communicative conventions, and visual language; patterns/signatures of the designer/studio, etc. Particulars include things like the concrete details of the design, individual design choices, and other physical or specific manifestations.

Critical Questions

More concretely, a design critique often proceeds by doggedly pursuing a range of questions, such as those laid out below. Note that these are not in order and are often asked in iterative and interleaving ways.

It is important to understand that this isn’t a checklist or a heuristic evaluation—the goal is to develop interpretive understandings that are more nuanced, sensitive, and insightful than you would otherwise achieve without engaging them. Responding to one of the questions below is meant to generate further questions and probes, not to prompt a simple “answer.” These probings or speculative Q&A iterations can collectively constitute the tasks identified in the previous section.

Also, one can introduce a theory or concept at any time to scaffold this work. For example, if one is trying to interpret some seemingly anomalous user experience data, one might deploy Dewey’s notion of experience to try to unpack what is happening. Or, if it seems important to try to understand the creative process that led to a work, creativity theory from social psychology could be introduced to help structure the interpretation of available evidence. For a critique with an emancipatory angle, deploying concepts from feminism, Marxism, or postcolonialism could help guide and direct critical attention.

The following is a relatively generic list of critical questions that might be asked about a design. They are offered as “ways in” to the critique; answering them in a literal way is not really a critique. Ideally, taking up these questions should iteratively lead to more refined and targeted questions. These questions and categories are not in any particular order.

• Purpose, rationality, design intention
  • How does the artifact manifest a “total rationality” (Nelson and Stolterman)?
  • Are explicit designer/creative intentions extant?
  • What implicit intentions can be inferred?
  • What is the design argument (which might be explicit or tacit)?

• Process of creation, design methods
  • By what process, via which creative activities, did this come into being?
  • Were the design processes rigorous? Were they replicable?
  • Does the design reveal how certain design processes or methods are likely to yield (or support) certain kinds of design outcomes
  • Whose voice, perspective, expertise helped to shape the process and its outcomes? Were any alternative voices or perspectives (apparently) left out of the product’s formative stages?

• Use, function
  • What results are possible, typical?
  • Who are (and are not) the stakeholders, beneficiaries, immediate users?
  • Distinguish between implied users (why type of user, what role, or subject position) and actual-historical users
  • What are the units of precision?

• Materials and craft
  • Who made it and with what sorts of skills?
  • What are the materials’ semantics?
  • How are material capabilities, qualities, and semantics leveraged?
  • How do material forms and information processing intermingle?
  • What other materials and material-forms are connoted?
  • How is the form-content stylized?

• Business and strategy
  • What is the business model or opportunity?
  • Who will pay for what, how, and when?
  • How does this design fit with business/brand strategy?
  • How does this design complement other products and services within its domain; how does it contribute to (or disrupt) existing product ecologies)?
  • How is this product differentiated from similar products?

• Elements/meaningful units of the interface or interaction?
  • Identifying significant features
  • How does the form annotate the content?
  • How does it organize the activity it supports (hierarchy, temporality, narrative)?
  • How does it all hang together?
  • How does it (intertextually) reference prior media and works?

• Human needs
  • Who are the users and stakeholders of this design?
  • What meaningful human needs does this serve (this question goes beyond mere functionality)?
  • Are there needs that the design is ambivalent about? Should serve but fails to?
  • Does the design leave certain groups or individuals outside or behind? Does it prioritize one group’s needs over that of another?

• The implied user, or, the user inscribed in the design
  • What kind of user does the design expect, assume, and best serve? (E.g., what kind of user does Adobe Illustrator expect—what must one already know, and what must one want to do, be willing to learn, etc.?)
  • Conversely: who is not a “good” or “ideal” user for this design? Who is left behind? Who should be left behind (not every interface should be for everyone), and who is left behind but shouldn’t be (e.g., when banking applications fail to accommodate elderly users)
  • What technical skills are normalized?
  • What ways of being and doing are normalized?
  • What activity- or domain-centric skills are normalized?
  • What lifestyles are assumed, prerequisite, or connoted?

• Experiences: Characterize the user experience
  • From the user’s perspective, what are the meaningful units of experience, and how are they composed or organized into a whole experience?
  • What are the moments of consummation or closure?
  • How is the experience punctuated?
  • Does it achieve flow? Which moments are especially flow-y? Why?
  • Are units of experience best measured in milliseconds (e.g., visceral experiences), minutes (e.g., completing tasks), hours (e.g., work sessions), months (e.g., behavior change), years (e.g., reflective experiences)?
  • Design poetics: What qualities or features of the design/interaction shape or even cause certain experiences or experiential qualities?
  • How much room for variation is there (i.e., do different users have very different experiences)?