Epistemology and Design: The Place of Judgment

In dialog surrounding the reviews of a recent paper a colleague and I submitted, one of the reviewers, resisting our call for a greater emphasis on criticism in interaction design on the grounds that psychology already does it, asked the following question:

How can you prevent the “anything-goes-subjectivism” when the judgments are not objective?

This is the kind of question that drives me–and I think anyone trained in the humanities–crazy. My immediate reaction is that this question is both naive and bigoted, not merely privileging that person’s own scientific background, but categorically excluding the possibility of intellectual contribution from anywhere in the liberal arts (art history, literary criticism, fashion design, philosophy, music, film, etc.).

But after some reflection, I realized that my reaction isn’t good enough. Here’s why:

The reviewer asking the question is an assistant professor of design at a prestigious university. This reviewer is not an idiot or a jerk; this is a person who understands how his own epistemological underpinnings lead to intellectual contribution via a proven “science of design” approach built around prototypes, scientific evaluation, and iteration. What’s missing is an understanding of how alternative epistemologies are productive of intellectual contribution that may be relevant to his own area. In other words, I doubt (at least I hope it’s not the case) that this reviewer would literally assert that the arts are categorically useless on grounds of their subjectivism. And yet, his stance on knowledge leaves little room for anything but that. As someone who advocates integrating humanist approaches, and in particular criticism, into interaction design, I realize I have a burden (fair or unfair; it doesn’t matter) to make my position at least comprehensible to people from these other traditions.

This blog is a very early attempt at developing that rhetoric.

Perhaps I should start with the word “judgment.” The reviewer asks how a judgment can be meaningful if it is not objective. My own intuitive response is that if it is objective, then how can it even be called a “judgment”? To me, judgment is what happens when an expert in a phenomenon that cannot be comprehensively or rationally understood–such as the human response to a work of art, and all the poetic and social relationships implied therein–makes informed statements about the quality, worth, and/or nature of that phenomenon.

Consider this simple example: a respected film critic says that a new film is excellent. Should we trust this film critic’s opinion over that of a friend or colleague (who is not in the film industry)? I think most of us do. And yet, this film critic has not conducted an empirical study and acquired objective data about the film. The judgment was grounded on her or his own subjective response, which has been reflected upon in light of both theory of film and systematic, professional encounters with thousands of other films. Here is a different example. How does National Geographic know whether a photo is worthy to be included on its pages? Surely the editors do not conduct a scientific review; rather, they have an eye for what makes a “National Geographic quality” picture. But what does that mean, to “have an eye for”? It’s a subjective judgment, so it’s not scientific. But if it’s anything-goes, then the only thing that could explain National Geographic’s success in publishing top quality photos is astronomically good luck.

Now I’d like to offer a couple examples that are more explicitly design-oriented. Both of the following are quotes from an introductory book on fashion, Icons of Fashion: The 20th Century, edited by Gerda Buxbaum (Prestel). The first quote below refers to Austrian designer Helmut Lang:

Whether masculine-cut or futuristically transparent women’s suits, whether utilitarian, rebelliously elegant, Day-Glo striped, or cuffed jeans, Helmut Lang’s clothes set the tone for the 1990s. He is among the true fashion inventors of the twentieth century, and his designs provoke irritation because they are often ahead of their time. Lang’s fashions frequently point to major social changes before their initial tremors can be felt: the nomadic image of globalization,the scars and triumphs of women’s liberation, the fusion of American and European culture, the yearning for simplicity, and the desire for luxury. All of these are evoked for us, sharply and beautifully, in Lang’s clothes. “I don’t believe fashion evolves on its own,” he once said. “There are more radical social changes behind it.”

Clearly, Lang is a successful designer, by any reasonable definition of the term. Equally clearly, his visionary work neither deploys nor is evaluated by any sort of “science of design” (at least the visionary part; aspects of its execution are likely informed by prototypes and market data). His understandings of social changes and future trends cannot be objective. And those who judge the quality of his work, from the fashion industry and corporate buyers to the consumer, do not do so on the basis of any objective data collection. Yet his success reveals that in the subjective judgment of thousands, his work is not only very good, but it is very good in particular ways. Lang is perennially successful at being “fashion forward,” a term that is cognitively empty in the scientific sense (really, it means just about nothing), and yet pervasively used in fashion to name the je ne sais quoi that makes a garment or accessory just feel right to thousands, if not millions.

Here’s how the same book introduces the work of John Galliano:

The historicism and romanticism of John Galliano’s designs are rivaled only by the spectacular and theatrical nature of his fashion shows. One of the great image-makers of the twentieth century, he is largely responsible for the media feeding frenzy which has typified the coverage of collections in the 1990s. Compared to France, the infrastructure in of the British fashion industry is less developed, so a spectacular show may be a designer’s only passport to media coverage.

If this paragraph is reasonably accurate, then Galliano is more successful at constructing reality than he is at discovering it. In other words, what Galliano does is not empirically determine what taste looks like at a moment in time and then design for it. Rather, using exotic marketing and glamourous imagery, he constructs a feeling that did not exist before, and consumers who engage with it start to want it. He is not finding demand, but rather creating it. This is science fiction, in a sense. The designer speculates, constructs a fantasy, and then uses a combination of tangible designs (here: clothes) and showmanship to generate consumer demand for his designs. Again, the question is, if it is all speculation and fantasy construction, and if that’s anything-goes, then any of us should be as good as Galliano. But we know very few actually are. There is a method, a rigor to Galliano’s speculative fantasy constructions; it is learned, cultivated, and nurtured. But it is not scientific.

One might object that the sort of fantasy construction I am talking about here belongs to the province of art, and really science has nothing to do with that. But I am talking about designers, not fine artists. Galliano, Lang, National Geographic, and so forth do not have the luxury to make art that pleases them and a microcommunity of like-minded intellectuals. They are all major players in the marketplace. Their speculations and fantasies must be able to gain traction at a sufficient scale to support their businesses. In short, their work is “serious” in the same way that interaction design is serious: it has to work for, be of value to, and feel right to their everyday end users.

I hope I have drawn a boundary between mere opinion and expert (subjective) judgment. Expert subjective judgment entails a lengthy, reflective, intellectual encounter with thousands of instances of similar cultural artifacts. It is a capacity to appreciate how the subtle mechanics of those artifacts, from the meter of a poem to rack and pinion mechanism of an oranger juicer’s arm, directly connect to felt experiences that are engaging, joyful, and even enlightening. This capacity has always been a part of design–industrial design, fashion design, architectural design, visual communications design–and it is an unfortunate commentary on our field that it has yet to be legitimated, much less developed, in interaction design.


  1. cbriggs

    Not sure if this is the exact quote or not, but i’ve read this before in various places:

    “The intuitive mind is a sacred gift and the rational mind a faithful servant. We have created a society that honors the servant and has forgotten the gift.”

    – Albert Einstein

  2. Wodom

    This post is on target and I’m in agreeance. I would argue that the entire field of HCI has an orientation toward the massive generation of consumer products and thus its criteria for producing “research knowledge” largely emphasizes generalizability and objectivity. The epistemological underpinnings in this dominant perspective have serious implications, from how we design for, research and ‘evaluate’ experience to approaching (or acknowledging) the perspective of sustainability. Criticism is constitutive of these areas of HCI research and has an essential role to play in future HCI discourse.

    A question I’m wrestling with and perhaps one to open up to the blog: how do two differing epistomologies co-exist within the same discipline (without constant unproductive clashing)? They both offer important perspectives in HCI and I think they ought to co-exist. However, considering a scenario in which they did: how do we effectively mitigate between metrics for evaluation?

  3. anandkshitiz

    This is brilliant! The last class and this post has made me find an answer to one of the questions top which i have been seeking an answer since long.. and also posted in my previous post, regarding the subjectivity and objectivity of designs that we look at.
    I was able to relate immediately to a field like photography, where it applies really well.
    Over a long period of time a debate has been going on in my mind about, we as designers having to create for objectivity. However, after this class and the understandings garnered form that I have come to the understanding that, there is subjectivity all around. As discussed in the class, a critique is and entirely subjective opinion, that develops over a time form the impression. When the judgment is passed, by these critics or rather subject experts, (which again are subjective) we tend to believe them more than we would to a classmate or any other person (not a subject expert).
    So ultimately we are designing for a subjective opinion, with the hope that the critic’s subjective opinion matches ours. This would lead to a larger audience believing the subjective opinions of the critics (as they are the subject exerts) and over a period of time, spanning across many people, the design becomes objective.

    @Will, i do agree on this point about the entire HCI field having the orientation of research towards the objectivity. The question that you pose is rather interesting and would love to hear more from others on this.. perhaps from the SUBJECT EXPERTS! 🙂

  4. Jon Kolko

    Well written, and a good continuation of what has been a long-standing argument, begun (as far as I can tell) by Leonard Bruce Archer, that Design is not Science, and not Art, and not a blend of the two. I wonder, however, if there is something else – besides “Good Design” (objectivist and usually dogmatic reference to the bauhaus) and besides judgment (entirely subjective, with emphasis based on the experience of the viewer) that can be used to categorize and evaluate design works? Considering how deeply the artifacts of design are embedded within culture, it seems like we should be subjectively defining the judgment constraints as they apply within a cultural framework, but then objectively conducting the judging….

  5. jeffreybardzell

    Thanks to everyone for the feedback. I just want to reiterate the scope of my entry. My goal is not to enter an ongoing argument about arts versus science in design, and it certainly is not to suggest that art/judgment is better than science. My specific concern was that as HCI begins to call itself a “design” field, many of its practitioners appear to be carrying forward their own ways of knowing (especially scientific ones), while categorically rejecting whole domains of design thinking (such as comparatively humanistic ones), even though both have at least some place in all other fields of design.

    I began the post with a quote, which (paraphrased) said that if knowledge isn’t objective, then how can it be other than anything-goes? If this is the level of discourse–and my experience in the field suggests that in many corners, this really is the level of discourse–then before HCI can meaningfully move forward in its critical capacity, we need at least to create a clearing of legitimacy for it. With that clearing, even if not everyone engages in this sort of design thinking (which is perfectly OK), then at least there is a recognition that this kind of work might be other than mere opinion.

    My problem is that every time I submit a paper, there seems always to be at least one reviewer (sometimes considerably more) who dismisses the whole work a priori on the grounds that it’s all “subjective,” that it’s not “science.” It’s this a priori dismissal that is the immediate target of the post, not the more broadly philosophical and intellectual problem of the place of judgment in interaction design (which, frankly, deserves a wee bit more than a blog post!). In other words, I am trying to craft a rhetoric, some version of which will have to appear at least in condensed form, at the beginning of all my more culturally oriented papers that include scientists and engineers in its audience.

    I’ve tried it before, but it hasn’t been persuasive to this audience; it either bounces off and has no effect or it comes off as if I am saying “art is better than science,” which is nonsense. One thing I like about this strategy is that it includes simple examples, such as the National Geographic example, which I think does help people intuitively understand the difference between mere opinion and expert subjective judgment.

  6. Gilbert Cockton

    Thanks to Jeffrey and everyone else for this great thread. The tactic here is to just advocate, explain and exemplify approaches from various disciplinary practices in the Arts and Humanities (of which there are many, and woe betide anyone who lumps them altogether as an Entity). But there is another tactic, that is very well supported in the Philosophy of Science Literature, as well as in Science and Technology studies. This tactic is to pull the rug from under the positivists feet, and to show that their critique of subjectivity applies just as well to the sciences as to more hermeneutically based disciplines. There are some very well rehearsed arguments here, for example, the objectivity of a claim such as “The cat is on the mat”. Putting aside the questions of how we know the two perceivable objects are a cat and a mat (which can be resolved in this case without encountering major issues of naming and ostension), the objective basis of “on” in sense data is tricky (especially when we add a fly on the ceiling or a mirror on the wall). Hapless positivists will attempt to give geometrical definitions of ‘on’, but as we rotate the ‘on’ relation through 3D they are forced to add more and more abstract concepts in an ever more complex mathematical system to shore up what should simply be a matter of looking and seeing. Things get even more complex in haptic feedback systems (e.g., force feedback for insertion of a peg into a hole). Adherence to exact geometry here for whether the peg is or is not correctly in the hole here results (or at did 10 or so years ago) in the force feedback vigorously ‘shaking’ the user’s hand as a virtual peg is pressed into a virtual hole. Getting the smooth feeling of the real world here is quite a challenge alongside the replication of the resistance to the peg when it is not properly ‘in’ the hole. While a 2 year old knows very well when a peg is exactly in a hole, a haptic VR system has major challenges.

    So, the best form of defence is attack, and there is not one aspect of the ‘real’world that cannot be shown to involve subjectivity and judgement in its perception and appraisal. This needs to be borne in mind when considering Jon’s “it seems like we should be subjectively defining the judgment constraints as they apply within a cultural framework, but then objectively conducting the judging….” I said something similar in my NordiCHI keynote abstract in 2004 (see ACM DL):

    “A focus on value creates a paradoxical discipline that fuses subjectivity and objectivity in a single process. HCI must be objectively systematic and reliable in the pursuit of subjective value. ”

    I was being naughty here of course. I have no problem at all with the idea of subjective value, or systematic and reliable procedures, but ‘objectively’ was a bit of a redundant wind up. If we have systematic and reliable procedures, who cares whether they are objective or not (assuming that objectivity is actually possible that is)

    Fight fire with cold water, not with fire!

  7. Jenny Brown

    I was doing some research for David Hakken’s “Ethnography of Information” class today, and I came across a website that seems to very appropriate for this discussion!

    Barbara Hall, an anthropologist at Penn State, wrote of three ethnography concepts that could be easily applied here: cultural relativism, triangulation, and “the validity of ethnography” in the following website: http://www.sas.upenn.edu/anthro/CPIA/METHODS/Objectivity.html

    Cultural Relativism:
    “Anthropologists generally subscribe to some form of cultural relativism, meaning that we believe that there is no one standpoint from which to judge all cultures and ways of being in the world. Because of this, we are conditioned to see various perspectives as “positioned” (Abu-Lughod 1991), and the things that we learn in the field as “partial truths” (Clifford 1986). Therefore, there is not one single truth in a research situation to be uncovered; there are many.”

    “Ethnographers should have more than one way to show how we arrived at the conclusions of our research; we expect to have a collection of fieldnotes, interviews, and site documents (where possible) which work together to support our claims. This is called triangulation.”

    The Validity of Ethnography
    She quotes another anthropologist, David Sanjek who wrote of “the ‘validity’ of ethnography”, with which “we can judge the clarity with which decisions regarding the application of theory to data are explained as well as follow ways in which events in the text are persuasively linked in making the conclusions presented there.”

    Can’t we take a cue from ethnography and adopt these ideas for design? Seems to me they would address the question that Will posed about how two differing epistomologies can co-exist within the same discipline.

  8. seanconnolly

    Scientists are cowards.

    There is nowhere, except in academia, where something needs to be objectively perfect before it is right. Yet only in manufactured environments can anything be objectively perfect.

    Newtonian physics allow your car to be engineered by engineers, allow your car to be built by robots, and this allows you to demonstrably get into a car and drive it… even though Newtonian physics are wrong.

    They’re mostly right of course – and they’re right in the most effective and applicable way – but they don’t hold up under all cases. They are not objectively correct except in in particular environments. Einstein comes along and corrects this with relativity (i.e. relative to the environment), and, you know what?

    He’s still a little wrong.

    He’s still wrong, but what Einstein proves that events occur only in relation to other events. The most real “reality,” is subjective.

    In Principia Mathematica, Bertrand Russel exhaustively lays each principle of mathematics (propositional logic) and, by exhaustive example, proves that if one starts with mathematic truths (axioms) and applies mathematic principle (logic) on natural numbers (set of reals (or, set of acceptable symbols)), one can solve any “question” that can be translated into the formal logic. He proved good, rigorous logic was always consistent.

    Goedel comes along twenty years later and proves him wrong.

    The later extension of Goedel’s proves that any formal system strong enough to replicate mathematics — which is pretty much what we’re looking for in logical, objective “sciences”– will always carry contradictions within it.

    There are only degrees of objective reality.

    Since there are only degrees of objective reality we are always only ever making choices (judgments) as to which environment we wish to understand and which micro facets we wish to understand within these subjectively chosen environments. Some “scientists” choose to work in arenas where can minimize the impact of the environment upon the operation (again, building a car), and because they can create artificial environments within which they can artificially control interaction. Because they conduct science in intellectually sterilized environments they can have this desired “repeatability” of results that connotes real science.

    But in real, unsterilized reality?

    People make choices. And, informed people make different choices. And informed, experienced people make still different choices.

    Now – and Jeff may disagree – but ever choice that people make is some percentage “correct.” And, I would say, that there is a percent chance that a total neophyte in the most complex science could still divine the most “correct” answer to a critical issue in that field.

    It ain’t highly likely. But, there’s a chance.

    There is a far greater chance, due to information, education, mentoring, and experience, that an expert in the field will address the critical issue in the complicated field. “Fuzzy logic” being a branch of mathematics now anyway, we can actually valuate and validate the percentile chance of experts adding to their field relative to middle-of-the-roaders, relative to beginners. (and the entire spectrum in between).

    Which brings me back to why scientists are cowards.

    It’s safe when you artificially sterilize your environment so you can reliably reproduce your results.

    It’s bold when you can aggregate a variety of somewhat correct evidence, choose to believe your own perception about that evidence, stand up with that belief, and make future-oriented predictions that turn out to be somewhat correct.

    THAT’S science.

    (P.S. this debate always reminds of “moral relativism”. This is the idea that since we cannot prove one religion is correct, all religions are equal. Well, I don’t agree. I don’t agree that religions which promote violence, intolerance, degredation, or ignorance are equal to those which do not. John F. Kennedy wrote a book called PROFILES IN COURAGE, and, in the preface his brother cites a quotation from Dante [and, from which, I got the ‘title’ for this post:]

    “the lowest circles of hell are reserved for those who, in the face of moral uncertainty, do nothing.”


    (Scientists are cowards)

  9. jeffreybardzell

    You make a number of valid and interesting points in your post. Unfortunately, the hyperbolic rhetoric undercuts your own authority.

    Much of my original post is about finding ways to engage others in other disciplines, to explain one’s own point of view constructively, so that even those with totally different backgrounds can understand your value. The whole point is to *avoid* name-calling and disciplinary bigotry. One reason I am so opposed to that is that my background and values are not the mainstream in HCI, so I am often a victim of this rhetoric myself.

    Thus, when you start and finish with “scientists are cowards,” not only do you engage in this sort of counterproductive mudslinging, but by doing it on my blog you might lead some to infer that this is what I am teaching you guys!

    So as a teacher, a friend, and this blog’s owner, I’d like to encourage you to express your points with even tempered, constructive rhetoric. Most of the points in the main body of your post would be far more persuasive if you avoided the name-calling and straw man accusations. Here’s an example of the latter: scientists artificially sterilize their environments, which is why they are cowards. Scientists are always accused of sterilizing their environments, but if you read the papers that are the products of this approach, most scientists acknowledge this problem and explain how they manage it (usually by carefully stating their conclusions or claims). They are not cowards; like those who rely on more critical approaches, they live with, manage, and develop theory and methods to improve their capacity to deal with the intrinsic limitation of their approaches.

    That said, scientific approaches, like all approaches, can and should be subjected to thoughtful critique and productive skepticism. This is a totally appropriate place for that, but please be careful about how you frame that critique rhetorically.

  10. seanconnolly


    I certainly appreciate the teaching, and, that is why I am here. Perhaps too, being a bit more humanities trained, I am inclined to react much as you suggest at the beginning of your post, rather than at the end.

    Thank you for taking the leadership moment to point out and teach.

    However, while you are right that scientists tend to point out where they have marginalized external influences, I do feel there is a consistent and embedded contempt of the humanities by hard science. I do believe it stems from their own pride in their own objectivity. I do believe their own objectivity is more false than the manner in which they wield it would suggest they are aware. And, I do take that personally. I do believe we are still living in an era dominated by oppressive logic over humanness. I do believe this third-and-future-waves of HCI are the logical result of this long standing suppression.

    But I do not believe all “scientists are cowards” and I should retract that statement. I do.

    Furthermore, I don’t really like it when people/critics/scientists form a strong polemic to refute presuppositions embedded in a culture. I would rather clearly state that which I believe is correct than over inflate the degree of my disagreements. Thank you again for taking the time to teach.


  11. The Essay and HCI « Interaction Culture

    […] that subjective = opinion = lack of rigor = not knowledge, I wrote a post distinguishing between opinion and judgment, and I’ll let that do its work and not restate it all here. Suffice it to say that I […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s