Languages of Interaction Design

I was particularly drawn to two ideas in the piece by Gillian Smith.

(i) Computation is a design material (of interaction design).
(ii) Interaction design is still drawing on the language of previous creative modes, and has not yet fully developed an independent language of interaction true to the medium of computation, networks, and telecommunications.

The first idea I was already somewhat comfortable with. The second idea is new to me. While I have recognized that the unique qualities of computation are still under-explored and under-exploited (recognizing analogies with the evolution of film and other new technologies), I had not recognized the importance, or necessarily even the existence, of “languages” to talk about the great new works that various technologies have enabled.

This idea of new design materials being used in old ways is fascinating, and I’m interested in learning of other historical examples of it (such as early television shows simply broadcasting video footage of radio shows). With computation and network technologies, for example, we are obsessed with putting old content online in digital form. In some sense, this is perhaps a radical new way to experience the old content, as it allows one to easily copy, access, search, and in some cases remix and edit (possibly collaboratively) the content. However, people are still typically composing books, lecture notes, and music in old ways and in old forms. An ebook is still in essence the same as a paper book, even if it does have hyperlinks and the ability to search and annotate. In the future, will authors still mainly write books to communicate complex ideas, or will they have the tools to compose dynamic, non-linear, hyperlinked webs of ideas or multi-modal narratives and interactive experiences? Will teachers still lecture, or will they provide exploratory virtual worlds for knowledge construction, discussions with real or artificial people, and the tools to create personally meaningful artifacts to learn and express oneself with?

Getting back to the idea of a language of interaction design…Smith almost seems to be saying that we need to sit down and work out an expressive language in order to move beyond our old uses of this new computational medium. “As yet, we have no fully developed language unique to interactive technology. So we are still drawing on the language of previous creative modes. ” To me, it seems that an expressive language will evolve as we learn about, and better exploit and shape, “the new and unique qualities of the medium of computation, networks, and telecommunications.” Or have we already shaped and exploited the technology enough to start creating a language, and merely forgotten to do so? I’m eager to learn more about the design and evolution of “languages of previous creative modes”.

Some questions…

What are useful distinctions or definitions of design materials? Wood, brick, and concrete? Electronics? Verbal language? Typography, iconography? Sound, film, animation? Computation, networks, and telecommunications? Interactive technology? Biotechnology? How do they relate? How are they different? What expressive languages exist for working with these materials?

Do we already have a working set of vocabulary words for this interactive, computational medium? What (unique) qualities does it possess? In what sense is it a “material without qualities” (as described by Lowgren and Stolterman in Thoughtful Interaction Design)?

How has computation been used incrementally, in old ways? (And is there a name for this phenomenon of wrapping old ideas in new technology?) How have its unique properties been leveraged in radically new ways? How can/should we leverage it in radically ways?


  1. jeffreybardzell

    This is an excellent entry that raises a number of key issues that we’ll be talking about all semester. I am not surprised you don’t really have answers to the questions you pose at the end, but by the end of the semester, while you may not have definitive answers, you’ll have workable answers (I hope!).

    Also, and this is a note to everyone, I love it when people take the time to spell out their ideas in more detail like this, and I certainly will read every word, but there is no pressure to write blog entries this long! All I asked for were about 2-4 paragraphs. So, to summarize, 2-4 paragraphs is my expectation and all that is required, but I will gladly read as many as you post!

  2. houssian

    Love your questions! Would love some answers too. Jeff can you get on that please, and make it quick;).

    One quote stood out from your post (yes this is a recurring theme for me) “An ebook is still in essence the same as a paper book, even if it does have hyperlinks and the ability to search and annotate. In the future, will authors still mainly write books to communicate complex ideas, or will they have the tools to compose dynamic, non-linear, hyperlinked webs of ideas or multi-modal narratives and interactive experiences?”
    Your questions begs another question, how will people consume what we now call “ebooks”? Jeff mentioned his recommendations on how to interact with a text, using stars for definitions and other suggestions for making a text “your own.” I personally LOVE ebooks, and until they started really sucking last year I have been a long time Palm user and I always had some of my favorite books loaded on it. I had some programs to mark, highligh, add comments, make a passage with a theme, and also be able to search (things that you mentioned) but I always want to much more. I want to be able to do more free form things like what Jeff talks about. I want to be able to access all those things I do on the palm on my other computers in an expanded view with perhaps more functionality. Imagine a really well written book that when they quote that study, article or other source you can actually READ that other book they are talking about, or at least a couple pages of it. Combine all that with some of what is happening with so-called web 2.0 applications like this one where we can comment on each other’s thoughts, so that as I read an assignment I could give you access to my notes, and me yours. While it won’t help you make the text your own, it could reinforce what we think the important topics are, show common weaknesses in our understanding (and let’s say Jeff could see them too, he could tailor his next lecture to help us out). This just all goes to show that this whole thing called computing is still in it’s infancy.
    We are taking baby steps, and I for one am happy to be here to see it happen.

  3. marty

    Houssian’s point is very interesting to me as it begs a discussion of the importance of the artifact in our interaction with technology.

    Many people today collects old 8-bit home computers and game software so that they can play the games just as they were experienced back in the day. The curious thing is, in most cases, you can experience the old 8-bit copmputer games pretty much flawllessly and faithfully on modern computers using free and widely available emulator software. Yet, these folks choose to hunt down old computers on eBay, clean and repair them, and play the games there. And it’s not like these folks are a fringe, there are lots of them in the so-called “retrogaming” space.

    And it’s not just old computer games. Similar enthusiasts of “golden age” radio programs of the 30-50’s collect old style radios (or modern retro-alikes) to listen to the classic radio shows that they enjoy, even though they could much more easily experience the shows on the widely-available MP3 sharing networks of the internet.

    Clearly, these nostalgia enthusiasts are willing to go to some lengths to experience their classic content in a classic context. There must be something compelling to them in the association between the physical aritfact through which we are entertained and the emntertinging media itself.

  4. Tyler Pace


    What about the people who repurchase those classics over new distribution channels like Xbox Live Arcade and Wii Shop? Mediating their classics with new media!

    I understand for XBLA it’s part of the social capital of gaming (“gamer capital” as Mia Consalvo puts it), but what about the classic Nintendo titles on the Wii which are exact replications of the original.

    Perhaps it all boils down to the importance of the remembered experience with a touch of ritual. For some, the remembered experience (therefore the one they are trying to relive) was playing the game on their couch in front of a TV. All you need to satisfy this gamer is the original game on a television. For others, the ritual of blowing on a cartridge, putting in the slot, powering on, slapping the top of the box and then hitting start was an important part of the experience, one worth revisiting.

  5. kaycereed

    In response to everyone above, especially to Marty and Tyler’s posts about using classic machines to consume classic software/games, I am completely guilty of this. While we own nearly all of the Nintendo, SNES, and Sega games on our computer in roms, there is something that completely separates playing a classic game on the computer versus getting out the console and playing some Mario with the original controllers on the television (and of course having to blow air into the bottom of the cartridge and the console because this somehow makes the damn thing work 🙂

    Further, here is a photograph taken in my family’s computer room: [] On occasion, I still walk by and put the 8 inch floppy disk of Where in the World is Carmen Sandiego into the drive 🙂 or play some Pacman with the joystick. It’s actually pretty incredible that it all still works since my dad brought the computer home in 1987. Goes to show just how good Apple products can be.

    also, boo on not being able to embed images directly into the blog post when commenting….


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s