Dave’s Realism in HCI post reminded me of an experiment that I saw when I visited Bill Buxton’s Alias research lab a few years ago. Buxton has worked for years with the use of computers to sketch and visualize things in 3D, but he shares the concerns that Dave, et al, voice in that earlier RiH post with regard to the “distance” that displays/screens place between us and the virtual things that we are interacting with.

Unlike Ishi, and the physical computing crowd, Buxton couldn’t throw the screen away though, since 3D models generally only live on screens of one sort or another, at least for now. So, before he sold his soul and moved to Redmond, he built all sorts of interesting things that tried to tie the manipulation of physical artifacts and physical gesturing with what was being experienced digitally, behind a display. Such approaches seek to marry the “roundtrip” sensations that we get when our hands and bodies are involved in manipulating objects in the world with virtual objects living behind those pesky, alienating screens.

One such device that he had conjured that really stuck with me is the one pictured here. It consisted of several LCD monitors, each connected to a Luxo-type arm, so that it could be held in hand and rotated around, as well as swiveled up and down around, a central point. In the physical world, that point was empty. But in the virtual world of those screens, it contained a 3D object (i.e. an automobile) being rendered in realtime so that the viewed perspective was correct to the (changing) position of the screen. The effect was like looking through a floating, movable window at the virtual object. And it was good. 😉