Shape Displays: Spatial Interaction with Dynamic Physical Form

Shape displays are a new class of I/O devices that dynamically render physical shape and geometry. They allow multiple users to experience information through touch and deformation of their surface topology. The rendered shapes can react to user input or continuously update their properties based on an underlying simulation. Shape displays can be used by industrial designers to quickly render physical CAD models before 3D printing, urban planners to physically visualize a site, medical experts to tactually explore volumetric data sets, or students to learn and understand parametric equations. Previous work on shape displays has mostly focused on physical rendering of digital content to overcome the limitations of single-point haptic interfaces—examples include the Feelex and Lumen projects. In our research, we emphasize the use of shape displays for designing new interaction techniques that leverage tactile spatial qualities to guide users. For this purpose, we designed, developed, and engineered three shape display systems that integrate physical rendering, synchronized visual display, shape sensing, spatial tracking, and object manipulation. This enabling technology has allowed us to contribute numerous interaction techniques for virtual, physical, and augmented reality, in collocated settings as well as for remote collaboration. Our systems are based on arrays of motorized pins, which extend from a tabletop to form 2.5D shapes: Relief consists of 120 pins in a circular tabletop, a platform later augmented with spatial graphics for the Sublimate system. Our next-generation platform, inFORM renders higher resolution shapes through 900 pins (see Figure 1). The Transform system consists of 1,152 pins embedded into the surface of domestic furniture. To capture objects and gestures and to control visual appearance, we augment the shape displays with overhead depth-sensing cameras and projectors. In this article, we wish to introduce readers to some of the exciting interaction possibilities that shape displays enable beyond those found in traditional 3D displays or haptic interfaces. We describe new means for physically displaying 3D graphics, interaction techniques that leverage physical touch, enhanced collaboration through physical telepresence and unique applications of shape displays. Our current shape displays are based on prototype hardware that enabled us to design, develop, and explore a range of novel interaction techniques. Although the general applicability of these prototypes are limited by resolution, mechanical complexity, and cost, we believe that many of the techniques we introduce can be transferred to a range of special-purpose scenarios that have different sensing and actuation needs, potentially even using a completely different technical approach. We thus hope that our work will inspire future researchers to start considering dynamic physical form as an interesting approach to enable new capabilities and expressiveness beyond today’s flat displays.

• Human-Computer Interaction and Visualization

• Robotics