Designing Hybrid Interactions through an Understanding of the Affordances of Physical and Digital Technologies

Two recent technological advances have extended the diversity of
domains and social contexts of Human-Computer Interaction: the
embedding of computing capabilities into physical hand-held
objects, and the emergence of large interactive surfaces, such as
tabletops and wall boards. Both interactive surfaces and small
computational devices usually allow for direct and space-multiplex
input, i.e., for the spatial coincidence of physical action and
digital output, in multiple points simultaneously. Such a powerful
combination opens novel opportunities for the design of what are
considered as hybrid interactions in this work. This thesis
explores the affordances of physical interaction as resources for
interface design of such hybrid interactions. The hybrid systems
that are elaborated in this work are envisioned to support specific
social and physical contexts, such as collaborative cooking in a
domestic kitchen, or collaborative creativity in a design process.
In particular, different aspects of physicality characteristic of
those specific domains are explored, with the aim of promoting
skill transfer across domains. irst, different approaches to the
design of space-multiplex, function-specific interfaces are
considered and investigated. Such design approaches build on
related work on Graspable User Interfaces and extend the design
space to direct touch interfaces such as touch-sensitive surfaces,
in different sizes and orientations (i.e., tablets, interactive
tabletops, and walls). These approaches are instantiated in the
design of several experience prototypes: These are evaluated in
different settings to assess the contextual implications of
integrating aspects of physicality in the design of the interface.
Such implications are observed both at the pragmatic level of
interaction (i.e., patterns of users' behaviors on first contact
with the interface), as well as on user' subjective response. The
results indicate that the context of interaction affects the
perception of the affordances of the system, and that some
qualities of physicality such as the 3D space of manipulation and
relative haptic feedback can affect the feeling of engagement and
control. Building on these findings, two controlled studies are
conducted to observe more systematically the implications of
integrating some of the qualities of physical interaction into the
design of hybrid ones. The results indicate that, despite the fact
that several aspects of physical interaction are mimicked in the
interface, the interaction with digital media is quite different
and seems to reveal existing mental models and expectations
resulting from previous experience with the WIMP paradigm on the
desktop PC.