Multimodal interaction: developing an interaction concept for a touchscreen incorporating tactile feedback

The touchscreen, as an alternative user interface for applications
that normally require mice and keyboards, has become more and more
commonplace, showing up on mobile devices, on vending machines, on
ATMs and in the control panels of machines in industry, where
conventional input devices cannot provide intuitive, rapid and
accurate user interaction with the content of the display. The
exponential growth in processing power on the PC, together with
advances in understanding human communication channels, has had a
significant effect on the design of usable, human-factored
interfaces on touchscreens, and on the number and complexity of
applications available on touchscreens. Although computer-driven
touchscreen interfaces provide programmable and dynamic displays,
the absence of the expected tactile cues on the hard and static
surfaces of conventional touchscreens is challenging interface
design and touchscreen usability, in particular for distracting,
low-visibility environments. Current technology allows the human
tactile modality to be used in touchscreens. While the visual
channel converts graphics and text unidirectionally from the
computer to the end user, tactile communication features a
bidirectional information flow to and from the user as the user
perceives and acts on the environment and the system responds to
changing contextual information. Tactile sensations such as detents
and pulses provide users with cues that make selecting and
controlling a more intuitive process. Tactile features can
compensate for deficiencies in some of the human senses, especially
in tasks which carry a heavy visual or auditory burden. In this
study, an interaction concept for tactile touchscreens is developed
with a view to employing the key characteristics of the human sense
of touch effectively and efficiently, especially in distracting
environments where vision is impaired and hearing is overloaded. As
a first step toward improving the usability of touchscreens through
the integration of tactile effects, different mechanical solutions
for producing motion in tactile touchscreens are investigated, to
provide a basis for selecting suitable vibration directions when
designing tactile displays. Building on these results, design
know-how regarding tactile feedback patterns is further developed
to enable dynamic simulation of UI controls, in order to give users
a sense of perceiving real controls on a highly natural touch
interface. To study the value of adding tactile properties to
touchscreens, haptically enhanced UI controls are then further
investigated with the aim of mapping haptic signals to different
usage scenarios to perform primary and secondary tasks with
touchscreens. The findings of the study are intended for
consideration and discussion as a guide to further development of
tactile stimuli, haptically enhanced user interfaces and
touchscreen applications.