Through the emergence of ever more intelligent devices and the progressive technological development, the way we interact with devices surrounding us will change in the future. The implementation of touchless interactions into everyday life may become the next paradigm shift in interface design. In this context, we have conducted a comprehensive study on the conception and use of touchless microgestures.
For our research, we designed microgestures from the ground up, both the motion sequence and the function triggered with it. The developed gestures were evaluated on the basis of prototypes and user tests. From the gained knowledge, we developed design rules as well as principles for the design of gesture-controlled interfaces.
In order to test and evaluate the developed microgestures, we created various functional prototypes. For the realization of the prototypes, we selected the motion capture camera Leap Motion as tracking tool for the hand motions. With these functional prototypes we conducted expert interviews and usability-tests.
While testing the usability of each prototype respectively microgesture, we focused on six previously defined criteria, which we later visualised in a spider chart. With the gathered data about the criteria and the gained insights from the interviews, the microgestures were accordingly adjusted and improved.
We eventually applied the revised microgestures to real use cases to see how they work out in everyday use. To simulate an everyday use we applied gesturecontrol interfaces to a living-room-like setting. Within this setting, we installed an AppleTV and a lamp, which both can be controlled only by using microgestures.
For the AppleTV we utilized the gestures to navigate through the User Interface. The slide gesture was used to navigate horizontally, the scroll gesture to navigate vertically. With a tap on the left or on the right side of his finger, the user can move through the navigation hierarchy.
The lamp can be controlled by using the tap gesture to turn it on and off or by using the drag and drop gesture to dim the light.
To apply the concept of microgestural interaction to a device or a general use case, we had to define the context and functionality. Since touchless interactions always had the problem of missing feedback, the user often does not recognize when he is detected by the device or when a certain function was triggered.
The following interaction concept constrains the possible uses to avoid that functions are unintended triggered and helps the user to control the device more enjoyable. The first method to improve the interaction is to limit the area of gesture recognition. The user has to point with his hand towards the object if he wants to interact with it. His arm vector has to hit a certain area around the device to be recognized by it.
The second parameter that should be adjusted is the time the device needs to recognize the user. If the user points towards the device, he has to stay within the area for a certain amount of time to be noticed. This ensures that only conscious movements by the user are recognized as gestures and the natural behaviour will not interfere with it.
This is a student project created during 3rd semester 2016/17 as part of a course in Invention Design — by Jörg Beck.
Max Mertens Janis Walser-Cofalka