Reading #2: Specifying Gestures by Example (Rubine)

Comments on others

Chris Aikens

Summary

In this paper Rubine introduces his toolkit GRANDMA, which serves as a tool to create gesture-based manipulation interfaces in a quick yet effective manner. These kinds of tools represent a mayor breakthrough in the world of gesture-based interfaces. One of the biggest barriers in the development of gesture based interfaces was the difficulty on creating them, this problem is now reduced to the correct usage of the right tools to automatically create a gesture based interface. As an example of its potential Rubine uses GPD, a program built upon the GRANDMA toolkit. In this program users can use gestures, both to sketch shapes and issue actions upon the shapes drawn (rotate, delete, copy…). Rubine explains how the gesture interface was added to GPD using GRANDMA in a relatively easy way.
The later part of this paper briefly describes the heart of GRANDMA in its statistical single-stroke gesture recognition. Every stroke or gesture is represented in the computer as a collection of 2D points in space and time, after some preprocessing is made some features are calculated from the stroke data. A feature in this case is a single numeric value that can be extracted from the gesture data; ideally a feature should be cheap to calculate (constant time per input point) and meaningful to the recognition of the shapes. Rubine proposes a way to determine from a stroke to which class (shape) it belongs, parting from the gesture, a set of features and an equivalent set of weights (importance) per each class or shape to be recognized linear classification is used to match strokes and shapes. The mentioned weights moreover do not have to be calculated by the programmer but instead they can be extracted by a set of examples for each class. Rubine finally exposes successful empirical results and discuses the extensions of his work and future directions.

Discussion

The work of Rubine already has shown its importance. The final part of the conclusions show some of the popular applications inspired on GRANDMA amongst them Garnet for the Palm Pilot interfaces and NeXT probably used as a predecessor for apple’s gesture recognizers. And these two already represent the most popular gesture-based products in the market.
I think that a key point of its success is simplicity, complex solutions do not tend to succeed. This one was easy and reliable and also the fact that the code for recognition is no longer hand coded gives much more flexibility and maintainability to the system.