Reading #29: Scratch Input Creating Large, Inexpensive, Unpowered and Mobile Finger Input Surfaces (Harrison)

Comments

Chris

Summary

This paper explores an alternative approach tan the classical pen for sketch recognition. Instead of using a tablet this system uses the sound of sketching to recognize gestures. This very novel approach enables users that do not have a tablet or digital pen input at hand to interact with a computer using the benefits of sketch recognition. This system uses a microphone adapted to a stethoscope as an input sensor. Basing on the features of the resulting sound wave like frequency and amplitude profiles, a particular signature or sound profile can be associated with different gestures. Due to the lack of spatial information in sound it is very difficult to distinguish shapes that are spatially different but have very similar sound profiles (e.g. M and W). The paper shows several applications of the scratch input (walls, mobile devices, tables and even fabric). In a set of simple gestures to be recognized that are dissimilar enough the achieved accuracies are of around 90% which is very impressive for an input source that is so limited.

Discussion

This paper opens a new door in sketch recognition. In the sketch recognition class several projects were inspired by this research work. The sound input presents many limitations to detect complex shapes because the available features are very limited, however it is good enough to recognize simple gestures, and provides an inexpensive and portable way of having an extra input source. I like the idea of some of the students in this class of having several microphones as input, this limits portability but improves accuracy so for instance, a blackboard with a grid of microphones behind can be used as an input device to control features of a classroom.