Reading #7. Sketch Based Interfaces: Early Processing for Sketch Understanding (Sezgin)

Comments on others

Oz

Summary

This paper addresses the problem of free sketch recognition. Unlike other papers dealing with gesture recognition posted here before, the free sketch does not intend that the user has a particular way of drawing a shape each time in order for the software to recognize it. This particular challenge defies previous approaches where a shape had to be drawn in the same way every time in order to be recognized accurately. For example in a “V” shape, the rubine features of starting and ending angles change dramatically if we start from the left or the right. This may be particularly annoying if the final application is a design tool for the early or creative stages of the process. So in the case you were using a Rubine or $1 for shape recognition in these types of applications and you were suddenly inspired about a design that came to mind, you then would have to stop the flow in order to remember how to draw a triangle “properly”.
The scope of the paper is in the early sketch understanding. And the approach they take to attack this problem is to start by identifying very low level geometric descriptions like lines and ovals. Their contribution was to develop a system that processed a stroke in three phases: approximation, beautification and basic recognition. The first intends to detect vertices in a shape in order to distinguish between the low level components of a shape. In order to do this they use several sources of information including curvature and speed. The second processes each of the detected components in order to make them look as the user intended. (i.e. make lines more straight, curves more smooth…). Finally basic object recognition is attempted to detect basic figures in the sketch such as ovals, rectangles and squares. This information could be used in specific application needs (e.g. the detection of a truss or a spring). The results in evaluation show a very good accuracy of 96% compared to 63% on previous works.

Discussion

The free sketch recognition attempts to take a natural human sketch in an early stage of a design process and make it into something understandable for the computer. I think a very important part in free sketch recognition is to identify the useful information in each particular domain. It is very greedy and perhaps useless to think of a general sketch recognizer at the high level. In other words, for instance an arrow in a mechanic engineering sketch could be identified as a force, and a rectangle as an object with mass, while in a UML diagram this arrow is an association and the rectangle is a class or interface. This paper addresses the low level shapes which I think is a good approach since only very basic shapes are really reusable across domains. Because of this, I think the mayor contribution is in the vertex detection, since the beautification and recognition could be improved substantially once the particular domain of the application is known.