Reading #12. Constellation Models for Sketch Recognition. (Sharon)

Comments on others

Danielle

Summary

In this paper recognition is based upon a constellation model or `pictorial structure' is used to aid in recognition. This basically implies that the recognition is not only based on the features of individual shapes but in the context around them.  The distance and position of each recognized subshape relative to the others becomes important in labeling it as one thing or another. Some of the shapes are mandatory and some are optional, in the example of a model constellation of a face the mouth, eyes and nose are mandatory and the ears can be modular leading to the model shown in the figure below. In this case individual and pairwise features are calculated to process recognition, but the pairwise features are only calculated between mandatory parts to reduce time complexity. Also mandatory labels are assigned first in order to provide a better context to the usually larger number of optional parts.

Discussion

This paper is a very good example of the use of context in sketch recognition. In this case shape labeling is not only made based on geometric features of individual shapes but also in how they are located relative to each other. Also I find it interesting that it is highly inspired in computer vision, which allows sharing techniques and algorithms in both worlds.

Reading #11 LADDER, a sketching language for user interface developers. (Hammond)

Comments on others

Jonathan

Summary

LADDER is a language that allows describing shapes in a high level almost natural language and then from this description automatically generates Sketch-based interfaces. These interfaces allow the user to draw shapes in a natural way and these shapes will be recognized, beautified and allow editing as described in LADDER language. The paper shows some related work but it seems there is nothing really alike LADDER to be formally compared; as a complete system is very innovative. Thus its contributions are very important to the field of sketch recognition, introducing the need of higher level languages to quickly generate sketch based interfaces and to reach a wider audience when it comes to the development of such interfaces.
LADDER does not intend to be a universal sketch recognizer builder; it is focused in diagram-like sketch interfaces with a fixed graphical grammar. A Shape in LADDER is defined by its components, constraints, aliases, editing behaviors and display methods. There are many features in LADDER that allow easy description of these attributes and methods: Hierarchical shape definition, abstract shapes and shape groups. This comes along with useful predefined shapes, constraints and display methods. A predefined shape beautification is also available based on the specified constraints using equation solving an ideal shape can be extracted out of a rough sketch.

(define shape OpenArrow
(description "An arrow with an open head")
(components
(Line shaft)
(Line head1)
(Line head2))
(constraints
(coincident shaft.p1 head1.p1)
(coincident shaft.p1 head2.p1)
(coincident head1.p1 head2.p1)
(equal-length head1 head2)
(acute-meet head1 shaft)
(acute-meet shaft head2))
(aliases
(Point head shaft.p1)
(Point tail shaft.p2))
(editing
( (trigger (click_hold_drag shaft))
(action
(translate this)
(set-cursor DRAG)
(show-handle MOVE tail head)))
( (trigger (click_hold_drag head))
(action
(rubber-band this head tail)
(show-handle MOVE head)
(set-cursor DRAG)))
( (trigger (click_hold_drag tail))
(action
(rubber-band this tail head)
(show-handle MOVE tail)
(set-cursor DRAG))))
(display (original-strokes)))

Discussion

By the time of this paper LADDER was still an early prototype considering its potential. The idea of a language able to describe shapes that is useful to automatically generate sketch recognizers to a particular domain is very powerful. The results and prototypes obtained with LADDER already show this power and also show that LADDER itself is a very good implementation of such idea. One of the usual tradeoffs in this kind of high level languages is between ease of use and scope of the language. As more complex domains are included either the language goes short of expressiveness or the language goes more complex. However since the scope of LADDER is limited the language is easy or natural enough to use by most developers and yet covers a very high range of its scope if not all.

Reading #10. Graphical Input Through Machine Recognition of Sketches (Herot)

Comments on others

Chris

Summary

This is one of the early papers in free-hand sketching. It presents the HUNCH system, a set of software programs that process a sketch in order to advance towards a general sketch recognizer. The work does not focus on a particular domain but rather tries to explore techniques that may work for several domains. Amongst these techniques they explore corner finding, latching, use of context and over tracing. Many of this properties are now of common use in sketch recognition. This paper emphasizes in heavily involve the user towards the agreement on the machine interpretation in contrast of having a fully automated machine that probably does not really reflects the user intentions.

Discussion

This paper was published in 1976 yet it already covers many of the aspects of preprocessing in sketch recognition that are widely used today. For instance the corner finding algorithm used by STRAIT is very similar to the one presented by Sezign in 2006 as they use curvature and speed. Although the paper is rather general and does not go really deep into specifics, solving the problems found in recognition, its early publishing date most likely inspired many of the papers that were published recently.