Paleo Links

By request of a lab-mate I post here the links to the current version of PaleoSketch and its supporting libraries. Note that these links will be available for a limited time only until we release an official version.

paleo
libraries

Reading #30: Tahuti: A Geometrical Sketch Recognition System for UML Class Diagrams (Hammond)

Comments

Jonathan

Summary

Sketch recognition is important because of it empowers the user with computer edition tools yet having a natural interface as in pen and paper. A clear application of this neat combination is Tauthi. In this work Hammond and Davis present a tool for drawing and editing UML diagrams using a pen input device. The work compares this implementation for drawing UML diagrams with a popular UML design application (Rational Rose) and with a popular drawing application (paint). The program offers two main modes, one where the strokes are beautified after recognition and one where the strokes are preserved as drawn by the user. The results show that users prefer the use of interpreted tauthi over paint and rational rose.
The software relies on several recognition techniques that are explained in the paper, such as corner finding, filtering, text detection (not recognition yet but planned as future work), grouping and others. The interface uses strokes both for drawing and for editing commands such as delete and move. Depending on the viewmode the strokes are edited accordingly to maintain consistency of the diagram.

Discussion

For us computer scientists that have to deal with UML diagrams, this is a magnificent application. I associated sketch recognition with diagram drawing since it is a domain were drawing with a marker in a blackboard feels much more natural. I usually draw UML first in a piece of paper or blackboard and then “beautify it” by doing it in Rose, Poseidon or any other UML tool. With this application however it may not longer be the case since hand drawings can be easily transformed into nice UML diagrams. However to make this a reality we still have path to cover not only in the UI and the recognizers but also in the available hardware, digital blackboards and notebooks are available now, but still are usually not affordable or present imprecise recognition. But I think this is a big step in the right direction.

Reading #29: Scratch Input Creating Large, Inexpensive, Unpowered and Mobile Finger Input Surfaces (Harrison)

Comments

Chris

Summary

This paper explores an alternative approach tan the classical pen for sketch recognition. Instead of using a tablet this system uses the sound of sketching to recognize gestures. This very novel approach enables users that do not have a tablet or digital pen input at hand to interact with a computer using the benefits of sketch recognition. This system uses a microphone adapted to a stethoscope as an input sensor. Basing on the features of the resulting sound wave like frequency and amplitude profiles, a particular signature or sound profile can be associated with different gestures. Due to the lack of spatial information in sound it is very difficult to distinguish shapes that are spatially different but have very similar sound profiles (e.g. M and W). The paper shows several applications of the scratch input (walls, mobile devices, tables and even fabric). In a set of simple gestures to be recognized that are dissimilar enough the achieved accuracies are of around 90% which is very impressive for an input source that is so limited.

Discussion

This paper opens a new door in sketch recognition. In the sketch recognition class several projects were inspired by this research work. The sound input presents many limitations to detect complex shapes because the available features are very limited, however it is good enough to recognize simple gestures, and provides an inexpensive and portable way of having an extra input source. I like the idea of some of the students in this class of having several microphones as input, this limits portability but improves accuracy so for instance, a blackboard with a grid of microphones behind can be used as an input device to control features of a classroom.

Reading #28: iCanDraw? – Using Sketch Recognition and Corrective Feedback to Assist a User in Drawing Human Faces (Dixon)

Deja vú… Oh! yeah, it was already posted several days ago. Here it is.

Reading #27: K-sketch: A 'Kinetic' Sketch Pad for Novice Animators (Davis)

Comments

Jonathan

Summary

Animators in the world today have many tools to work with and the internet has shown that they are very interesting ideas from people with all types of backgrounds that not necessarily have an animation background. For this novice user as well as for the experienced user that needs a fast prototype of his idea, a tool that enables a quick transition between the creative stage and a simple animation is highly desired. Usually the animation process begins with a storyboard. K-Sketch basically enables the user to create a sketched storyboard that can be animated immediately allowing the user to better visualize the idea.
The paper discusses the implementation of K-sketch as well as the user interface. Studies took place to determine the optimal set of operations in the UI. The resulting tool was tested and compared to MS Powerpoint as a reference of simple animation. The users preferred K-Sketch as it showed a more natural way of animating objects.

Discussion

In fact animation tools today still have a relatively steep learning curve, plus they don’t feel as natural as the storyboards drawn on pen and pencil. In my case I used to spend some time in middle school doing sketches in the top corner of notebooks to pass pages rapidly “animating” the sketches. Off course the task was tedious but hey, what else there is to do in middle school? I guess kids now can instead draw on simple sketch in their tablets and enjoy the magic of animation using tools like K-Sketch in very simple steps. I like very much the path of investigation of this paper, creating magic with a pen is made easier.

Reading #26: Picturephone: A Game for Sketch Data Capture (Johnson)

Comments

Jonathan

Summary

This is a more detailed and particular description of the PicturePhone game described in reading #24. The Picturephone game consist of 4 consecutive steps

• Player A is given a text description, and must make a drawing that captures that description as accurately as possible.
• Player B receives the drawing and describes it in words.
• Player C is given Player B’s description and draws it.
• An unrelated player Player D is asked to judge how closely Player A and C’s drawings match, which assigns a score to players A, B, and C.

This enables to collect information on both word to sketch interpretation and also from sketch to words. The implementation allows researchers to use the collected data for their purposes such as training recognizers.

Discussion

Whooop! Short paper! However short, it gets to the point and accurately describes the application. The picturephone game is slower than Pictionary-like games, this can make it boring and not as fun. However it may report more reliable labels as they reward the accuracy of the picture and the descriptions. It serves as a nice compliment to the user that enjoys this type of low paced games.

Reading #25: A Descriptor for Large Scale Image Retrieval Based on Sketched Feature Lines (Eitz)

Comments

Sam

Summary

Internet changed the way we use and retreive information, now the problema is that there is so much information that efficient queries are necessary. Searching engines like google have done an amazing job at searching text. However searching for images is not always easy, particularly when the images in the database do not contain metadata. The paper proposes a way of searching images based on sketches of the desired image. The problem here is that the processing in each image has to be simple enough to allow the query to return results in feasible time. The contribution is a descriptor based on structure tensors that allows easy and efficient implementation to obtain query results in low times over a large database (less than 4 seconds for a 1.5 Million images database).

Discussion

This application shows a really interesting way of doing queries. The results are very impressive in terms of time and accuracy. The resulting images are not always as expected (draw a building return a railroad) but this is not necessarily a bad thing, as actually it gives the user the feedback that he draw a building that looks like a railroad. I don’t think sketch queries will replace text queries for images, as in usual cases it is easier to search for “tree” than to draw a tree. However they are multiple cases where this can be a very nice option as when a very abstract picture is searched or the user is not really sure of how to describe the image he clearly has on his mind. Also this enables images in the result set that do not have the adequate metadata, and allows Multilanguage search. (If I search for “tree” the image will metadata “arból” will most likely be discarded even if they mean the same).