Paper #13 - Perceptually Based Learning of Shape Descriptions for Sketch Recognition

Paper:
Perceptually Based Learning of Shape Descriptions for Sketch Recognition
(Olya Veselova and Randall Davis)

Summary:
The authors are interested in better understanding natural multi-domain sketch interaction. Their goal is to capture a sufficient enough shape description for system recognition. Much of their insight stems from Goldmeier’s work on perceived shape similarity, in which people are either biased or ignorant of certain geometric properties. Goldmeier refers to these properties as singularities, and these singularities and additions by the authors create a qualitative vocabulary for their approach. The authors then use Goldmeier’s second set of experiments which gave those singularities different perceptual importance. From the authors’ own analysis, they’ve created a default relevance ranking of supported constraints in their approach. Using heuristics, they have adjusted those relevance scores based on obstruction (i.e., number of geometric primitives between two shape parts), tension lines (i.e., alignment of corners of a shape), and grouping (i.e., combination of individual elements of a shape as a whole).

A user study was conducted to determine if their system produced the same classification as people. 33 people were shown 20 variations of 9 different symbols, where the system’s performance was measured by the proportion of times the system agreed with the majority answer. When the majority agreed over 80% of the time, the system was in that majority 83%. When it guessed randomly, the system agreed with the majority 50% of the time. Of the system’s errors, most of them were false negatives attributed to it paying more attention to individual detail of symbols than people in general. For false positives, they stemmed from lack of global symmetry detection and “must not” constraints.

Discussion:
Unlike the papers which primarily focusing on attack sketch recognition problems by employing techniques to better recognize shapes in isolation, I found it interesting that this paper instead focuses on understanding shapes at the perceptive level. This is important as well, since systems will attempt to recognize shapes that are not in isolation. The author’s goal of finding constraints that coincide with how people differentiate between shapes seems just as important in making more powerful sketch recognizers. In addition, this paper thus far cites the oldest source (1923!).

In this particular paper, I see faults in shapes being limited to only lines and ellipses. These shapes only cover a fraction of symbols that people realistically draw, so a logical next step would be to expand the study which incorporates more complex shapes.