Paper #19 - Three main concerns in sketch recognition and an approach to addressing them

Paper:
Three main concerns in sketch recognition and an approach to addressing them
(James V. Mahoney and Markus P.J. Fromherz)

Summary:
The problem which the author wishes to tackle is matching models of curvilinear configurations to hand-drawn sketches. Three sketch recognition concerns are introduced: 1) coping with variability and ambiguity of sketches, 2) providing interactive performance or better, and 3) offering easy or automatic extensibility to new shapes and configurations. Each of these concerns is addressed in a framework.

For the first concern, variability and ambiguity need to be addressed. Since variability also corrupts one-to-one mapping between a model of a figure and its instance, a lack of one-to-one mapping between segmentation primitives and model elements causes segmentation ambiguity. The authors handle this ambiguity by using dedicated pre-processing to address the effects of drawing variability. This is done by first creating subgraphs in the data graph of descriptions, applying alternative descriptions using a general perceptual organization. Then the descriptions are rectified by finding the alternative descriptions which are preferred towards the rest: proximity linking, link closure, virtual junction splitting, spurious segment jumping, and continuity tracing.

For the second concern, a solution is sought out for coping with the difficulty in efficient structural matching. The first step is accomplished by exploiting internal constraints to the sketch problem. The authors develop a constrained optimization approach to subgraph matching, using a data graph from standard image processing operations, for recognition in this case. The second step uses any available external constraints focused on as added criteria to limit the search in the prior step in order to improve processing time.

For the last concern, a technique was proposed to automatically learn models from user specified examples. The proposed automated model would acquire new examples using a highly effective learning scheme. This would be done by having the scheme use as few training examples as possible, preferably positive examples, in an incremental fashion.

Even though this matching approach deals with mostly curvilinear configurations, the authors claim it can extend naturally to other abstract relations among strokes.

Discussion:
I can see why Dr. Hammond had this paper originally assigned as reading prior to the Sharon paper, since the Sharon paper shares the same idea of labeling parts for more complex shapes. Naturally, the Sharon paper cites this paper, but it was cited by stating that matching parts could be done as an isomorphic graphing problem, something which that paper did not do at all. I completely agree with the authors stating that the three main concerns introduced in the paper are important in sketch recognition system, but their approach to cope with those concerns suffer the same faults that I saw in the Sharon of doing part matching of symbols. There are some important classes of symbols that can't be split into parts intuitively, thereby limiting the usefulness of the approach in this paper similarly.