Paper #4 - "These Look Similar!" Issues in Automating Gesture Design Advice

Paper #1:
"These Look Similar!" Issues in Automating Gesture Design Advice
(Chris Long, et al)

Summary:
New interaction technologies in state-of-the-art interfaces can be difficult for interface designers, since they aren’t familiar enough with these technologies to incorporate them in their interfaces. Traditional interface tools provide little help, so the quill design tool was created to give unsolicited advice to designers on creating and improving their interface. Gestures are fast, commonly used, and easier to remember than text commands, but can be difficult for people to learn and computers to recognize. quill tries to resolve this by continually analyzing gestures of both problem types.

Experiments were done to gauge how many people judged expressions similarly, which yielded results of 99.8% accuracy for dissimilar gestures and 22.4% accuracy for similar ones. Thus, quill was developed to warn designers of gestures potentially similar to people. The gesture recognizer is trained by the designer with the Rubine algorithm by drawing ten to fifteen example gestures per gesture class, and then into gesture groups if there are many classes. Designers then test gesture recognition by drawing them. quill uses an active feedback system to offer unsolicited advice about any problems. It first analyzes gestures, then offers advice when the designer pauses. When gestures are similar or misrecognized, warnings appear with advice on fixing them.

Pilot studies of quill demonstrated various issues related to advice. The first concerns the user interface. Users were originally forced to ask for advice, but most did not, so quill gave users unsolicited advice instead. When given too late, users usually ignored them. Given at the earliest, and it may distract or interrupt the user. Given too soon, and the information may get out-of-date. Thus, advice is delayed and given as soon as a gesture is tested and evaluated. Concise meanings with hyperlinks to details in English prose and graphics are given in advice. For implementation issues, analyses are done in the background with the aim to increase user freedom and decrease user confusion. In quill, analysis can be started automatically by the system or manually by the user. User actions are disabled in advice computation for user-initiated analyses, while any action is allowed with affected analysis cancelled in system-initiated ones. The last issue was the metric which quill’s models used to predict human-perceived similarity. Users disagreed with it at times, especially when the models overestimated similarity to gestures with letter-based characteristics. The flaw was attributed to the models being based on non-letter data, so separate models for letter and non-letter shapes would be ideal.

Paper #2:
Visual Similarity of Pen Gestures
(Chris Long, et al)

Summary:
Gestures, or pen-based commands, are desirable features in pen-based user interfaces for their speed, rate of use, and iconic meanings. Problems exist with current gesture use due to difficulty for people to remember and computers and recognize. The researchers in this paper conduct gesture similarity experiments to be used in gesture design tools. Previous related work on gestures have been used in devices, such as Apple’s Newton and 3Com’s Pilot, and in various applications. Previous psychological experiments on geometric shapes simpler than gestures have been done to determine perceived similarity based on people’s stimuli and metric. INDSCAL, a version of multi-dimensional scaling (MDS), is a technique used by the researchers to reduce data set dimensions in order to see patterns plotted out.

Two trials were conducted to understand principles people use to judge gesture similarity. In the first trial of the gesture similarity experiments, a set of widely diverse gestures were shown to participants on the monitor in triads (groups of three gestures). All possibilities were generated randomly, and the participants had to choose the least similar-looking gesture to them. The goal of the experiment was to determine measurable similarity properties of gestures and to produce a model of gesture similarity. From the MDS plots, short, wide gestures registered as similar to narrow, tall ones, and both types differed from square ones. Angle of bounding box contrasted thin and square gestures, but not tall vertical and short horizontal ones. Aspect features were created to represent this.

For the second trial, three new gestures sets were designed to explore: 1) total absolute angle and aspect, 2) length and area, and 3) rotation-related features such as cosine and sine of total angle. In addition, relative importance of the features were being tested. The procedure and analysis used in this trial was similar to the one used in the previous one. For the results in determining similarity, the significance of the absolute angle couldn’t be determined in the first set, neither length nor area were significant in the second set, and gestures with horizontal and vertical lines were perceived more similar than diagonal lines.

Based on both trials, length and area were insignificant features in judging similarity, while the logarithm of aspect were more significant than the aspect itself. Both similarity experiments resulted in different similarity models, but the researchers prefer the model from the first trial since it was a slightly better similarity predictor.

Discussion:
I found the advice timing aspect of the user interface challenges in the first paper quite humorous. For some reason, the problem of when to display unsolicited advice reminds me a lot of the paper clip from older versions of Microsoft Office applications. That paper clip was so annoying. I can understand the difficulties the authors faced in trying to balance user convenience with program accuracy.

For the second paper, there were two areas that the authors brought up in the without going into much detail. I’m not exactly sure why it was omitted in the paper, because they seemed quite relevant. The first area concerned the gestures created by one of the authors for the two trials. According to the paper, the gestures were created based on one of the author’s personal intuition of spanning a wide range of possible gesture types and differences in orientation. I wish there was more elaboration on the specifics behind those gestures, since the trials were based on their uses. The second area concerned the results of the trials. According to the authors, the differences of the participants’ results from the first trial split into two different groups. The authors mentioned analyzing those groups, but didn’t mention what those differences were (to my knowledge). A pity they left out that information as well.