Assessing Icon Appropriateness and Icon Discriminability with a Paired-Comparison Testing Procedure

Traditional methods of evaluating icon comprehension and discriminability have relied on a sequence of multiple screening tests to measure various aspects of icon meaning, image content, and the user's perception of the icon. The most frequently used procedures have been the icon appropriateness test to determine the best conceptual design from a group of icon candidates, followed by the icon matching test to ensure that individual icons are not confused with others in a set. This paper describes an automated paired-comparison test procedure that provides reliable measures of both icon appropriateness and icon discriminability using the same test method with a single metric. The procedure was validated in two experiments involving the design and evaluation of two different mouse-pointer icons. In Experiment 1, the procedure was used as an icon screening test to determine the most appropriate and meaningful icon that best represented each concept from two different sets of proposed icon variants. In Experiment 2, the same procedure was then used to confirm the discriminability of the final icon selections, and to verify the accuracy of results from the initial appropriateness test.