Marks, Channels, and Dead Ends: Stop Running Graphical Perception Studies and Start Modeling Visualizations as Images
Authors
Khairi Reda (University of Illinois Chicago), Shambhawi Sharma (University of Illinois at Chicago), Luc Renambot (University of Illinois Chicago), Fabio Miranda (University of Illinois Chicago), Saeed Boorboor (University of Illinois Chicago)
Links
Sign in to access the preprint PDF.
Sign in
Keywords
Visualization evaluation, graphical perception, vision models, summary percepts
Abstract
Graphical perception studies are the visualization community's preferred tool for evaluating visualizations. By measuring how accurately people interpret various arrangements of visual marks and channels, they aim to establish best practices for encoding data visually. We argue this model of assessing visualizations is fundamentally flawed, and no amount of additional empirical studies will fix it. The problem is that visualization frameworks theorize effectiveness at the level of the encoder (i.e., which data-to-visual mappings work best for a given context). Human perception, however, consists of a fundamentally incompatible decoder that operates at a different level, namely, retinal images. This encoder-decoder asymmetry means that experimental results and guidelines are often poor predictors of actual perceptual performance. Moreover, the image that actually reaches the visual system is not determined by the encoding specification alone, but is rather emergent from interactions between encoding rules, the input data, and even micro-design parameters, all of which are invisible to encoding theory. Consequently, small changes in data distributions or seemingly minor design variations can substantially change how a visualization is perceived, even when the nominal encoding specification remains unchanged. The space of such interactions is vast and cannot be covered by running one study after another. We argue that visualizations should be studied as images, and evaluated using computational models of human vision that take pixels as input. Such models capture the perceptual representations the visual system actually constructs, shifting evaluation toward modeling the decoder rather than anchoring on abstract encoding specifications. This approach is scalable, human-grounded, and sensitive to the emergent image properties that encoding theory and graphical perception studies both miss. We first describe methodological and theoretical weaknesses of the current paradigm, and propose a theory of visualization perception grounded in summary-statistical accounts of vision. We then demonstrate how image-based vision models can predict visualization discriminability in scatterplots while reproducing established results. We close by outlining a research agenda for vision-based visualization evaluation.