VIS Panels: Challenges and Opportunities for Evaluating AI-Assisted Generation of Visualizations

Session chair:Devin Lange, Astrid van den Brandt, Huyen N. Nguyen, Nils Gehlenborg

Room:America South

Time:Tuesday, Nov 10 @ 10:00 - 11:30 (US/Eastern)

Evaluating which visualization is correct for a given situation is a longstanding question in the visualization research community, and even claiming that there are correct and incorrect visualizations is debated. With the advancing capabilities of large language models, visualizations are now being constructed, on demand, in response to natural language requests. But how do we know whether such systems produce acceptable visualizations? In this panel, we will probe this high-level question and explore its implications for the visualization research community. We will explore different aspects that make this a challenging and relevant question: scale, plurality, technical ecosystems, generalizability, and trust. These dimensions provide structure to the wicked problem of evaluation. Through this discussion we will highlight the immediate practical challenges this question is surfacing and will discuss the epistemological underpinnings of benchmarking and visualization research.

Presentations in this session