IEEE VIS 2024 Content: Towards Inline Natural Language Authoring for Word-Scale Visualizations

Towards Inline Natural Language Authoring for Word-Scale Visualizations

Paige So'Brien - University of Calgary, Calgary, Canada

Wesley Willett - University of Calgary, Calgary, Canada

Screen-reader Accessible PDF
Download camera-ready PDF

Room: Bayshore II

2024-10-14T16:00:00ZGMT-0600Change your timezone on the schedule page
2024-10-14T16:00:00Z
Exemplar figure, described by caption below
This image is a screenshot of an editor application where authors can create and embed word-scale visualizations for text using LLM capabilities. The screenshot of the application includes a text area where authors can add their content. Below the text area there is a search bar for authors to submit plain language instructions for creating a visualization. In the text area, the numbers 1 2 3 4 are highlighted and used to generate a bar chart of the four values displayed inline with the text.
Fast forward
Abstract

We explore how natural language authoring with large language models (LLMs) can support the inline authoring of word-scale visualizations (WSVs).While word-scale visualizations that live alongside and within document text can support rich integration of data into written narratives and communication, these small visualizations have typically been challenging to author. We explore how modern LLMs---which are able to generate diverse visualization designs based on simple natural language descriptions---might allow authors to specify and insert new visualizations inline as they write text.Drawing on our experiences with an initial prototype built using GPT-4, we highlight the expressive potential of inline natural language visualization authoring and identify opportunities for further research.