ProgressiVis: A Language and Environment for Progressive Data Analysis and Visualization
Authors
Jean-Daniel Fekete (Université Paris-Saclay, CNRS, Inria), Christian Poli (Inria)
Presentation
- Session
- Let's dig into the data (from France)
- Time
- Wednesday, Nov 11, 09:12 – 09:24 (US/Eastern) · session 08:00 – 09:30
- Location
- Hall America north
Links
Sign in to access the preprint PDF.
Sign in- Download Supplemental Material
Keywords
Progressive Visualization, Progressive Visual Analytics, Progressive Data Analysis, Latency, Scalability.
Abstract
We introduce ProgressiVis, a language and graphical user interface designed to scale interactive visualization and visual analytics by relying on the Progressive Data Analysis (PDA) programming paradigm. Existing systems cannot remain interactive for open-ended exploration with large datasets and complex analyses because their processing time increases with size and complexity; at some point, time exceeds human latency limits (≈10 seconds). PDA systems guarantee that their latency remains within specified limits. They split long computations into smaller execution chunks so that processing a large dataset or applying complex computations still takes time, but they produce intermediary results continuously, improving quality and allowing analysts to monitor the computation, interact with it, and make decisions, often before the whole computation is complete. Implementing PDA systems is not currently supported by any programming environment, forcing programmers to reimplement the data analysis and visualization components from scratch, a long and difficult task. ProgressiVis is a language and system designed to run its programs progressively, including data management, data analysis, and visualizations. It introduces novel programming constructs and provides new services, such as progressive quality assessment, as well as JupyterLab notebook extensions to better support progressive visualization and program execution. This article describes the language, shows examples of ProgressiVis programs that perform visualization and analysis at scale, and explains how it addresses the technical and scientific issues raised by PDA, enabling data exploration to reach new limits.
For Practitioners
This work is intended primarily for data scientists and practitioners needing scalability in data exploration. It is also interesting to visualization and visual analytics developers who need scalability with controlled latency, as it shows how to implement Progressive Data Analysis.