VEIL: How Visual Encoding Hijacking Induces Bias In Vision Models
Authors
Suranjana Sooraj (University of California, Davis), Xuyang Chen (UC Davis), Madhumitha Venkatesan (University of California Davis), Dongyu Liu (University of California at Davis)
Presentation
- Session
- Is the Model Even Thinking?
- Time
- Thursday, Nov 12, 08:18 – 08:27 (US/Eastern) · session 08:00 – 09:30
- Location
- Hall America south
Keywords
Time-series Classification, Chart-Based Representations, CNNs, Interpretability, Visual Encodings
Abstract
Rendering time series as chart images for CNN-based classification has become increasingly common in time-series classification (TSC). However, it remains unclear whether models learn underlying temporal patterns or rely on encoding-specific visual cues introduced by the chart design. We present VEIL: a systematic study that examines how different chart encodings influence learned representations using complementary analyses of similarity, transferability, and attribution. Attention-guided training appears to mitigate this effect when encoding sensitivity is consistently identified across multiple diagnostics, but provides limited or negative benefit when such signals are absent. These findings position VEIL within a broader question of how machines perceive visualizations---extending graphical perception from human readers to vision models---and show that visualization design choices shape learned representations in ways that warrant the treatment of chart-based TSC as a representation and measurement problem rather than a simple modeling decision.
For Practitioners
This paper will be of interest to visualization practitioners, machine learning engineers, and data scientists who use chart images or other visual encodings as model inputs for time-series classification. Practitioners can use VEIL to diagnose whether models are learning task-relevant temporal structure or relying on encoding-specific visual cues, helping them compare encodings, interpret attribution results more carefully, and treat visualization design as a consequential modeling decision.