Embedding Video into Collaborative Visualizations: Hybrid Gesture Interaction and Snapping Fisheye Support for Remote Sensemaking
Authors
Adrian Kristanto (Monash University), Tim Dwyer (Monash University), Maxime Cordeil (The University of Queensland), Jens Emil Sloth Grønbæk (Aarhus University), Benjamin Tag (University of New South Wales), Christophe Hurter (Fédération ENAC ISAE-SUPAERO ONERA, Université de Toulouse, France), Jesper Kjeldskov (Monash University)
Presentation
- Session
- The more the merrier
- Time
- Tuesday, Nov 10, 10:24 – 10:36 (US/Eastern) · session 10:00 – 11:30
- Location
- Hall America north
Links
Sign in to access the preprint PDF.
Sign in- Download Supplemental Material
Keywords
Collaborative Visualization, Video Conferencing, Gesture Interaction, Fisheye Lens
Abstract
Remote and hybrid data-analysis meetings force collaborators to split attention between shared visualizations and separate video galleries, weakening deictic reference, nonverbal grounding, and shared awareness. We present ManyHans, a web-based collaborative visualization system that embeds participants’ video streams directly into the visualization space and supports hybrid interaction through mouse input and mid-air hand gestures. ManyHans contributes three design ideas for collaborative visual analysis: (1) embodied pointing and annotation that let participants reference data with their hands inside the shared visual workspace, (2) on-demand video embedding that brings a participant into the visualization when presenting while preserving a conventional gallery view for exploration, and (3) a snapping fisheye lens that combines semantic zoom with Voronoi-based target expansion to stabilize gesture-based selection in dense network visualizations. We developed the system through a RITE-style iterative design process with 26 participants across nine group sessions, then validated the final design in three additional group sessions. Our findings show that embedded video and gesture-based reference improve engagement, grounding, and communication around data, but prolonged mid-air interaction introduces fatigue and reduced precision during exploration. From these results, we distill a central design implication for collaborative visualization: embodied interaction should be available as a lightweight presentation and handoff mechanism within a hybrid interface, rather than as the primary mode for sustained analysis.
For Practitioners
This paper presents a system called ManyHans for remote collaborative data visualisation where the collaborating participants video streams are embedded in the visualisation. The idea is that the participants to not need to shift attention between the visualisation and their collaborators' video streams in order to communicate effectively. The system features gesture interactions to point at notable parts of the visualisation as it is discussed and to add annotations in a natural handwriting style. The system was developed followed user-centred design principles and we report our findings about the trade-offs between embodied gesture interaction and embedded video streams which are beneficial for communication, and focused exploration, which is accomplished with less distraction with traditional mouse-based interaction. Our final system design is hybrid which separates video streams into a gallery view beside the visualisation during mouse interaction, but which embeds the video stream into the visualisation when hand gestures are used to point to parts of the visualisation.