A Query-Efficient Stochastic Volume Rendering Framework for Time-Varying Implicit Neural Volumes
Authors
Alper Sahistan (SCI Institute), Haichao Miao (Lawrence Livermore National Laboratory), Zhimin Li (Vanderbilt University), Peer-Timo Bremer (Lawrence Livermore National Laboratory), Joshua A Levine (University of Arizona), Valerio Pascucci (University of Utah)
Presentation
- Session
- Turn up the volume(s)!
- Time
- Thursday, Nov 12, 13:24 – 13:36 (US/Eastern) · session 13:00 – 14:30
- Location
- Hall America north
Links
Sign in to access the preprint PDF.
Sign in- Download Supplemental Material
Keywords
Volume rendering, ray tracing, implicit neural representations, time series data, scientific visualization.
Abstract
Time-varying implicit neural representations (INRs) provide a compact representation of scientific volumes and, for modalities such as dynamic X-ray computed tomography (CT), are often the only practical way to represent the data. However, interactive volume rendering of INRs is challenging, as cheap memory lookups are replaced by expensive neural inferences, hindering the performance. Therefore, conventional volume rendering methods such as ray marching with dense sampling are often impractical. While resampling, caching, and retraining can mitigate this cost, they compromise convenience and accuracy and become impractical for time-varying data. We tackle these challenges using a query-efficient stochastic volume rendering framework based on delta tracking. Our system employs a four-stage pipeline that exploits heterogeneous parallelism, using ray tracing cores for traversal and tensor cores for batched neural evaluation. Furthermore, we present strategies to reduce INR queries via ray budgeting and query pruning, thereby increasing per-frame performance. Using our renderer, many time-varying INRs can be rendered directly from their original representation. The system achieves ~30-40 FPS at 1024x1024 resolution on an RTX 4090 GPU and converges to high-fidelity images. Moreover, the system enables interactive temporal exploration of the continuous domain, with timestep updates taking approximately 1-2 ms.
For Practitioners
Visualization/rendering researchers, neural materials and graphics researchers, simulation scientists, data scientists, and computer scientists can learn to interactively visualize time-varying data without relying on complex I/O systems or cumbersome data transformations if they already have implicit neural datasets.