Road's End - Reflections on the VIS 2025 Review Process
IEEE VIS 2025 in Vienna has come to a close and we—Niklas Elmqvist, Holger Theisel, and Melanie Tory, the 2025 Overall Papers Chairs (OPCs)—have reached the end of the road. Melanie will continue as OPC for VIS 2026 in Boston together with Tobias Isenberg (Inria) and Alex Endert (Georgia Tech), whereas Niklas and Holger (together with OPC assistant Petra Specht) can step down with what we think was a successful full paper program behind us. But before we do, we would like to report on the changes we made to the VIS review process this year.
To help us understand the impact of these changes, we administered a survey to the IEEE VIS 2025 international program committee (IPC) asking about their experiences this year and their thoughts about the coming year. We can summarize the findings from 97 responses as follows:
- The review load of approximately six papers (half as primary and half as secondary) was perceived to be just about right for most IPC members.
- 61.9% of IPC members felt that keeping three reviewers (2 PC members and 1 external) was acceptable; 23.7% wanted to go back to four reviewers, and 14.4% were undecided.
- Allowing for an additional week for submitting supplemental materials was almost uniformly seen as positive, with 83.5% in favor, 8.2% undecided, and 8.2% against.
- The new student reviewer program was perceived as a positive initiative that most (69.1% in favor, 14.4% against, and 16.5% undecided) IPC members wanted to continue in future years.
The student reviewer program is intended to increase the reviewer pool and combat reviewer fatigue in our community by allowing primary reviewers to invite a junior Ph.D. student as an “advisory” reviewer for every submission. The student learns the ropes under the mentorship of the primary while also providing another set of eyes on each paper. This also seems to have worked as intended. Except for some problems with implementation (some reviewers who were invited as regular reviewers nevertheless marked themselves as student reviewers) and miscommunication (there are already Ph.D. students who are experienced reviewers in our community), the program was very successful:
- We received a total of 99 student reviews for 537 submitted papers, meaning that approximately 20% of primary reviewers invited a student reviewer for a paper.
- All student reviewers were duly credited as full reviewers for the VIS 2025 full paper program (exposing otherwise hidden labor), with some of them even receiving review distinctions.
- The IPC’s self-reported analysis of student reviews showed that they had no conclusive impact on review outcomes: only 5.6% of respondents felt the student review had a direct impact even if many positively remarked on the feedback student reviewers provided.
We are trying to improve some implementation details of the student reviewer program, but we are overall counting it as a success and are recommending that the VIS Steering Committee (VSC) continue it as an experimental program for at least three years before making a decision on its future.
Another experimental program that we wanted to report on is the VIS public peer review repository. Like the student reviewer program, this initiative was spearheaded by the VIS 2025 OPCs and approved by the VSC as an experiment for this year’s conference. The goal is to improve transparency in the VIS review process by publishing anonymized peer reviews for accepted papers in a public repository. Only accepted papers where both the authors and all reviewers agree are included in the repository. For VIS 2025, a total of 16 papers fulfilled this criteria, yielding a total of 52 reviews (three reviews per paper resulting in 48 regular reviews, as well as—interestingly—4 student reviews). The repository has been publicly published on OSF. We hope that future years will add to the corpus.
The public review program is perhaps a little more controversial than student reviewers. For example, in our survey, 16.7% of IPC members felt that the experiment should be stopped, whereas 65.6% felt it should continue, 12.5% wanted to relax the constraints so more reviews would be published, and 5.2% wanted to work towards open reviews. We think that there are clear benefits to making the VIS review process more transparent; for example, the four student reviews in the corpus give insight into that experimental program. Furthermore, the presence of public reviews can help new researchers learn the craft of peer review. However, several people have approached us with concerns; for example, that the review corpus could be used to fingerprint specific reviewers. Nevertheless, we count this year as a success and recommend that the VSC lets it continue for a full three years before being evaluated.
And that’s all, folks! It was our great honor to serve as your OPCs for the VIS 2025 conference. We could not have done it without the support of the 12 Area Papers Chairs (APCs), 205 IPC members, and many external and student reviewers from the community. And, as we said in the opening session, we had invaluable support by our indefatigable OPC assistant Petra Specht.
Thank you all! VIS 2025 OPCs signing off.
VIS 2022 Financial Blog Post
VIS 2022 was the 26th year of VIS, but in many ways it was a year of firsts: the first in-person meeting post-pandemic and the first hybrid VIS. Throw in the uncertainty of a change in both venue and organizational team leading to shortened planning time plus a dash of global uncertainty, and you have a wonderful blend of chaos and excitement.
One thing that people who have never witnessed the inner workings of a conference might not realize is just how complex and expensive the entire conference can be. Fiscal decisions range from as massive as choosing a venue to as small as choosing whether to have bagels or pastries for breakfast on a given day. In this blog, we’re aiming to provide some transparency into the income and expenses associated with VIS 2022. This transparency is in part in service to the community so participants know where their money goes. The goal of this post is to provide an approximate, high-level breakdown of IEEE VIS income and costs.
TL;DR: We didn’t bankrupt VIS! All kidding aside, with a lot of careful fiscal dialog and almost constant eyes on spending, supporters, and registration (using many uncertainty visualization and thanks to the experience of the Finance chairs), we ended up running a surplus of ~$140,000 (roughly 20%) for VIS 2022. This surplus will support other VGTC initiatives, including providing additional support for VIS 2024. The figure below summarizes the major funds and expenditures for the conference.
Income
| Income Category | Income |
|---|---|
| Registration | $513,000 |
| Supporters | $122,000 |
| Surplus Reinvestment | $39,000 |
| VGTC | $25,000 |
| Other | $10,000 |
| Total | $709,000 |
Costs
| Cost Category | Cost |
|---|---|
| Food and Beverage | $190,000 |
| A/V | $168,000 |
| Administrative Services | $74,000 |
| Conference Management | $34,000 |
| Travel Expenses | $32,000 |
| Onsite Expenses | $31,000 |
| Publication Fees | $14,000 |
| Facilities/Venue | $7,000 |
| Miscellaneous | $15,000 |
| Total | $565,000 |
Context
VIS 2022 was a hybrid conference, offering synchronous online and in-person participation. We hosted the conference in Oklahoma City, OK, USA with a total of 1,302 attendees (614 in-person, 496 paid virtual, and 192 complimentary virtual via the Diversity & Inclusivity program). The conference ran from October 16–21 and included workshops, panels, tutorials, keynotes, and paper sessions. A mid-week reception was held at the First American’s Museum.
Sources of Funds
VGTC
VGTC allocates a proportion of their budget annually to support VGTC conferences. Their ~$25,000 contribution helped support PCS, the Doctoral Colloquium, Student Volunteers, Diversity fellowships, and VISKids.
Registration
The biggest source of uncertainty and income (~$513,000) came from registration. We ended up with 952 paid registrants: 514 in-person and 438 online. Registration fees were kept in line with past conferences, and complimentary registration was offered to some participants to support diversity and inclusivity.
Supporters
Supporters provided ~$122,000 in funds in exchange for visibility and participation opportunities at the conference. New sponsors, including local OKC contributors, helped boost support to one of the highest levels in recent years.
Expenditures
Facilities
Hosted at the Omni Hotel and the First American’s Museum, facilities cost was relatively low at ~$7,000.
A/V
A hybrid conference meant significant A/V demands, totaling ~$168,000. Prioritization of events and volunteer support helped keep costs down.
Food & Beverage
Catering included breakfast, coffee breaks, and a mid-week reception, totalling ~$190,000.
Publication Support
Costs for publication systems and services came to ~$13,000.
Attendee Support
This included testing kits, diversity registrations, volunteer support, and hotel subsidies, totaling ~$47,000.
Administrative Services
IEEE administrative consulting and coordination cost ~$59,000.
Miscellaneous
Included advertising, planning, and supporter program management: ~$15,000.
In Summary…
VIS 2022 was a financial success, thanks to better-than-expected attendance, affordable venues, strong support, and massive volunteer contributions. Hybrid conferences are expensive and complex but can succeed with careful planning and community support. We invite continued conversation about future VIS formats.
Danielle Albers Szafir, UNC-Chapel Hill
Déjà Review - When You Have Seen the Paper Before
As IEEE VIS Overall Papers Chairs, we’ve observed a troubling pattern that undermines the integrity of our peer review process: papers rejected from one venue being resubmitted to the next with minimal or no revisions. While we understand the frustration of rejection, this practice damages our community and wastes our most precious resource: the time and goodwill of our people.
Yes, there is such a thing as Reviewer Roulette. Sometimes individual reviewers significantly impact a paper’s fate, and sometimes excellent work gets unfairly rejected. However, attributing every rejection to a “grumpy Reviewer 2” represents a cynical dismissal of the peer review process. This mindset prevents authors from extracting value from critical feedback and improving their work. It’s the same mindset as the willful child who never accepts blame and thus never learns and grows.
When a paper gets rejected, the fault often lies at least partly with the authors. Even groundbreaking research can fail to communicate its contributions effectively. If reviewers misunderstand your work, that signals a communication failure that you need to address. While it can feel frustrating that your brilliant work is rejected because of a trivial concern, this is the nature of the game at the top tier of academia.
So before you dismiss that negative review out of hand, consider this: a colleague volunteered hours of their time to read your paper carefully and provide detailed feedback. They did this without compensation. And the words they wrote were intended for you alone, likely never to see the light of day. Dismissing these comments without consideration not only disrespects this selfless labor, but it squanders an opportunity for improvement.
Reviewing represents significant labor that keeps our conferences and journals running. When authors submit unfinished papers “just to get reviews,” they impose substantial costs on the community. It’s even worse when they resubmit work unchanged to a new venue because this means that prior reviews are then effectively wasted. Each submission triggers hours of work from multiple reviewers, area chairs, and program committees. Ignoring this labor isn’t sustainable.
We’ve previously discussed these concerns in our post The Cost of Submission, but the issue persists. As OPCs and APCs, we regularly encounter papers that appear virtually unchanged from previous submissions. Many reviewers have reported similar experiences, often recognizing papers they’ve reviewed before at other venues.
If you’re reviewing a paper that seems familiar, speak up. (Note that while IEEE allows this practice, it seems that the ACM does not. That’s a bad call by the ACM for all the reasons outlined here.) In our view, you’re uniquely positioned to assess whether authors have addressed previous feedback. Including this observation in your review is not only appropriate but actively helpful to the process. Tell your fellow reviewers and PC members if a paper appears unchanged from a previous submission—and if you want to avoid being confrontational, add it to the private section of the review form. You may even copy and paste your old review unchanged; if the authors did not invest any effort into revising their work, why should you invest effort in reviewing it anew?
For authors, we offer this guidance: at minimum, address the surface-level comments from your reviews. Fix the typos, clarify the confusing passages, and strengthen the weak arguments that reviewers identified. Better yet, engage seriously with substantive criticism. When reviewers question your methodology, challenge your assumptions, or identify gaps in your evaluation, give this feedback due consideration. Sometimes reviewers are wrong, but more often they’re highlighting genuine weaknesses that, once addressed, will strengthen your contribution.
The peer review system, despite its flaws, remains our best mechanism for maintaining research quality. It depends on mutual respect between authors and reviewers. Authors deserve thoughtful and constructive feedback. Reviewers deserve to see their efforts valued through meaningful revisions. We can create a virtuous cycle where careful reviews lead to better papers, which inspire more thoughtful reviewing. Or we can spiral into a system where cynical resubmissions breed resentful reviews, degrading the process for everyone. The choice is ours.
VIS 2025 takes place in an anniversary year
Otto Neurath and the ISOTYPE - 100 Years of Visual Language
We could not have picked a more fitting year for IEEE VIS in Vienna!
2025 is a special anniversary year celebrating the development of a visual language. One hundred years ago, Otto Neurath (https://en.wikipedia.org/wiki/Otto_Neurath) and his team in Vienna began developing what would become the ISOTYPE - short for the International System of Typographic Picture Education. What started in the context of post-World War I Austria as a practical method for educating the public about social and economic conditions has since become an important reference point in the history of visual communication.
Otto Neurath (1882-1945) was an Austrian philosopher, sociologist, and political economist. He is still widely known for his work with the Vienna Circle, a group of logical positivists who sought to clarify philosophical problems through the tools of science and language. After working on housing reform and public education in Vienna during the 1920s, he founded the Museum of Society and Economy in Vienna. His main aim was to visually present statistical and social data using standardized pictograms rather than text or abstract graphs. He was not just interested in aesthetics; many of the people Otto Neurath hoped to reach had limited literacy or formal education. He intended the visual format to be more inclusive, immediate, and easily comparable across languages.
ISOTYPES (https://en.wikipedia.org/wiki/Isotype_(picture_language)) consists of standardized and abstracted pictorial symbols representing social-scientific data. They have specific guidelines for combining identical figures using serial repetition. For example, a single symbol (e.g., a figure representing 1,000 workers) would be repeated across the image rather than scaled proportionally, like pie charts or bar graphs. Otto Neurath worked closely with the German artist Gerd Arntz, who developed a coherent set of pictograms. Arntz’s symbols were deliberately geometric and straightforward, designed to be easily reproduced and recognizable even at small sizes. Marie Neurath, Otto’s later wife, was also crucial in shaping the ISOTYPE method and led much of its development after his death. ISOTYPES are often referred to as the Vienna Method of Pictorial Statistics.
Although the ISOTYPE method is often associated with its graphic output, Neurath saw it as part of a broader project called visual education. He was interested in how people learn from images and how knowledge could be structured visually - not just to inform but to shape public understanding. He explicitly rejected the idea of “illustrating” text; for him, the image should carry the content independently without requiring a written explanation. The underlying principles of the ISOTYPE system still resonate in today’s visualization guidelines. As a visual depiction of data visualization is central to communicating science, policy, and social change, it is helpful to revisit Otto Neurath’s work and his contribution to education.
IEEE VIS 2025 in Vienna, a century later, will be the perfect venue to remember Otto Neurath’s work.
A Breach of Ethics at VIS - What Happened and What's Next?
Last August, our community faced a serious ethics violation that led to the rejection of three conditionally accepted papers in the last round. The issue arose when a program committee (PC) member, serving as the primary reviewer for all three papers, failed to disclose a significant conflict of interest: one of the authors was their former Ph.D. advisor, a lifetime conflict in our field. We now want to update the community on the outcome of this matter and use it to sound a cautionary note for us all.
Over the past few months, both IEEE VIS leadership and IEEE itself have been addressing this issue. While we have little insight into the IEEE side, we wanted to let you all know that the IEEE VIS Steering Committee (VSC) has now agreed on disciplinary action for the offending PC member. This person—whose identity has been protected from all but the VIS OPCs, the TVCG EiC, and the VSC chairs—has been barred from publishing at or reviewing for VIS for two years, starting with VIS 2025. This includes not being allowed to present TVCG papers at the conference. The person has been notified of this penalty by the VSC chairs. After the end of two years, the slate will be wiped clean.
We hope that this decision will bring closure to both the authors of the three rejected papers, who were blameless in all of this, as well as the community as a whole.
Let this outcome also serve as a cautionary tale for all members of the VIS community to take conflicts of interest very seriously. Our rules, which are captured in the IEEE VGTC Reviewer Ethics Guidelines, are very clear, but we will nevertheless reproduce them here.
You have a conflict with a paper if:
- You are a co-author of the work.
- You have a strong affiliation with the same institution as one of the authors. This includes, but not limited to your current employment as a professor, adjunct professor, visiting professor, or similar position, in the role of a consulting or advisory arrangement, previous employment with the institution within the last 12 months, being considered for employment at the institution, any role as an officer, governing board membership, or relevant committee, or the current enrollment as a student.
- You have been directly involved in the work and will be receiving credit in some way. If you’re a member of the author’s thesis committee, and the paper is about his or her thesis work, then you were involved.
- You suspect that others might see a conflict of interest in your involvement. For example, even though Microsoft Research in Seattle and Beijing are in some ways more distant than Berkeley and MIT, there is likely to be a perception that they are “both Microsoft” and folks from one should not review papers from the other.
- You have collaborated with one of the authors in the past three years (more or less). Collaboration is usually defined as having written a paper, book or grant proposal together, although you should use your judgement.
- You were the MS/PhD advisor of one of the authors or the MS/PhD advisee of one of the authors. Funding agencies typically consider advisees to represent a lifetime conflict of interest.
- You are related to one of the co-authors. This includes, but not limited to spouse, child, sibling, or parent, as well as any affiliation or relationship of your spouse, of your minor child, of a relative living in your immediate household or of anyone who is legally your partner that you are aware of.
- Other relationships, such as close personal friendship, that you think might tend to affect your judgement or be seen as doing so by a reasonable person familiar with the relationship.
Note that these rules were last changed in May of 2009. In other words, they have remained unchanged for more than 15 years. Any claim of ignorance of the rules is moot and cannot be used as a defense.
We also want to communicate to the authors and the community as a whole that we, the IEEE VIS leadership, take these concerns with the utmost gravity and will pursue transgressions diligently. Please keep this in mind as the review requests begin to trickle in this spring.