IEEE VIS Junior Reviewer Guide

Overview

The IEEE VIS Junior Reviewer Program is a training program that introduces an optional junior reviewer role to the IEEE VIS full paper peer review process. The program allows each primary reviewer to optionally invite one junior reviewer per paper, complementing the existing review structure of primary, secondary, and external reviewers. This initiative aims to expand the community’s pool of skilled reviewers while providing valuable experience to early-career researchers.

Eligibility and Selection

The program is designed for Ph.D. students, postdocs, and researchers who are new to the visualization and visual analytics fields. While master’s juniors are generally not recommended as reviewers due to their typically shorter engagement with the research community, exceptions can be made for those continuing into Ph.D. programs. Postdoctoral researchers and established researchers transitioning to visualization from other fields are also welcome to participate as junior reviewers.

Note that we use the term “junior” broadly to encompass anyone new to reviewing for IEEE VIS, whether they’re new Ph.D. students or seasoned professionals from other fields. This inclusive definition reflects our belief that we’re all learners when engaging with new material.

Unlike external reviewers, junior reviewers may come from the same institution as other reviewers on the paper. In fact, primary reviewers are encouraged to recruit their own Ph.D. students or postdocs as junior reviewers. The key consideration is having a clear line of communication with the junior reviewer to facilitate effective mentoring throughout the review process.

Finally, the program is optional. Primaries can assign a junior reviewer, but don’t have to. Primaries should do so only if they are able and willing to spend the necessary time on the junior reviewer program.

Benefits for Junior Reviewers

Junior reviewers receive several valuable benefits from participating in the program. Most importantly, the program provides hands-on experience with the peer review process, helping junior reviewers develop critical evaluation skills and understand how to write effective reviews. Furthermore, they receive formal credit for their service as an IEEE VIS reviewer, which can be included in their academic service record. Finally, through mentorship from experienced reviewers, junior reviewers gain insight into the publication process that will benefit their own research and future contributions to the visualization community.

Timeline and Process

The junior reviewer program follows the main conference review timeline, with specific steps at each stage:

1. Reviewer Recruitment (April)
   • Primary reviewers may invite one junior reviewer per assigned paper through PCS.
   • Direct communication with the junior reviewer is encouraged before sending the formal PCS invitation.
2. Discussion Period (May)
   • Junior reviewers are encouraged to calibrate their reviews and ratings with other reviewers. They should also participate in the paper discussion on PCS.
   • Primary reviewers write their own review and provide feedback on junior reviews.
   • Primaries verify that junior reviewers have marked their expertise level as “junior”.
   • Primaries assess review quality and request revisions if necessary.
   • Primaries may contact APCs/OPCs if inappropriate reviews need to be removed.
3. Decision Phase (May)
   • Primary reviewers may incorporate junior reviews into their summary as they see fit and if the secondary reviewer agrees.
   • Junior reviews can be considered partially or disregarded entirely (even removed if necessary).

In practical terms, the junior expertise rating carries a weight of 0 in PCS. This means that their rating will have no impact on the weighted average in the system.

Mentoring Responsibilities

Primary reviewers who invite junior reviewers commit to mentoring them throughout the review process. This mentorship involves guiding them through the review criteria, ensuring they understand the expectations for constructive feedback, and providing feedback on their completed reviews.

Important: Primary reviewers should do their best not to influence the junior reviewer’s decision or try to align their opinion with their own.

Junior reviewers will be directed to the standard IEEE VIS reviewing guidelines and receive clear instructions about their role in the process. Their reviews should maintain the same professional standards as other reviews, though they are considered advisory in nature.

Writing a Junior Review

As a junior reviewer for IEEE VIS, you have the opportunity to contribute to the peer review process while learning from experienced researchers. Your review is considered advisory, but it can still provide valuable perspective to both the authors and other reviewers.

Review Form

When you submit your review, you must mark your expertise level as “junior” in the review form. This is crucial as it signals to authors that your review is advisory in nature. Your primary reviewer will verify this designation.

Time Management

Reviewing requires significant time and attention. For your VIS review, plan to dedicate at least a full day to the process. You should plan to:

• Read the paper multiple times.
• Take detailed notes.
• Draft and iteratively revise your review.
• Consider asking your primary reviewer for feedback before the review deadline.

Writing Your Review

Focus on providing a thorough, constructive review. While several reviewing guides are available (see resources below), here are key points to remember:

• Neutrality: For non-anonymized submissions (IEEE VIS supports both single-blind and double-blind reviewing), maintain objectivity regardless of author identities. Don’t let author reputation influence your assessment positively or negatively.
• Objectivity: Take a balanced approach:
  • Avoid letting a single aspect determine your entire assessment;
  • Consider both strengths and weaknesses;
  • Keep an open mind throughout the process; and
  • Look at the work holistically.
• Calibration: New reviewers often face calibration challenges:
  • Some are overly critical, focusing too much on minor issues.
  • Others may be too lenient, missing significant flaws.
  • Both extremes are natural; awareness helps you find balance

Working with Your Primary Reviewer

Your primary reviewer serves as your mentor in this process. While you should form your own opinions about the paper, don’t hesitate to seek guidance:

• You can share your draft review for feedback before submission.
• Ask questions about the review process.
• Seek clarification about aspects of the paper you find unclear.
• Approach this as a learning opportunity.

During the Discussion Phase

Once other reviews become visible in PCS during the discussion phase:

• Read them to see what experienced reviewers emphasize.
• Notice what they consider important (or not).
• You don’t need to modify your review to match others.
• Don’t feel pressured to align your rating with the average.

Remember, this isn’t an exam where you need to catch every point others make. Instead, think of your review as one lens through which to help the authors improve their paper.

Participating in Discussions

Engage in the discussion forum, even if briefly:

• Acknowledge that you’ve read other reviews.
• Share your perspective.
• Respond to questions about your review.
• Indicate whether you agree with the primary’s direction.
• Ask questions if you’re uncertain about something

Graduating from the Program

When are you sufficiently experienced to work independently as a full reviewer for a paper? In other words, when do you graduate from the junior reviewer program? This is a discussion that is worth having with your advisor (the primary reviewer), but the short answer is that it is up to you. The intention with the program is to train new reviewers who will sooner or later join the reviewer pool for the VIS conference. In other words, the only one who can truly say when you are ready is you. (Of course, you will still have to be invited to review by someone on the program committee who is not your advisor; only junior reviewers can come from the same institution.)

Review Resources and Guides

• The IEEE VIS Reviewing Guide
• How to Review HCI/Visualization Papers (Niklas Elmqvist)
• Mistakes Reviewers Make (Niklas Elmqvist)
• Tips for being a Good Visualization Paper Reviewer (John Stasko)

Questions & Concerns

For questions about the Junior Reviewer Program, please contact the IEEE VIS Overall Papers Chairs at opc@ieeevis.org.