How to Review a Conference Paper?

You have accepted a review invitation, the deadline is two weeks away, the form has eight empty fields, and nobody ever showed you how to do this. That is a common starting point for a first-time reviewer. Peer review is one of the most consequential jobs in research, and the review form rarely explains how to do it well.

Quick answer: To review a conference paper, read it twice, once for the big picture and once against the venue’s criteria. Then write a short summary, specific strengths and weaknesses, and a score that matches your text. Judge originality, technical soundness, clarity, relevance to the conference and significance, and keep the paper confidential.

Last verified against official sources: October 2026.

This guide covers how to structure your reading, what the criteria mean in practice, how scores and confidence ratings work, how to handle conflicts of interest and double-blind review, how to write feedback authors can use, and what to do when a rebuttal arrives. The principles apply to computer science venues such as NeurIPS, ICML and ACL as well as to engineering, humanities and social science conferences. Review forms on HotCRP, EasyChair, OpenReview or Microsoft CMT use different labels, but they ask for the same things.

What Does a Conference Paper Reviewer Actually Do?

A conference reviewer helps the program committee decide whether a paper belongs at the event, and gives the authors specific feedback they can use whatever the outcome. Being asked means the committee thinks you know the area well enough to judge it. It is also a real intellectual responsibility, and it is easy to underestimate how much skill it takes.

The Formal Side of the Role

Each submission goes to a few reviewers, commonly two or three, and the conference sets the number. You read the paper, assess it against the conference’s review criteria, assign a score on the venue’s rating scale, and write a structured report. Many computer science conferences, including NeurIPS, ICML and ACL, also have a rebuttal phase in which authors reply to your concerns before a final decision.

If the venue uses a meta-reviewer or area chair, that person combines your review with the others. Your written comments feed straight into that summary, so a vague score with no explanation puts the area chair in a difficult position.

The Less Formal, But Equally Important Side

You are also a quality filter. A paper that slips through with flawed methodology or unacknowledged prior work does not only affect one venue. It becomes part of the literature that others cite and build on.

You are expected to flag any conflict of interest before you start. If you know the authors, have collaborated with them recently, or have a financial stake in their results, you should not review the paper. The details are in the ethics section below.

The Golden Rule of Reviewing

Review the paper the way you would want yours reviewed. That means constructive feedback on weaknesses, specific praise where the work is strong, and a score that honestly reflects your view rather than a reflex based on the first two pages.

Reviewers who write dismissive one-liners, give reject scores without explanation, or ignore whole sections of a paper are failing the process. Authors may disagree with your conclusion, but they should never finish your review feeling they wasted their time.

Key Criteria for Evaluating a Conference Paper

Many venues ask you to judge five things: originality, technical soundness, clarity, relevance to scope and significance. Without a framework you end up with vague impressions, and that helps neither the authors nor the committee. The weight each criterion gets differs by field, so read the call for papers and the reviewer instructions first.

Originality and Contribution

The first question after reading is what the work actually adds. Originality does not need a new field or a revolutionary method. Applying a known technique to a genuinely underexplored problem can be a solid contribution. What you check is whether the paper does something that was not already in an accessible earlier publication, and whether that something matters.

Ask whether the contribution is stated clearly or only implied. Some papers bury the novelty inside the methodology when it belongs in the introduction. If you struggled to find what is new, say so in your feedback.

Significance is a separate question. A small incremental gain on a crowded benchmark may be technically original yet offer little value. Strong contributions open new directions, solve a recognised problem cleanly, or change how researchers think about something.

Technical Soundness and Methodology

Once a paper claims something new, ask whether the claim holds up. The experiments, proofs or analyses must support the conclusions. In computer science and other STEM fields, check that baselines are fair, datasets are appropriate and metrics are well chosen. In the social sciences, check whether the design controls for obvious confounds and whether the sample size and statistics fit.

Watch for cherry-picked results shown without variance, missing ablations, and comparisons against outdated baselines that flatter the proposed method. These are not necessarily bad faith, but they weaken the claim and authors deserve to know.

You do not need to re-run experiments. You are assessing internal consistency: do the methods described produce the results shown, and are those results interpreted correctly?

Clarity and Presentation

A sound paper that nobody can follow has limited value. This criterion covers writing quality, logical order, and whether figures and tables help. You are not copy-editing, but if the central argument is hidden by poor structure or undefined notation, that is a substantive problem.

Check whether the related work motivates the paper’s direction or reads like a literature dump, and whether the conclusions match the results. If the abstract promises something the paper does not deliver, that gap matters.

Where authors may not be native English speakers, separate awkward phrasing, which editing can fix, from unclear reasoning, which it cannot. Recommend a language check if needed, but do not penalise grammar heavily when the ideas are sound.

Relevance to the Conference Scope

Every conference has a defined scope, and part of your job is to say whether the paper fits. At ICML a paper should sit within machine learning research, not merely use a machine learning tool on the side. At ACL a submission should connect to computational linguistics or natural language processing. Many venues list their topics explicitly, so check the call for papers.

When a paper is interesting but misaligned, note the mismatch in your review instead of marking it down on technical grounds. If you cannot tell whether it fits, ask the meta-reviewer or area chair rather than deciding alone.

Ethical Integrity of the Research

Research ethics is a criterion in its own right. Check whether the paper describes appropriate oversight: ethics board approval for human-subjects work, data licensing compliance, informed consent where it applies. In machine learning work, consider whether the authors acknowledge risks such as misuse or biased outputs.

If a paper seems to rely on generative AI without disclosing it, check the venue’s policy before raising it, because what counts as a violation varies. The principle is that the methods section must not misrepresent how the results were produced.

The 5 Key Elements of Peer Review (and the 5 C’s)

A good review has five parts, and the “5 C’s” are a handy way to check yourself before you submit. Review systems such as HotCRP, EasyChair and OpenReview usually ask for the same elements even when the form labels differ. The NeurIPS 2025 form, for example, asks for a summary in your own words, strengths and weaknesses, and about three to five actionable questions.

The 5 Core Elements of a Conference Paper Review

Summary of the work. Write two to four sentences in your own words saying what the paper does. This is a check on yourself: if you cannot summarise the contribution, you have not understood it well enough to score it. Authors read this closely during the rebuttal to see whether you engaged with their paper or a misreading of it.

Strengths. Name what the paper does well, concretely. Not “the writing is clear” as a throwaway compliment, but “the ablation study covers the obvious alternative explanations” or “the theoretical framing connects cleanly to prior work”. Even a rejected paper usually has something worth naming.

Weaknesses. This section carries the most weight for authors. Say what is missing, wrong or insufficiently justified. “The experiments are unconvincing” gives them nothing. Say which experiment and why: missing baselines, a small sample, an unaddressed confound.

Questions and clarification requests. Many forms separate these from weaknesses, and for good reason. Some concerns dissolve if the authors can explain a choice or point to a result you missed. Keeping genuine questions apart from firm criticisms makes the rebuttal cleaner and shows intellectual honesty.

Recommendation and score. The score should follow naturally from everything above. A score without supporting reasoning is almost useless to the area chair.

What the 5 C’s Actually Mean

The 5 C’s are a teaching framework used in reviewer development programs, and different sources label them slightly differently. They are not a formal checklist from any single conference body, but they map well onto good reviewing.

Competence means you are qualified to judge the work. If a paper is outside your expertise, flag it for reassignment or be explicit in your review about the limits of your assessment. A confident-sounding review of a paper you do not fully understand is one of the more damaging things a reviewer can submit.

Completeness means covering all the major dimensions: originality, methodology, soundness, clarity and relevance. Skipping one because it seems fine is a mistake, since “fine” is worth a sentence.

Consistency means applying the same standards across papers. If you penalise one author for missing ablations, apply the same scrutiny to every paper in your batch. Inconsistent standards make scores hard to compare.

Constructiveness separates a useful review from a discouraging one. It names the problem and gives a direction, so even a rejected paper leaves better than it arrived.

Confidentiality covers the ethical side. You do not share the paper, discuss it publicly, or use its unpublished ideas in your own work. Any conflict of interest must be disclosed before you engage with the paper, not after you have read it.

How to Structure Your Review Step by Step

Read the paper twice, then write. Rushing through a single pass tends to produce a shallow, inconsistent review, and reading for review is different from reading for your own research because you read with a purpose.

First Read: Get the Big Picture

Read the abstract, introduction and conclusion first, then the rest at a normal pace without detailed notes. The goal is to understand what the authors claim, what problem they solve, and whether the work feels coherent. Ask: is there a clear research question, does the conclusion follow from the middle, can you identify the contribution? Also check that the paper follows the venue’s format and page limit. If it does not, tell the chairs instead of judging the paper on that point alone.

Red flags show up best on a first read, such as a claimed scope that is bigger than what is demonstrated, or a method that seems disconnected from the results. Afterwards, jot a one-line impression like “technically solid but contribution unclear” or “interesting problem, weak evaluation”. It is a sanity check later, when hours in the details risk hiding the overall picture.

Second Read: Assess Against Criteria

The second read is the analytical work. Go section by section with the criteria in mind, and for each issue write down exactly what it is. Do not note “results are unclear”. Note which table is confusing and why. Check the methodology, the experiments, whether the literature review is adequate, and what the contribution is worth.

In computer science venues, pay particular attention to baselines, reproducibility and whether the claims are statistically supported. In the humanities and social sciences, argumentation, theoretical framing and engagement with existing scholarship matter more. Also check originality against the venue’s rules: has the work been published or posted in a form the double-blind policy would flag?

Writing the Review: Summary, Strengths, Weaknesses, Recommendation

Use the five elements above as your skeleton, and make weaknesses actionable. “The evaluation is limited to a single dataset” is useful. Adding “testing on a comparable public benchmark would strengthen the generalisation claim” is better. Frame weaknesses as solvable problems rather than verdicts.

A short sample of how the main sections can read:

Summary: The paper proposes a lightweight method for X and evaluates it on three benchmarks.
Strengths: The ablation in Section 5 isolates each component, and the code description is complete enough to replicate.
Weaknesses: (1) Major: the comparison omits the 2024 baseline Y, which uses the same data. (2) Minor: Figure 3 does not label the axes.
Questions: How was hyperparameter Z chosen?
Recommendation: Weak accept. My score would rise if the missing baseline is added.

Do not list every small issue you noticed. Prioritise the problems that affect validity or significance, and mark which are major and which are minor. A long list of equal-weight complaints frustrates authors and gives the committee little guidance.

Finally, check that the text and the score tell the same story, and declare any conflict of interest before you begin. If you realise mid-review that you know the authors despite the blind setup, tell the program committee at once.

Entering Your Review in the System

In Microsoft CMT, assigned papers appear in the Reviewer Console only after the chair starts the review process, so an empty list means reviewing has not opened yet. Click Enter Review and answer the questions the chair set up; required fields are marked with an asterisk. CMT autosaves every few minutes, and Save Draft stores your work in between. After you submit, you receive a confirmation email, and only chairs can delete a review, so check the text before you click Submit.

If you prefer to write offline, CMT’s Import Reviews option lets you download an XML file, fill in your answers in a plain text editor (not Word, which breaks the format), upload it again and check each review with View.

Review Scores and Rating Systems Explained

Your score signals overall direction, but the written justification is what the area chair and authors rely on. Scales differ by venue, so always read the reviewer guidelines before you pick a number.

How Conference Rating Scales Work

Platforms like HotCRP, EasyChair and OpenReview let program committees configure their own scale, so labels depend on the venue. Typical formats are:

A “3” can mean weak accept at one conference and borderline reject at another, so do not carry habits from one venue to the next.

The NeurIPS 2025 reviewer guidelines show how detailed a form can be. The overall score runs from 1 (Strong Reject) to 6 (Strong Accept), with 4 (Borderline accept) and 3 (Borderline reject) marked for sparing use. Quality, clarity, significance and originality each get a separate score from 1 (poor) to 4 (excellent), and confidence runs from 1 (an educated guess) to 5 (absolutely certain).

What the Score Actually Represents

Your overall score is a holistic judgment across originality, soundness, methodology, clarity and relevance. It is a weighted synthesis, not a simple average. A genuinely original idea does not cancel out a flawed experimental design, so the score must reflect the severity of the problems. Many conferences also ask for sub-scores such as presentation quality or novelty. Fill them in honestly.

The Confidence Score

Your confidence score tells the meta-reviewer how much weight your opinion deserves. If you are a strong match for the topic, say so. If the paper drifts into a methodology from another field, mark lower confidence. Claiming high confidence with a thin review creates problems for the area chair, and inflating confidence does not make your review stronger. It only makes errors harder to catch.

Calibrating Your Score Honestly

Calibration is hard for first-time reviewers. There are two failure modes.

Too generous. Reviewers sometimes inflate scores because they feel uncomfortable recommending rejection, or because the topic interested them even though the execution was weak. Generosity to a flawed paper does not help the field.

Too harsh. Some apply an impossibly high bar and treat any imperfection as a reject, especially at competitive venues with low acceptance rates. The question is whether the paper as submitted makes a sufficient contribution for this venue, not whether it is the best possible version of itself.

A quick test: if your text lists only minor issues but your score is a reject, or your text lists fundamental methodological flaws but you chose borderline accept, the two disagree. Fix whichever one is wrong. If the paper is strong but needs a few clarifications, that reads as a weak accept or a revision, not a rejection.

Ethical Considerations: Conflict of Interest and Double-Blind Review

Peer review depends on trust, and it breaks down when a reviewer hides a relationship with an author or when anonymity is violated. The Committee on Publication Ethics (COPE) guidelines for peer reviewers set the same basics that most conferences follow: declare conflicts, keep the manuscript confidential, stay within your expertise, and be objective and courteous rather than hostile.

Recognizing and Declaring a Conflict of Interest

A conflict of interest exists whenever your relationship with the authors, their institution or their work could influence your judgment, even unconsciously. HotCRP, EasyChair and OpenReview all include formal ways to flag conflicts before assignments are made.

Declare a conflict if:

• you co-authored with one of the authors within the last few years
• you are at the same institution as an author, or were recently
• you have a close personal or professional relationship with an author
• you have a financial interest in the work being accepted or rejected
• you have had a recent adversarial relationship with an author, such as a public disagreement
• the author is at your former institution, controls a grant you have applied to, or is a direct competitor in a niche subfield

When in doubt, declare it. Area chairs would far rather reassign a paper than have a compromised review go undetected. What you should never do is accept the assignment and let the conflict quietly color your review. That breaches research ethics and can have professional consequences.

Understanding Double-Blind Review

In double-blind review, authors do not know who their reviewers are and reviewers do not know the authors. The purpose is to reduce bias from gender, institutional prestige and reputation. Your obligation is to protect that anonymity in both directions.

• Do not try to identify the authors. Searching arXiv for a preprint, googling a distinctive phrase, or scanning references for self-citations all undermine the process, and the NeurIPS 2025 reviewer guidelines name searching arXiv as an example of what not to do. If you recognise the authors anyway, tell the area chair and continue as objectively as you can.
• Do not reveal your identity. Some reviewers are tempted to hint at who they are, particularly with positive remarks. Let the score and feedback stand on their own.
• Be careful with preprints. If a public preprint exists, you may stumble on it. Many committees ask you to base your review only on the submitted version and not penalise authors for having a preprint, but check the venue’s policy.
• Do not hand the paper to someone else. Sharing a submission with a student or colleague breaks confidentiality unless the venue has a formal route for it. NeurIPS 2025, for example, says not to use sub-reviewers.
• Do not contact the authors. Direct contact breaks the anonymity model, so questions go through the committee (see the FAQ).

From our experience at MS International: We use double-blind review for our conference submissions, so the identities of authors and reviewers stay confidential during the review. Each paper is reviewed for originality, methodology, clarity, structure, APA formatting and fit with the chosen conference. Authors receive an acceptance, revision request or rejection decision by email, and most decisions come back in about 10 working days. Accepted papers may be considered for conference proceedings where applicable. We do not claim any particular indexing for them.

Field-Specific Differences in Reviewing Criteria

The same five criteria carry different weight in different fields, and applying the wrong lens makes you either dismiss solid work or miss real weaknesses. Use the table as a starting point and then follow the venue’s own instructions.

Computer Science and STEM Fields

The emphasis falls on empirical validation and reproducibility. A paper claiming a new model or algorithm needs rigorous, clearly described results honestly compared with baselines, and the setup should be complete enough for someone to replicate. Incremental improvements are not automatically rejected, but the contribution must be scoped clearly: “we improve accuracy by X% on benchmark Y under conditions Z” is a real claim, and a vague assertion of novelty is not.

The bar for clarity is practical. Can a reader in the subfield follow the method without guessing? Consistent notation, correct pseudocode and proper citation of related work all count. Some venues welcome well-executed work that disproves a hypothesis, so check the call before marking a negative result down.

Humanities and Qualitative Social Sciences

You will not find benchmark tables, and that is not a flaw. Ask whether the interpretive approach is appropriate and consistently applied, whether the argument is coherent, and whether sources are used responsibly. Significance is often judged by how far the paper advances a scholarly conversation, whether by challenging a consensus, introducing a neglected primary source, or connecting fields that were not in dialogue.

If you are a STEM-trained reviewer assigned a humanities paper, say so. Applying positivist standards to an interpretive argument misleads the committee, so scope your comments to your background.

Quantitative Social Sciences

Papers in economics, psychology and political science sit between the two poles. Reviewers often focus on threats to internal and external validity that authors may have underplayed. Contribution claims should be grounded in what the field already knows, and a paper that repeats existing findings with no added insight is a harder sell however clean the execution.

Cross-Field Venues and Interdisciplinary Work

Interdisciplinary work needs the most flexibility, because authors may answer to two traditions at once. Ask whether the goals are coherent and the methods suit them, not whether the paper looks like a pure CS paper or a pure sociology paper. If the assignment is outside your main area, tell the area chair up front and scope your comments honestly so the committee can weight them.

How to Handle the Rebuttal and Author Response

A rebuttal can and should change your score, but only when it addresses a substantive concern you raised. Many review systems, including HotCRP, EasyChair and OpenReview, can run a phase where authors respond before final decisions, and newer reviewers may wonder how much weight to give it.

What the Rebuttal Phase Is Actually For

Authors use it to correct factual misunderstandings, clarify ambiguous methodology, or point to evidence you missed. It is not a second submission. A reply like “we will add more experiments in the final version” is not a rebuttal. It quietly accepts that the concern was valid. A strong rebuttal either shows you misread something or gives a specific counterargument to a technical point. Both are good reasons to revise.

Check the venue’s rebuttal rules too, because they shape what you can expect. At NeurIPS 2025, for example, the FAQ says rebuttals should serve only to clarify reviewers’ and area chairs’ questions, and that no revision of the paper is allowed until the camera-ready stage. In that setting the question is whether the response answers your concern about the submitted paper, not whether the authors have rewritten it.

Before the window opens, re-read your own review with fresh eyes. Reviewers sometimes write imprecise criticisms or merge two separate issues, and you need to be confident you said what you meant. If a comment was unclear or overstated, you can say so in your update.

How to Respond When a Rebuttal Changes Your View

Acknowledge what shifted. Be explicit about what changed. For example:

The authors’ clarification about the dataset preprocessing resolves my concern in point 3, so I have raised my score from 3 to 4. My concern about the missing baseline in point 1 remains.

A reply like “noted” does not help the meta-reviewer understand what happened.

Keep your other concerns intact. Answering one issue does not erase unresolved ones elsewhere. Reviewers sometimes overcorrect after a persuasive rebuttal and lift scores on dimensions that were never addressed. Evaluate each concern on its own.

Do not capitulate under pressure. An author restating a claim more forcefully, or expressing frustration, is not a substantive response. Base your view on what the paper and the rebuttal demonstrate.

When You Keep Your Score, or Disagree with Co-Reviewers

If the rebuttal does not change your assessment, keeping the score is entirely legitimate. Note briefly why the response did not shift your view, which makes the review defensible. Because authors in a double-blind process cannot lobby reviewers directly, the rebuttal is their one formal channel.

At many conferences a reviewer discussion phase follows. If co-reviewers move significantly and you do not share their view, say so in the thread. You do not need consensus. The area chair needs to see the spread of opinion, not just the average.

What Not to Do

• Do not ignore the rebuttal. At a minimum, check whether the factual errors you identified were real or a misreading on your part.
• Do not change your score without a written reason. Unexplained shifts look arbitrary and undermine trust in what scores mean. This applies in both directions: never updating is as much a problem as updating only because an author sounded confident.
• Do not confuse a polished rebuttal with a strong paper. A well-written response that does not resolve a technical concern leaves the concern unresolved. Some authors now use generative AI to draft rebuttals, so read for substance, not style.

Can You Use ChatGPT to Review a Conference Paper?

Whether you may use ChatGPT or another AI tool depends on the venue, and policies change from year to year, so read the current reviewer instructions first. Do not use it to write your review or to judge the paper. For example, NeurIPS 2025 states that reviewers must keep the review process confidential and may not share submissions with anyone or with any LLM.

What the Policies Actually Say

ICML 2026 shows how far policies can differ. Each paper is handled under one of two policies. Under the conservative policy, using LLMs at any stage of reviewing is strictly prohibited. Under the permissive policy, LLMs may help you understand the paper and related work and polish your review, and the paper may be fed only to privacy-compliant LLMs. Asking an LLM to assess quality or significance, identify strengths and weaknesses, suggest key points or an outline, or write the review is not allowed under either policy. ICML also states that reviewers are fully responsible for the entire content of their reviews, and that hallucinated content is subject to disciplinary action.

The core problem is accountability. When you agree to review, you vouch for the assessment with your own expertise. If you paste the PDF into a language model and copy out its answer, that accountability is hollow, and the authors deserve a judgment from a qualified human.

Where AI Tools Can Legitimately Help

Where your venue allows it, general help that does not involve the paper’s text is the safest use.

• Clarifying unfamiliar terminology. Asking a tool to explain a technique at a conceptual level is no different from looking it up in a textbook.
• Understanding prior work. A general overview of a citation you do not recognise can orient you, but verify anything factual before relying on it.
• Checking your own writing. Once you have written the review in your own words, fixing grammar is reasonable, provided the venue allows it and you do not paste in the paper’s content.

The Risks You Should Take Seriously

Generative AI tools hallucinate. They produce confident text that can be factually wrong, so a critique of a paper’s methodology may rest on an inaccurate reading or invented context about your field. You could submit claims that are simply incorrect, damaging to the author and embarrassing for you.

There is also a confidentiality risk. Papers under review are confidential documents. Pasting content from a submission into a commercial AI tool potentially exposes it to a third party without the authors’ consent, which is a research ethics problem and not a technicality.

What This Means for Reviewer Development

Leaning on AI means you do not build your own ability. Judging originality, evaluating soundness and writing constructive feedback are skills built through practice. If you feel unsure, the better path is to tell the area chair honestly, or ask a senior colleague to mentor you. Asking the area chair or a mentor for guidance is a better route than submitting a review that a machine largely wrote.

Common Mistakes Reviewers Make

These mistakes come from time pressure, habit and a fuzzy idea of what a review is for, not from bad intentions. Each one is easier to avoid once you can name it.

Vague Criticism Without Actionable Detail

“The writing is unclear” or “the experiments are insufficient” tell authors nothing. A comment like “Section 3.2 does not define how hyperparameter X was selected, which makes the results hard to reproduce” gives them something concrete.

Dismissing a Paper for Not Citing Your Own Work

If a relevant citation is genuinely missing, note it, but frame it as a gap in the related work and not a personal grievance. Requesting self-citations as a condition of acceptance crosses an ethical line.

Submitting Reviews That Are Too Short

A two-paragraph review saying “interesting but needs more experiments” does not help the committee decide. Review guidelines on HotCRP, EasyChair and OpenReview usually indicate the expected depth. Reading the feedback you received on your own papers, and asking what made it useful, is a good way to learn what a helpful review looks like.

How Reviewing Makes You a Better Researcher

Reviewing is one of the fastest ways to sharpen your own research instincts, so treat it as training and not only as an obligation. It is easy to see it only as a favor to a program committee chair, and that is a missed opportunity.

You Learn to See Your Own Work Differently

Reading submissions forces you to ask about clarity, soundness and significance, the same questions your own reviewers will ask. When you struggle with a methods section, you write yours more carefully. When a paper’s claims outrun its evidence, you become more vigilant about your own. Published papers have already survived review and lost their rough edges, while reviewing shows you the messy middle where much of the learning happens.

You Build a Sharper Eye for Research Quality

Recognising weak originality. After reading many submissions that repackage existing ideas, you develop a cleaner sense of genuine novelty, which improves how you position your own work.

Spotting methodological gaps. Checking whether a study’s method supports its conclusions trains you to anticipate what reviewers will look for in yours.

Judging relevance to scope. Seeing dozens of submissions that miss the mark makes you better at targeting your own, whether at a specialised workshop or a broad conference.

You Understand the Review Process From the Inside

Knowing how area chairs weigh scores, how a rebuttal is read in context, and what a borderline rating really means removes much of the anxiety around author responses. Understanding what the committee is trying to do also helps when you write your own rebuttals and camera-ready revisions.

Reviewing Builds Your Academic Network and Research Ethics Instincts

Program committees are made up of active researchers in your field. Careful reviewing, with conflicts handled correctly, is one way to take part in that community. Double-blind review also asks for discipline: judging the work without assumptions about its authors. Practising that makes you a more rigorous thinker generally.

Reading published reviews at venues with open review, such as those on OpenReview, gives a concrete sense of thorough, fair feedback, and comparing your instincts with experienced reviewers is one of the best ways to improve. If your institution offers formal guidance, take it.

FAQ

Reviewing outside your exact sub-topic, scoring a paper as borderline, revisiting your score after the author response and declining an invitation politely are the judgment calls a reviewer faces, junior PhD students included.

How long should a conference paper review be?

There is no universal length, so follow the venue’s review form. Write enough to justify your score and cover each criterion. A review that covers the summary, strengths, weaknesses and questions in a few focused paragraphs is usually enough, and a two-sentence “interesting work” helps nobody.

What if I’m not an expert in the paper’s specific sub-topic?

Review what you can assess confidently and flag the rest. A line like “I’m not a specialist in X, so I cannot fully judge the theoretical novelty” is honest and legitimate. Lower your confidence score and tell the area chair, who will prefer that to false confidence.

Can I recommend rejection if the paper is poorly written but the ideas are good?

Yes, because clarity is part of the criteria. If the core contribution is strong, say so explicitly and recommend major revision instead of flat rejection where the venue offers that option.

What does a review score of “borderline” actually mean?

It means the merits and problems roughly balance, and the exact meaning depends on the venue’s scale. NeurIPS 2025, for example, marks its two borderline scores for sparing use. Explain what would need to change to move the paper above the line.

Do I have to read every reference in the paper?

No. Spot-check the citations central to the paper’s claims, and verify that a cited paper really says what the authors claim.

What happens after I submit my review?

Your review joins the others on the platform. The area chair or meta-reviewer reads them all, may start a discussion, and recommends a decision to the program committee. You may be asked to respond to a rebuttal.

Is it okay to contact the authors directly if I have questions?

No, not in double-blind review. Direct contact breaks anonymity. Send clarifying questions through the program committee, or put them in your written review so authors can answer in the rebuttal.

Should I change my score after reading the author response?

Only if the rebuttal addresses your concern with new information or a real clarification. Forceful pushback without resolving the issue is not a reason, and any change should come with a written explanation.

What’s the golden rule of reviewing?

Write the review you would want to receive on your own paper. Give specific, actionable feedback, with no vague criticism and no unearned praise. If you point out a flaw, say where it is and why it matters.

Does reviewing a paper mean I can’t cite or build on it until it’s published?

Correct. Material you see through peer review is confidential. Using ideas from an unpublished submission in your own work without the authors’ knowledge is a serious research ethics violation.

How do I decline a review invitation without causing problems?

Reply promptly and use the system’s decline option, with a short reason such as a conflict of interest, expertise or time. In CMT, use the decline link in the invitation email or change your response on the Manage Reviewer Invite page. A quick answer lets the chairs reassign the paper sooner.

Can a junior PhD student review conference papers?

Sometimes, usually through a mentored route. IEEE VIS 2026, for example, lets each primary reviewer invite one junior reviewer per paper, aimed at PhD students and postdocs, and junior reviews are advisory. Check the venue’s call and ask your advisor.

Final Thoughts

A thorough, fair review shapes which research gets seen, debated and built upon, so the effort matters. You will not always be certain, because some methods are hard to judge and some contributions sit at the edge of scope. Write what you actually think, be specific, and stay open to the rebuttal.