Reference integrity and experiment evidence that survive review

dateModified: 2026-09-27

Part 3 · Experiment design and reporting standard

3.1 Baselines: pick them to answer a question, not to look busy

Each baseline exists to rule out an alternative explanation:

|---|---|---|

Rules:

• Reuse official code where it exists; re-implementations are a predictable attack surface. If you must re-implement, verify your version reproduces the published number within noise before you compare, and report that check.
• Match data and tuning budgets. Comparing your heavily tuned system against a baseline's default configuration is a known reason for rejection and it is visible in the table.
• Prefer fewer strong baselines to many weak ones.

3.2 Ablation design

• Change one factor per row. Two simultaneous changes make attribution impossible.
• Report cost alongside quality (parameters, latency, memory) — it is how you prove the trade-off is worth it.
• Put the ablations in a table, not in prose; label columns by component presence (`✓`/`✗`).
• Do not omit ablations whose result is unfavourable. If a component adds little, say so and keep the simpler variant — this is a contribution, not a weakness.

3.3 Statistical reporting (the "is it noise?" defence)

Minimum standard for a top venue:

• Report mean ± standard deviation over ≥3 independent seeds; ideally ≥5 for noisy RL/tuning-heavy settings. State that seeds cover training (not just inference).
• Report the sample size for every number (n = how many examples/queries/runs).
• Use confidence intervals or significance tests for headline comparisons. For paired setups (same test set, two systems) use a paired test (paired bootstrap or McNemar's test for accuracy); the unpaired two-sample t-test is the wrong default for comparing systems on identical inputs.
• Prefer effect size and interval over a bare p-value. "Δ = +1.4 BLEU, 95% CI [0.6, 2.2], n=2,000 sentences" is stronger than "p < 0.05".
• Do not run statistical tests on your own reported improvement across metrics and then claim significance globally. Multiple comparisons inflate false positives; if you test many hypotheses, say so.

3.4 Tables and figures

• Bold the best result; underline the runner-up; state within the caption whether the difference is significant.
• One decimal place more than the metric's noise level — usually 1–2 decimals. Writing three decimals for accuracy in percent invites questions about noise.
• Table captions must be self-contained: what task, what dataset, what split, what the numbers mean, and what is highlighted. Reviewers read captions and figures before anything else.
• Never let a chart imply what the data does not support: label axes with units, start bar-chart axes at zero (or say clearly that you did not), show error bars when the point is a comparison, and never truncate a y-axis to manufacture a visible difference.
• Every figure must be readable in greyscale and legible at 2x zoom in the PDF. Use colourblind-safe palettes; distinguish data series by marker and line style, not only colour.
• Put the most important figure early (Introduction or Method), not in the appendix.

3.5 The three highest-value "extras"

• Error analysis. Where does your method fail? Quantify failure categories and show two concrete cases. Nothing earns reviewer trust faster.
• Efficiency table. Parameters, FLOPs, latency, peak memory, training cost. Cheap to produce, and it answers the most frequent rebuttal question before it is asked.
• Negative / boundary result. Report the setting where the method does not help. It converts "the authors oversell" into "the authors are reliable".

---

Part 4 · Academic English for non-native writers

Academic English is a register, not a talent: restricted vocabulary, predictable sentence frames, and heavy use of nominalisation. You can learn the register without speaking like a native, and reviewers reward clarity far more than flourish.

4.1 Four tense zones (get these right and 30% of your grammar errors vanish)

|---|---|---|

Tense discipline is a credibility signal: present tense for method and past tense for results is what a native-reading reviewer expects, and drifting between them reads as careless.

4.2 Sentence frames that always work

Introducing a problem

• "Despite substantial progress on X, existing methods still struggle with Y when Z holds."
• "X has become the default approach for Y; however, it assumes Z, which fails when ..."
• "While A achieves strong results on B, it requires C, which is impractical for D."

Positioning against prior work

• "Unlike X, which relies on A, our approach derives B directly from C, removing the need for D."
• "Concurrent work explores a related idea; we differ in that ..."
• "Our setting differs from the standard one in three respects: ..."

Describing your method

• "We introduce X, a module that maps A to B while preserving C."
• "Our method consists of three stages: first ..., then ..., and finally ..."
• "Formally, given X, we define Y as ..., where λ controls ..."

Reporting results

• "As shown in Table 2, our method improves X by 3.2 points over under identical settings (n = 2,000, 3 seeds)."
• "The gain concentrates on examples where ..., which is consistent with our hypothesis in §3."
• "Removing module X costs 2.1 points, indicating that most of the improvement comes from Y."

Honest hedging (mandatory)

• "These results suggest ..." / "This is consistent with ..." / "We hypothesise that ..." / "we do not claim ..."
• Never "clearly", "obviously", "proves beyond doubt", or any construction implying proof where you have evidence.

4.3 Transition words in review-proof writing

|---|---|

A single connection word per sentence is plenty. Two ("However, moreover, therefore...") is a stylistic tell of unedited prose.

4.4 Six errors that mark a text as unproofed

• "respectively" misuse — it is only valid when two lists are mapped item-to-item in the same sentence order: "Models A and B achieve 71.2 and 73.9 BLEU, respectively." If the mapping is not obvious, write two sentences.
• Countability — "an accuracy", "a feedback", "a research", "an advice" are wrong; use "an accuracy of X%" or "pieces of feedback". Note: "data" may be treated as singular mass ("the data is") or plural ("the data are"); pick one per venue style and hold it.
• Article/determiner omissions — "the results are shown in Figure 2" (not "results are shown"). Every countable noun needs a determiner.
• Comma splices — join independent clauses with a semicolon, a full stop, or a conjunction, never a bare comma: "The model converges quickly; however, it overfits after epoch 10." (Or "... quickly, but it overfits ...".)
• "which" vs "that" — restrictive clause (defining) uses "that" with no comma; non-restrictive (extra information) uses "which" with a comma: "the model that we trained" vs "the model, which we trained for 40 epochs, ...".
• Sentence length spirals — past ~35 words, comprehension drops sharply. Split: one idea per sentence, and put the important clause last.

4.5 Cutting 15% of your word count without losing meaning

Replace these with one word each:

|---|---|

Then read each paragraph asking: what is new information here? Delete the sentence that is not new. Most first drafts contain 10–15% sentences that only restate the previous paragraph.

4.6 How to use assistance honestly

• Use language assistance for polishing your own sentences, never for generating claims, citations or experiments.
• Verify every sentence after assistance: language models routinely introduce confident but false statements, and especially false citations (see Appendix B).
• Disclose in the venue's designated field, exactly as instructed. Policies differ: some venues restrict models to grammar correction, others require describing which parts were assisted. The CFP is the authority, not blog posts.
• Keep your own diff history. If anyone asks "which part was generated?", you should be able to answer with a file, not a memory.

---

Part 5 · Venue selection and submission strategy

5.1 Comparative picture (verified 2026 figures)

|---|---|---|---|---|---|

Verify every row before you rely on it. CFP dates, page limits, and policy details move every cycle; acceptance rates move every year. The stable parts are the *culture* columns.

5.2 The ARR model changes your calendar (NLP only, but worth understanding generally)

Under ARR you submit to a review pool, receive reviews and a meta-review after a 10-week cycle, may revise and resubmit with reviewer continuity, and then commit the reviewed paper to a participating venue, whose programme chairs decide acceptance [ref:arrdates]. Consequences:

• The real deadline is the final ARR submission date for the target venue, not the venue's own papers deadline. Example pattern from the ARR dates page: EMNLP 2026 required a final ARR submission by 25 May 2026 with commitment by 2 August 2026; EACL 2027's final ARR submission was 3 August 2026 with a 11 October 2026 cycle end [ref:arrdates].
• Reviewer registration is mandatory for all authors, and non-compliance can trigger desk rejection.
• From October 2026, review is only guaranteed when the submission brings a qualified service contributor; caps apply (20 submissions per single author, 5 as first/co-first author) and every author needs an OpenReview profile with affiliation history, career status, ORCID and, where applicable, DBLP / ACL Anthology links [ref:arrcfp].
• Practical consequence: plan cycles backwards. To target a venue you must usually be camera-ready-one-round-of-reviews earlier than you would with a plain deadline.

5.3 Choosing a venue: a decision tree

1. Which community reads this?  (method-centric → ML venues; language-centric → ACL family; visual → CV venues)
2. Which reviewer would consider this a strong result?  (their yardstick decides novelty)
3. Is your strongest evidence breadth or depth?
     depth/theory → ICML / ICLR / NeurIPS;   breadth/application → AAAI / IJCAI / applied tracks
4. Does the venue's page limit destroy your content?  (8-page ACL main vs. 9-page NeurIPS body + appendix)
5. Is your work a negative result, a replication, or a resource?
     → look for dedicated tracks (Datasets & Benchmarks, Reproducibility, Findings-equivalent outlets)
6. Check the CFP for: LLM-use policy, limitations requirement, compute/reporting requirements,
   supplementary-material rules, and whether anonymous preprint posting is permitted before review.

5.4 Avoiding desk rejection (the 10-minute pre-flight)

Before you press submit, verify mechanically:

• [ ] Page limit respected including references? Anonymous submission / double-blind rules followed (no identifying author identity, no "our prior work [ours]")?
• [ ] Required statements present: Limitations / Ethics / Reproducibility / Impact (whatever the CFP demands)?
• [ ] Every reference resolves: correct title, correct authors, existing DOI (see Appendix B)?
• [ ] All authors registered as required (ARR: all authors) and OpenReview profiles accurate?
• [ ] Format/template correct (official LaTeX style of the venue), no black-box-incompatible fonts, PDF/A issues checked?
• [ ] Supplementary material allowed and correctly anonymised? Not referenced as the only source of a core claim?
• [ ] LLM-use policy field filled exactly as requested?
• [ ] Figures legible in print greyscale and at 2x zoom? Table captions self-contained?

5.5 Timeline template

|---|---|

---

Part 6 · Rebuttal: turning three reviewers into a decision

A rebuttal is not a defence; it is additional evidence delivered under time pressure. Reviewers read it to decide the paper's fate, and at the large venues the decision is made in area-chair discussion where your answer is the last input.

6.1 Principles

• Answer every comment. Unanswered minor points accumulate into "the authors did not engage".
• Lead with the answer, not with gratitude. One thanks-sentence maximum, then direct response.
• Classify each comment before writing (see table). A misclassified comment produces a mismatched answer.
• New experiments must be immediately interpretable: a table with the numbers asked for, referenced to the updated section.
• Concede accurately. If a reviewer is right, say "We agree" and show the change. Concessions cost little and buy credibility for the points where you disagree.
• When a reviewer is factually wrong, correct them politely and *cite your own text*: "We may have been unclear: Section 3.2 defines X as ...; the setting you describe corresponds to line 214. We have rewritten the sentence to prevent this reading." Never accuse them of not reading.
• Do not promise what you cannot deliver in the camera-ready. A promised experiment that never appears is cited against you later.

6.2 Comment taxonomy → response pattern

|---|---|---|

6.3 Templates

Agree and fix

> Reviewer X: The comparison omits [baseline], which is the standard method for this task.

We agree. We have now run [baseline] using the authors' released code with the default settings
(Table 4, rows 3–4). Our method remains ahead by 1.8 points, and the gap narrows on [subset],
which we now discuss in Section 5.3. We have removed the sentence claiming X and replaced it with
the measured statement.

Clarify a misreading

> Reviewer Y: The method requires gold labels at inference time, which makes it unusable.

We see how the draft read that way, and we have rewritten Section 3.2. To be explicit: gold labels
are used **only** during training (lines 214–219). At inference only [signal] is available; we
report this setting in Table 2, rows 5–6. The confusion came from our use of the term "supervision"
without qualification, which we have now corrected throughout.

Scope / future work

We agree this is beyond the current scope. We have added it explicitly to Limitations:
"Our analysis assumes a fixed vocabulary; extending to open-vocabulary settings is left to future work."

6.4 Budgeting the rebuttal

• Day 1: classify comments; list what requires new evidence. Do not write yet.
• Day 2: run the feasible experiments; prepare one table.
• Day 3: write responses; every response references a concrete change (section/table number).
• Day 4: edit for tone; remove anything defensive; a colleague reads it cold.
• Avoid promising lengthy reruns with <48h left. Prefer analyses of existing runs (they are fast and often equally convincing).

6.5 After a rejection

• Separate fixable from fundamental causes. Missing baseline / misread claim / unclear writing = fixable in days. Weak problem formulation = stop and reformulate.
• Steel-man each review: assuming the reviewer is reasonable and smart, what in your paper caused this reading? The answer is usually a missing sentence, not a missing method.
• Recycle all prior reviews into the next version before adding a new idea. Mixing an unaddressed complaint with new content reliably reproduces the same rejection.
• Consider the venue down-shift track (workshop, Findings-equivalent, journal version, or a neighbouring conference). Rejection notes are often reusable evidence when you resubmit.

---

Part 7 · Tool stack and safe automation

Prices change constantly; treat all pricing columns as "check current price".

7.1 Core stack

|---|---|---|

7.2 Automate the boring verification

Three automations pay for themselves on the first submission:

• Figure pipeline: every figure is generated by a script from committed raw results; `make figures` regenerates all of them. This removes "figure doesn't match table" (a real and fatal defect).
• Result-pipeline check: a script asserts that every number quoted in the manuscript exists in the results files; a mismatch fails the build. Catches silent copy-paste drift and version changes.
• Reference check: parse the `.bib`, verify DOI existence against a bibliographic database, list duplicates, list references never cited in the text, and list in-text citations missing from the `.bib`. See Appendix B.

7.3 Where language models belong in this workflow (and where they do not)

Safe uses: paraphrasing *your* sentences; checking consistency of tense and notation; producing a first draft of a check-list you verify; summarising your own paper into abstract bullets that you then rewrite; generating alt-text and captions you check.

Unsafe uses: generating related work from memory (it fabricates titles); writing citations (this is the exact failure that led to desk rejections at ICLR 2026 [ref:iclr2026]); making up numbers; summarising a paper you have not read.

Rule of thumb: a language model may touch your prose, never your evidence. Every output touching evidence needs an external check — DOI for references, files for numbers.

---

Appendix A · Submission-readiness checklist (run 60 minutes before submitting)

Claims and structure

• [ ] Every contribution bullet maps to a measurable result; every result maps to a claim (claim→evidence matrix complete)
• [ ] Abstract's numbers appear identically in the body
• [ ] Title contains the mechanism and the task; no unsupported adjectives
• [ ] Limitations section is specific, not boilerplate

Method

• [ ] Overview figure exists and each numbered stage is referenced in the text
• [ ] Notation table exists; no symbol used with two meanings
• [ ] Key equations numbered and referenced by number
• [ ] Complexity / cost stated

Experiments

• [ ] Includes current strong baselines + canonical baselines + ablation baselines
• [ ] ≥3 seeds; mean ± std reported; sample sizes stated
• [ ] Significance test or confidence interval for headline comparisons
• [ ] Efficiency table (params / FLOPs / latency / memory)
• [ ] Error analysis or failure cases included
• [ ] Tables: best bolded, runner-up underlined, caption self-contained

Related work

• [ ] Grouped by theme; every group ends with the unsolved part
• [ ] Closest competitor cited, even if inconvenient
• [ ] Recency sweep for the last 6 months completed after the draft was finished

English and format

• [ ] Tense zones disciplined (present=method, past=results)
• [ ] No "respectively" misuse; no it-is-important-to-note-that padding
• [ ] Figures legible in greyscale and at 2x zoom
• [ ] Hedges used for every claim that exceeds the evidence

Policy

• [ ] LLM-use field filled per CFP
• [ ] Required statements present (Limitations / Ethics / Reproducibility / Impact)
• [ ] All authors registered / profiles accurate (ARR)

Appendix B · Reference integrity: how to avoid a hallucinated-citation desk rejection

ICLR 2026 desk-rejected submissions whose references pointed to papers that do not exist, after checking every flagged reference against multiple bibliographic databases and web search, with human confirmation and an appeal channel [ref:iclr2026]. The cost of an unverified bibliography is now the whole submission.

Ten-minute procedure:

• Extract every citation key from the manuscript; verify each exists in the `.bib`; remove orphan entries.
• For each entry, check title, first author, year and venue against a database (DBLP, Semantic Scholar, Crossref, OpenAlex); prefer DOI lookups over manual typing.
• Flag every entry you cannot confirm by DOI. These are the ones to delete or replace — never guess-finish one.
• Detect duplicate DOI/title pairs introduced by merged BibTeX files.
• Check every in-paper year matches the published year; check arXiv identifiers and versions.
• If you used any automated suggestion for a reference, treat it as unverified by default (step 2 applies doubly).

Automated version: keep this as a script (check_refs.py) run in CI; see §7.2 item 3.

Appendix C · Simulating your own reviewers

Before submission, run the following exercise once per co-author and collect answers independently, then compare. Where the three readings disagree, you have found a writing defect, not three opinions.

• What is the one claim of this paper? (Compare answers. Differences here are fatal.)
• What is the weakest part of the evidence?
• Which baseline did you expect that we did not include?
• Which sentence would you have quoted in a rejection?

Reviewer-simulation prompts for this exercise are in PROMPT_PACK.md, with guardrails preventing fabricated citations.

Appendix D · Glossary

• ARR — ACL Rolling Review, the reviewing pool and cycle system used by ACL-family venues [ref:arrcfp].
• Camera-ready — the final version after acceptance, often allowing one extra page.
• Desk reject — rejection before review, for policy or formatting violations.
• Commitment (ARR) — submitting an ARR-reviewed paper to a participating venue for an acceptance decision [ref:arrdates].
• Meta-review — area chair's synthesis and recommendation.
• Author response / rebuttal — the author's synchronous reply window.
• ABD (ablation study) — component-removal experiment that attributes the gain.
• Confidence interval — range that would contain the true effect under repeated sampling; more informative than a bare p-value.

Appendix E · References (verified 2026-09-27)

All links checked on 2026-09-27. Facts labelled "verify" move each cycle; consult the venue's official page before relying on them.

• [ref:neurips2025] NeurIPS Program Committee Chairs, "Reflections on the 2025 Review Process" — 21,575 valid submissions, 5,290 accepted (24.52%). https://blog.neurips.cc/2025/09/30/reflections-on-the-2025-review-process-from-the-program-committee-chairs/
• [ref:iclr2026] ICLR 2026 Program Chairs, "A Retrospective on the ICLR 2026 Review Process" — 19,525 valid submissions, 779 desk rejects, 27.4% acceptance, LLM policies and hallucinated-reference enforcement. https://blog.iclr.cc/2026/03/31/a-retrospective-on-the-iclr-2026-review-process/
• [ref:cvpr2026] CVPR 2026 Program Committee news release — 16,092 submissions, ≈25.2% accepted, 4,089 papers. https://cvpr.thecvf.com/Conferences/2026/News/Technical_Program
• [ref:arrcfp] ACL Rolling Review — Call for Papers (sustainable reviewing policy effective October 2026; author caps; mandatory OpenReview profile fields). https://aclrollingreview.org/cfp
• [ref:arrdates] ACL Rolling Review — Dates and Venues (10-week cycles; venue commitment dates). https://aclrollingreview.org/dates
• [ref:arxivmoderation] arXiv moderation policy — first-time posters require endorsement by a qualified arXiv author in their field; since October 2025 computer-science review/position papers require prior peer review. https://info.arxiv.org/help/moderation/index.html
• arXiv monthly statistics (volume trend by year). https://arxiv.org/year/stat/2026

**Cited facts must be re-verified every submission cycle.** Acceptance rates, page limits, policy fields and dates change. A stated number that is one cycle out of date is a factual error in your own manuscript.

FAQ

References (verified 2026-09-27)

• NeurIPS 2025 review reflections
• ICLR 2026 review retrospective
• CVPR 2026 technical program news
• ACL Rolling Review CFP
• ACL Rolling Review dates
• arXiv moderation policy