AIME-Con 2026 · Educational Measurement as an AI-Native Profession
Peer review should use AI twice
Once for the reviewer, once for the editor, both run by the journal
Derek Briggs · University of Colorado Boulder
Why review needs help
Review was strained before generative AI
8M+
scientific publications in 2025, double the count five years earlier
4.5
invitations per completed review in 2025, twice the 2018 figure
53%
of reviewers used AI for review tasks in 2025, up from 29% a year earlier
The right comparator for AI-assisted review is the median human review, not the best one. (Stephen Turner, UVA, in Science)
Sources: Brainard, Science 393 (2026); Silverchair report (2026); Frontiers reviewer survey (2025)
What 14 journals say now
No journal sets its own reviewer rules
13
of 14 journals require authors to disclose generative AI use
12
of 14 bar reviewers from putting manuscript text into an AI tool
0
of 14 set their own reviewer or editor rules. Twelve inherit them from seven publishers.
The stated reason is confidentiality, not AI quality. Yet publishers already run their own AI on these manuscripts.
Source: Briggs, policy scan of 14 journals (3 NCME, 9 other measurement, 2 comparisons), 4 Oct 2026
The proposal
Two uses, both run by the journal
Use 1 · Reviewer
Review first, then audit the AI's review
  1. Write an independent review.
  2. Read the journal's AI review of code, math, statistics, and reporting.
  3. Revise or not, and tell the editor which.
Use 2 · Editor
Ask AI how the human reviews fit together
  1. Receive the human reviews.
  2. Ask AI where reviewers agree, conflict, and what no one checked.
  3. Decide, with the AI report as one input.
In both uses, humans write every review and make every decision.
The evidence · ICLR 2025 randomized trial
Reviewers revise after AI feedback
20K+
human reviews got optional AI feedback on their drafts
27%
of reviewers who got feedback revised, using 12,000+ AI suggestions
2/3
of revised reviews were judged better by an independent panel
Caveat: the AI commented on reviewers' drafts, not the manuscript. At AAAI-26, all 22,977 main-track papers got an AI review in under 24 hours, at about $1 each; it was weakest on novelty and significance.
Sources: Thakkar et al. 2025; Nature Machine Intelligence 2026, via Science; AAAI-26 pilot report, 2026
Unresolved
Whose judgment is it?
Accountability
When a reviewer revises after reading the AI's critique, who answers for the review? No journal policy in our field covers a review the journal itself supplies.
Independence
If every reviewer reads the same AI review, the reviews are no longer independent ratings. The editor may see more agreement but have less evidence.
Precedent: COPE and NCME let a reviewer bring in a colleague if the journal knows who helped. Does that rule extend to AI?