Anti-Autoresearch 🛡️

· · · · · · · ·

🔬 The field has tolerated unreliable autoresearch long enough — Anti-Autoresearch is the read that finally catches it.

天下苦 autoresearch 久矣 —— Anti-Autoresearch 替研究者们一眼看穿不靠谱的工作。

🏆 Built on a battle-tested foundation: ARIS (~12.5k★ · HuggingFace Daily Papers #1 · 78+ skills across 7+ platforms). Anti-Autoresearch points ARIS's production audit DNA (experiment-audit · paper-claim-audit · citation-audit · kill-argument) outward — auditing a third party's submission instead of your own.

Autoresearch has gone mainstream, and a fast-growing share of what reaches the review pile is machine-generated — and a lot of it doesn't hold up: tables that don't match the text, baselines that aren't there, open-sourced code that won't reproduce its own paper. Reviewers, area chairs, and honest authors increasingly need to verify that, not just suspect it.

Regardless of who or what wrote a paper, does the science hold together and reflect its own evidence? Anti-Autoresearch audits a submission for self-consistency and fabrication, and produces a span-anchored, reviewer-ready report. It is not an opaque AI-text classifier (no authorship probabilities, no "AI-written" verdict) and does not judge misconduct — it surfaces discrepancies a human reviewer should investigate. Separately, it lists transparent, itemized AI writing-style impressions in a quarantined, zero-verdict-weight section (a paper can be integrity-CLEAN while listing many), because reviewers react to them.

---

🧭 What's inside

46 integrity patterns across 8 families — the coverage vocabulary every finding cites — plus 13 zero-weight AI writing-style impressions and 2 advisories:

	Family	Catches
A	Numeric self-consistency	数值自洽:table vs text vs delta arithmetic that doesn't add up
B	Method & scope	方法与范围:the described method/scope ≠ what was actually done
C	Baseline integrity	baseline 诚信:missing, weak, or unfairly configured comparisons
D	Experiment integrity	实验诚信:fake ground truth, phantom results, code ≠ numbers (needs code)
E	Citation integrity	引用诚信:fabricated, misattributed, or retracted references
F	Presentation & surface	表面信号:layout / prose / figure signals — capped at minor
G	Proof & derivation	证明诚信:skipped obligations, circular or invalid derivations
H	Evaluation design & validity	评测设计:data leakage, LLM-judge validity, selective reporting

Delivered as 11 skills + 1 orchestrating workflow on a deterministic spine: a span-anchored, hashed evidence ledger → LLM auditors that only propose findings → a rules-only reporter that lists every proposal with what the auditor said about it, and summarizes — with 8 patterns eval-gated end-to-end (GRIM · GRIMMER · statcheck · delta arithmetic · hedge-density · …) and the whole gate in CI.

---

📰 News

• 2026-07-10 — 🧮 Math was invited to overturn false alarms — and honestly declined. We built exact calculators to re-check the arithmetic behind numeric criticals, hoping to auto-clear false alarms like rounding differences. Adversarial review broke every version — the killer was "dropout of exactly 50%" vs a table's 50.4%: a calculator can't know whether a number is exact or rounded. The calculators stayed, demoted: they attach their full working to the report for the human, and nothing gets auto-cleared.
• 2026-07-10 — 🛡️ Criticals must now survive their own defense. A critical has to state which innocent explanations it ruled out (blank → drops to major), face one fresh refutation thread whose only power is a ⚠️ CONTESTED marker for the human, and — if numeric — declare exactly which numbers it computes over. Reports stamp adjudicator: deterministic-rules-v2. ⚠️ Re-run the auditors on old findings files instead of re-adjudicating them.
• 2026-06-28 — ✍️ v0.5 — the AIS track. 13 itemized AI writing-style impressions now live in a separate, zero-verdict-weight report section — a paper can be integrity-CLEAN while listing many. Transparent impressions, never an authorship verdict; the 5 pure-style patterns moved out of family F.

Earlier updates (2026-06)

• 2026-06-27 — 🧬 v0.4 — the taxonomy grows to 8 families (new: proof & derivation G, evaluation design H — the full map is in What's inside above). The deterministic eval gate grew 3→8 patterns (GRIM / GRIMMER / statcheck + a conservative hedge-density screen), CI landed, and prior-art acknowledgments were added.
• 2026-06-26 — 🚀 v0.1 — initial release. Reviewer-side integrity forensics for autoresearch / AI-Scientist papers: evidence ledger, deterministic adjudicator, observability tiers. Not an AI-text detector.

🚀 Quickstart

Agent workflow (normal use)

Anti-Autoresearch runs as a Claude Code skill workflow — the Python tools are the deterministic spine inside that workflow, not the usual interface.

# 1) Install the skills + workflow (global, or pass a project's .claude/skills dir)
git clone https://github.com/wanshuiyin/Anti-Autoresearch.git
./Anti-Autoresearch/tools/install_anti_autoresearch.sh              # → ~/.claude/skills
# project-local instead: ./Anti-Autoresearch/tools/install_anti_autoresearch.sh ./.claude/skills

# 2) Wire the cross-model reviewer (end state: Claude Code exposes mcp__codex__codex)
claude mcp add codex -- codex mcp-server
claude mcp list

# 3) Audit a paper
claude
> /anti-autoresearch ~/papers/submission

The run writes REPORT.md + report.json + claims.json + per-skill *.findings.json into the paper directory. Put the code/result artifacts alongside the paper to unlock L2 checks; PDF/source-only runs are observability-limited by design.

Zero verdict weight — th