archwall 把架构画全Every architecture, drawn in full

ScientistTwo：从想法到论文，自己跑一圈ScientistTwo: idea to paper, all on its own

agentsAI scientistmulti-agentresearch

ScientistTwo overview: input is a research problem plus the current best paper and its code; 1 ideas: find limitations, check novelty; 2 experiments: a coding agent and a critic test on a small slice, then all data, and results feed back to the ideas; 3 ablation: keep only the parts that help; 4 writer: draft and improve; 5 peer review: an AI reviewer scores 1 to 10, below 8 a rebuttal runs new experiments, at most 2 rounds, and the draft goes back to the writer; 6 meta-review: accept gives the final paper and code, or the idea goes back to step 3 once

ScientistTwo 是 Google Cloud AI Research 的研究代理。给它一个研究问题，加上当前最好的论文和代码，它自己想点子、做实验、写论文、审论文，不用人管。

每步都一个套路：一个代理做出东西，评审决定接收、淘汰或打回改。每个循环有轮数上限。

One pattern at every stage: 1 make: a specialist agent makes a candidate, such as ideas or limitations, code and results, or a draft; 2 judge: a critic asks one question per stage, such as is it new or is it better, and answers accept, refine or reject; 3 verdict: accept goes to the next stage, reject is dropped; 4 refine: a refine agent fixes it from the notes and sends a new version back; every loop has a round limit
每一步都是同一个套路：做出来，评审判，要么过，要么改，要么丢

1. 想点子。 列出最好方法的短板，出点子去补，按新颖度排队。

Stage 1, ideas: 1 the input is the research problem, the best paper and its code, and a limitation extractor lists what it gets wrong; 2 a limitation verifier checks the list is enough, and more are found if not, at most 16 rounds; 3 an initial idea generator writes one idea and a novelty checker scores it against 2 papers found on Google Search; 4 an idea generator adds newer, different ideas, and the seed ideas are ranked by novelty, most original first
想点子：先找现有方法的短板，再出点子，按新颖度排队

2. 做实验。 点子先在一小部分数据上试，比基线好才跑全量。

Stage 2, test one idea: 1 a baseline coder reruns the best paper on a small slice of the benchmark; 2 a subset coder builds the idea and a subset critic compares it with the baseline; 3 verdict: good scales up, bad is dropped, promising gets tuned for at most 2 rounds and otherwise counts as bad; 4 a full-set coder runs every dataset and a full-set critic and engineer do the final check; out come code, results, good or bad, and lessons
做实验：先小后大，小规模上不行的点子不花大算力

结果交给进化器，写出更好的点子，再由选择器挑最好的。

Stage 2, outer loop: 1 execution traces of every idea tried, what worked and what failed; 2 an idea evolver reads them and writes better ideas, plus the next untried seed idea, 2 ideas per round; 3 the implementer runs them small first, and results go back into the traces, up to 4 rounds, stopping early at 4 good ideas; 4 a selector picks the best idea with its results and code; if no idea is good, the whole run stops
进化：从成败记录里长出新点子，最后挑一个最好的

3. 消融。 一次拿掉一部分，看哪部分有用，必要时改一次方法。

Stage 3, ablation: 1 an ablation planner takes the best idea, results and code and writes one plan per part; 2 an ablation coder runs every plan; 3 an ablation critic checks whether the breakdown is clean, and clean goes on to writing; 4 otherwise a full-set engineer revises the method, a result comparison keeps it only if it is better, and the ablation runs again, at most once in the paper's setting
消融：拆开看每部分的贡献，没用的砍掉

4–5. 写和审。 AI 审稿人打分，低于 8 分（满分 10）就补实验、改论文。

Stages 4–5, write and review: 1 an initial drafter built on PaperOrchestra writes a full paper in ICLR 2025 format from the idea, results and ablations; 2 a peer reviewer, ScholarPeer, scores 1 to 10, and 8 or more goes to meta-review; 3 below 8, a rebuttal planner picks new experiments and a rebuttal coder runs them; 4 a draft enhancer updates the text, tables and figures and sends it back, at most 2 review rounds
写和审：回应审稿意见靠补实验，不靠改措辞

6. 元评审。 最后一关：接收，或打回再改一次。改完不更好，就交上一版。

Stage 6, meta-review: 1 a meta-reviewer reads the paper and review; accept gives the final paper and code; 2 on refine, a full-set engineer changes the idea itself; 3 a result comparison checks the new results, and if not better the change is dropped and the previous paper ships; 4 if better, ablation, draft and review run again and the new paper returns, at most once in the paper's setting
元评审：最后一道关，改了不更好就交上一版

四项检查保证诚信：代码能重跑，违规代码筛掉，引用是真的，方法和代码对得上。

Integrity: 1 rerunnable: the coding agent writes scripts that can be rerun, every score reproduced in 49 of 49 papers; 2 rules kept: a validation filter right after experiments drops rule-breaking code, 0 of 49 broke rules; 3 real references: a search-backed model finds made-up citations and the writer fixes them, 0 of 1,814 made up; 4 method matches code: the coding agent checks code against the paper and the writer fixes the method section, 49 of 49 match
诚信：四项检查，确保论文说的和代码做的一致

多数代理用 Gemini 3.6 Flash，四个（含实验编码）用 Claude Code 加 Opus 4.8。一个问题平均 2 到 3 天、3765 美元。

Models, time and cost: 1 Gemini 3.6 Flash runs most agents; the experiment coder, ablation, rebuttal and draft enhancer use Claude Code with Opus 4.8; 2 swapped for Antigravity on Gemini 3.8 Flash on 5 ICLR 2026 tasks, it still found new best methods; 3 2 to 3 days per problem, mostly idea refinement and the review loops; 4 $3,765 on average, tokens plus virtual machines, measured on 33 NeurIPS 2025 problems
运行：一个问题两三天，平均约 3765 美元

在 107 个取自 NeurIPS、ICLR、ICML 论文的问题上，86 个超过人类最好成绩，平均提升 25.2%。

How it was tested: 1 107 problems, each the problem and code of an accepted paper: 38 NeurIPS 2025, 5 ICLR 2026, 64 ICML 2026 spotlights; 2 86 of 107 beat the human best, 80.4%, with a 25.2% average gain; 3 ScholarPeer rated 7.5 of 10 with 91.9% accepted, the Stanford Agentic Reviewer 5.7 with 72.1%; ScholarPeer is also used inside, Stanford was unseen in development; 4 above the average accepted ICLR 2026 and NeurIPS 2025 paper, below ICML 2026 spotlights, and every other AI research agent got 0% from Stanford
测试：107 个题目，86 个超过人类最好成绩

ScientistTwo is a research agent from Google Cloud AI Research. You give it a research problem, plus the current best paper and its code. It comes up with ideas, runs experiments, writes the paper and reviews it, with no human in the loop.

Every stage works the same way. One agent makes something, and a critic accepts it, rejects it or asks for a fix. Each loop has a round limit.

One pattern at every stage: 1 make: a specialist agent makes a candidate, such as ideas or limitations, code and results, or a draft; 2 judge: a critic asks one question per stage, such as is it new or is it better, and answers accept, refine or reject; 3 verdict: accept goes to the next stage, reject is dropped; 4 refine: a refine agent fixes it from the notes and sends a new version back; every loop has a round limit
Every stage works the same way: make it, judge it, then accept, fix or drop it

1. Ideas. It lists what the best method gets wrong, writes ideas that fix it, and ranks them by novelty.

Stage 1, ideas: 1 the input is the research problem, the best paper and its code, and a limitation extractor lists what it gets wrong; 2 a limitation verifier checks the list is enough, and more are found if not, at most 16 rounds; 3 an initial idea generator writes one idea and a novelty checker scores it against 2 papers found on Google Search; 4 an idea generator adds newer, different ideas, and the seed ideas are ranked by novelty, most original first
Ideas: find what the best method gets wrong, then rank fixes by novelty

2. Experiments. Each idea runs on a small slice of the benchmark first. Only ideas that beat the baseline get the full benchmark.

Stage 2, test one idea: 1 a baseline coder reruns the best paper on a small slice of the benchmark; 2 a subset coder builds the idea and a subset critic compares it with the baseline; 3 verdict: good scales up, bad is dropped, promising gets tuned for at most 2 rounds and otherwise counts as bad; 4 a full-set coder runs every dataset and a full-set critic and engineer do the final check; out come code, results, good or bad, and lessons
Experiments: small first, so weak ideas never get the big compute

Results from every idea feed an evolver that writes better ones. A selector picks the best.

Stage 2, outer loop: 1 execution traces of every idea tried, what worked and what failed; 2 an idea evolver reads them and writes better ideas, plus the next untried seed idea, 2 ideas per round; 3 the implementer runs them small first, and results go back into the traces, up to 4 rounds, stopping early at 4 good ideas; 4 a selector picks the best idea with its results and code; if no idea is good, the whole run stops
Evolve: grow new ideas from what worked and what failed, then pick the best

3. Ablation. It removes parts one at a time to see which ones matter, and may revise the method once.

Stage 3, ablation: 1 an ablation planner takes the best idea, results and code and writes one plan per part; 2 an ablation coder runs every plan; 3 an ablation critic checks whether the breakdown is clean, and clean goes on to writing; 4 otherwise a full-set engineer revises the method, a result comparison keeps it only if it is better, and the ablation runs again, at most once in the paper's setting
Ablation: take it apart, see what each part adds, cut what doesn't help

4–5. Writing and review. An AI reviewer scores the draft. Below 8 out of 10, a rebuttal agent runs new experiments and the draft is updated.

Stages 4–5, write and review: 1 an initial drafter built on PaperOrchestra writes a full paper in ICLR 2025 format from the idea, results and ablations; 2 a peer reviewer, ScholarPeer, scores 1 to 10, and 8 or more goes to meta-review; 3 below 8, a rebuttal planner picks new experiments and a rebuttal coder runs them; 4 a draft enhancer updates the text, tables and figures and sends it back, at most 2 review rounds
Write and review: answer the reviewer with new experiments, not new wording

6. Meta-review. A last reviewer accepts the paper or sends the idea back for one more fix. If the fix doesn't beat the old results, it is dropped and the previous paper ships.

Stage 6, meta-review: 1 a meta-reviewer reads the paper and review; accept gives the final paper and code; 2 on refine, a full-set engineer changes the idea itself; 3 a result comparison checks the new results, and if not better the change is dropped and the previous paper ships; 4 if better, ablation, draft and review run again and the new paper returns, at most once in the paper's setting
Meta-review: the last gate; a fix that isn't better is dropped

Four checks keep it honest: code that reruns, a filter for rule-breaking code, real citations, and a method section that matches the code.

Integrity: 1 rerunnable: the coding agent writes scripts that can be rerun, every score reproduced in 49 of 49 papers; 2 rules kept: a validation filter right after experiments drops rule-breaking code, 0 of 49 broke rules; 3 real references: a search-backed model finds made-up citations and the writer fixes them, 0 of 1,814 made up; 4 method matches code: the coding agent checks code against the paper and the writer fixes the method section, 49 of 49 match
Integrity: four checks that the paper says what the code does

Most agents run on Gemini 3.6 Flash. Four, including the experiment coder, use Claude Code with Opus 4.8. One problem takes 2 to 3 days and $3,765 on average.

Models, time and cost: 1 Gemini 3.6 Flash runs most agents; the experiment coder, ablation, rebuttal and draft enhancer use Claude Code with Opus 4.8; 2 swapped for Antigravity on Gemini 3.8 Flash on 5 ICLR 2026 tasks, it still found new best methods; 3 2 to 3 days per problem, mostly idea refinement and the review loops; 4 $3,765 on average, tokens plus virtual machines, measured on 33 NeurIPS 2025 problems
Running it: two to three days and about $3,765 per problem

On 107 problems taken from NeurIPS, ICLR and ICML papers, it beat the human best on 86, by 25.2% on average.

How it was tested: 1 107 problems, each the problem and code of an accepted paper: 38 NeurIPS 2025, 5 ICLR 2026, 64 ICML 2026 spotlights; 2 86 of 107 beat the human best, 80.4%, with a 25.2% average gain; 3 ScholarPeer rated 7.5 of 10 with 91.9% accepted, the Stanford Agentic Reviewer 5.7 with 72.1%; ScholarPeer is also used inside, Stanford was unseen in development; 4 above the average accepted ICLR 2026 and NeurIPS 2025 paper, below ICML 2026 spotlights, and every other AI research agent got 0% from Stanford
Tested: 107 problems, 86 beat the human best

来源Source: scientist-two.github.io

← 全部架构← All architectures