Optimize reactions in parallel
Task spaceMaximize yields across twelve amide-coupling reactions by allocating a shared experimental budget in a simulated laboratory.
Optimizing several reactions together requires deciding where additional experiments will be most useful. Agents must balance improving difficult reactions with refining promising conditions, using results from each plate to guide the next.
Agents allocate 96 experiments across twelve successive plates of eight wells, choosing which reaction and condition to test in each well. They then nominate a condition for each reaction. Scoring averages the yields from separate confirmation experiments equally across all twelve reactions, with unassigned reactions contributing zero.
Scoring
The mean confirmation yield across the assigned reactions must close at least 90% of the gap from the random-search baseline to the task-constrained oracle. This is a normalized improvement threshold, not 90% raw yield.
RESULTS
Maximize yields across twelve amide-coupling reactions by allocating a shared experimental budget in a simulated laboratory.
| Model | Pass@1 | Cost ($)Model/API inference cost only, averaged per task attempt. Lab and labor costs are not reported. Repeats are averaged using the same task and family weights as scores. | |
|---|---|---|---|
 GPT–5.6 Solxhigh | 0.0% | $3.00 | |
 Gemini 3.8 Flashhigh | 0.0% | $1.26 | |
 Claude Opus 5xhigh | 0.0% | $3.29 | |
| Claude Fable 5.1xhigh | 0.0% | $4.74 | |
| GPT–6 Astraxhigh | 0.0% | $7.85 | |
 Grok 4.6xhigh | 0.0% | $2.90 | |
0%50%100% | |||
Model trajectories
One example per model.
Run grading
Fail
| Measurement | Recorded result | Outcome |
|---|---|---|
| Normalized score | 43.5 % | × Fail |
Pass condition: The mean confirmation yield across the assigned reactions must close at least 90% of the gap from the random-search baseline to the task-constrained oracle. This is a normalized improvement threshold, not 90% raw yield.
Optimize reactions in parallelGPT–6 Astra xhigh
GPT–6 Astra xhighRun gradingFail
00:0000:01
Model activity
Loading transcript…