FairBakeFable 5.1
- Maker
- Anthropic
- Harness
- Claude Code 2.1.316
- Effort
- high
- Runs
- 2 a task, 60 in all
- Run dates
- 1 to 6 Nov 2026
- Prices as of
- 9 Sep 2026, list
- Solved
- 50 of 60
- 83% · 3rd of 10
- Cost per fix
- $3.14
- 10th lowest of 10
- Time per run
- 3m12s
- median · 10th lowest of 10
- Lines to review
- 142
- median · 7th lowest of 10
All 30 tasks
Where it failed
Solved on every run: 23 tasks. Failed at least one run: 7 tasks.
| Task | Runs | Run 1 | Run 2 | All agents |
|---|---|---|---|---|
| Some intercepted calls are refused before the service sees themRust · Bug fix | $4.47 · 8m58s · 70 lines2.4M tokens in · 36K out | $3.16 · 5m24s · 69 lines1.8M tokens in · 21K out | 16 / 19 | |
| Import fails when the week has no data yetPython · Bug fix | $1.75 · 2m33s · 97 lines1.1M tokens in · 12K out | $1.39 · 2m19s · 69 lines808K tokens in · 9.2K out | 14 / 19 | |
| Every player from one provider shows zero pointsTypeScript · Bug fix | $1.01 · 1m42s · 78 lines583K tokens in · 7.6K out | $1.60 · 3m11s · 176 lines760K tokens in · 15K out | 13 / 19 | |
| Computer bidders hoard their fundsTypeScript · Bug fix | $3.46 · 6m41s · 145 lines2.0M tokens in · 31K out | $1.71 · 3m28s · 56 lines660K tokens in · 15K out | 13 / 19 | |
| One provider's teams appear with nobody on themTypeScript · Bug fix | $2.65 · 4m26s · 260 lines940K tokens in · 21K out | $2.74 · 4m44s · 258 lines1.7M tokens in · 23K out | 4 / 19 | |
| Pages load with their numbers blank once the season startsTypeScript · Bug fix | $1.90 · 3m12s · 168 lines1.1M tokens in · 14K out | $1.87 · 3m24s · 120 lines1.1M tokens in · 14K out | 1 / 19 | |
| Hundreds of players missing from an importPython · Bug fix | $2.65 · 4m16s · 214 lines1.7M tokens in · 20K out | $2.26 · 3m56s · 84 lines966K tokens in · 17K out | 0 / 19 |