FairBakeHaiku 5.5
- Maker
- Anthropic
- Harness
- Claude Code 2.1.316
- Effort
- medium
- Runs
- 2 a task, 60 in all
- Run dates
- 1 to 6 Nov 2026
- Prices as of
- 7 Oct 2026, list
- Solved
- 45 of 60
- 75% · 7th of 10
- Cost per fix
- $0.046
- 2nd lowest of 10
- Time per run
- 1m32s
- median · 4th lowest of 10
- Lines to review
- 58
- median · 3rd lowest of 10
All 30 tasks
Where it failed
Solved on every run: 19 tasks. Failed at least one run: 11 tasks.
| Task | Runs | Run 1 | Run 2 | All agents |
|---|---|---|---|---|
| Flag ledger entries that clash with earlier onesPython · Feature | $0.067 · 4m10s · 843 lines2.1M tokens in · 55K out | $0.064 · 3m37s · 722 lines2.6M tokens in · 41K out | 15 / 19 | |
| Import fails when the week has no data yetPython · Bug fix | $0.010 · 40s · 7 lines345K tokens in · 4.0K out | $0.008 · 34s · 17 lines195K tokens in · 3.2K out | 14 / 19 | |
| Every player from one provider shows zero pointsTypeScript · Bug fix | $0.011 · 39s · 7 lines341K tokens in · 4.4K out | $0.024 · 1m31s · 27 lines1.0M tokens in · 13K out | 13 / 19 | |
| Computer bidders hoard their fundsTypeScript · Bug fix | $0.032 · 2m19s · 50 lines1.2M tokens in · 22K out | $0.021 · 1m29s · 37 lines598K tokens in · 15K out | 13 / 19 | |
| Choosing "Uncategorized" doesn't stickJavaScript · Bug fix | $0.023 · 1m21s · 55 lines724K tokens in · 18K out | $0.035 · 13m28s · 54 lines1.3M tokens in · 22K out | 12 / 19 | |
| Watch a list of events instead of onePython · Feature | $0.061 · 3m06s · 342 lines2.5M tokens in · 34K out | $0.041 · 2m16s · 303 lines1.4M tokens in · 24K out | 12 / 19 | |
| One failed check silences the other alertsPython · Feature | $0.13 · 6m33s · 1137 lines6.1M tokens in · 78K out | $0.19 · 8m20s · 1603 lines9.7M tokens in · 108K out | 11 / 19 | |
| When offers are equal, the sale is credited to the losing buyerPython · Bug fix | $0.047 · 2m46s · 197 lines1.9M tokens in · 28K out | $0.037 · 2m02s · 189 lines1.4M tokens in · 21K out | 10 / 19 | |
| One provider's teams appear with nobody on themTypeScript · Bug fix | $0.028 · 1m44s · 88 lines998K tokens in · 16K out | $0.025 · 1m33s · 90 lines779K tokens in · 16K out | 4 / 19 | |
| Pages load with their numbers blank once the season startsTypeScript · Bug fix | $0.022 · 1m41s · 35 lines788K tokens in · 11K out | $0.016 · 53s · 35 lines441K tokens in · 7.7K out | 1 / 19 | |
| Hundreds of players missing from an importPython · Bug fix | $0.042 · 2m09s · 20 lines1.7M tokens in · 23K out | $0.031 · 1m50s · 46 lines1.1M tokens in · 17K out | 0 / 19 |