FairBakeSonnet 5.5
- Maker
- Anthropic
- Harness
- Claude Code 2.1.316
- Effort
- medium
- Runs
- 2 a task, 60 in all
- Run dates
- 1 to 6 Nov 2026
- Prices as of
- 1 Nov 2026, list
- Solved
- 44 of 60
- 73% · 8th of 10
- Cost per fix
- $0.37
- 6th lowest of 10
- Time per run
- 43s
- median · 1st lowest of 10
- Lines to review
- 37
- median · 2nd lowest of 10
All 30 tasks
Where it failed
Solved on every run: 19 tasks. Failed at least one run: 11 tasks.
| Task | Runs | Run 1 | Run 2 | All agents |
|---|---|---|---|---|
| Private screenshots open for outsiders, public ones don'tJavaScript · Bug fix | $0.12 · 28s · 24 lines257K tokens in · 2.6K out | $0.12 · 27s · 32 lines232K tokens in · 2.6K out | 17 / 19 | |
| Server takes traffic after a failed startupPython · Bug fix | $0.12 · 3m17s · 64 lines195K tokens in · 3.8K out | $0.11 · 2m33s · 54 lines214K tokens in · 3.6K out | 17 / 19 | |
| Every player from one provider shows zero pointsTypeScript · Bug fix | $0.084 · 37s · 9 lines218K tokens in · 1.9K out | $0.18 · 1m07s · 26 lines410K tokens in · 6.5K out | 13 / 19 | |
| Computer bidders hoard their fundsTypeScript · Bug fix | $0.33 · 1m57s · 30 lines799K tokens in · 12K out | $0.26 · 1m20s · 22 lines525K tokens in · 9.3K out | 13 / 19 | |
| Choosing "Uncategorized" doesn't stickJavaScript · Bug fix | $0.12 · 26s · 34 lines200K tokens in · 2.9K out | $0.14 · 32s · 36 lines274K tokens in · 3.3K out | 12 / 19 | |
| Watch a list of events instead of onePython · Feature | $0.90 · 3m52s · 721 lines2.2M tokens in · 35K out | $0.85 · 3m46s · 708 lines2.1M tokens in · 31K out | 12 / 19 | |
| One failed check silences the other alertsPython · Feature | $2.14 · 8m03s · 1657 lines7.2M tokens in · 80K out | $1.45 · 5m45s · 1088 lines4.6M tokens in · 50K out | 11 / 19 | |
| When offers are equal, the sale is credited to the losing buyerPython · Bug fix | $0.23 · 52s · 108 lines360K tokens in · 6.8K out | $0.17 · 46s · 85 lines286K tokens in · 4.4K out | 10 / 19 | |
| One provider's teams appear with nobody on themTypeScript · Bug fix | $0.20 · 1m03s · 40 lines483K tokens in · 5.4K out | $0.21 · 53s · 50 lines465K tokens in · 5.0K out | 4 / 19 | |
| Pages load with their numbers blank once the season startsTypeScript · Bug fix | $0.18 · 50s · 40 lines263K tokens in · 3.3K out | $0.20 · 53s · 39 lines346K tokens in · 4.1K out | 1 / 19 | |
| Hundreds of players missing from an importPython · Bug fix | $0.22 · 58s · 12 lines514K tokens in · 4.8K out | $0.26 · 1m12s · 33 lines601K tokens in · 5.7K out | 0 / 19 |