Benchmark aggregates
Results pool session observations across all included runs. Each benchmark page contains per-arm Median, IQR (Q3 − Q1), p90, and Max, along with session and token usage and cost availability.
Overall: 4702 sessions · 1667960323 input tokens · 11390739 output tokens · 4270120 reasoning tokens · 1683621182 total tokens · 888.3683 known cost (credits) across 573 priced arm groups · 167 unpriced arm groups.
| Benchmark | Sessions | Arm/context groups | Input tokens | Output tokens | Total tokens | Known cost (credits) | Unpriced groups |
|---|---|---|---|---|---|---|---|
| grpc__grpc-go-2735 | 6 | 5 | 1387837 | 22329 | 1416350 | 0.9818 | 1 |
| grpc__grpc-go-2951 | 4 | 2 | 1424524 | 20273 | 1447193 | 1.0344 | 0 |
| grpc__grpc-go-3361 | 13 | 5 | 5446745 | 50576 | 5527485 | 3.8071 | 0 |
| task-00-hi-overhead | 142 | 10 | 2520161 | 10588 | 2540633 | 1.9100 | 4 |
| task-01-rename-local | 266 | 22 | 21781298 | 274869 | 22127973 | 7.7726 | 14 |
| task-01b-rename-local-large-context | 120 | 8 | 9486455 | 138308 | 9662287 | 3.7923 | 4 |
| task-02-rename-cross-pkg | 467 | 18 | 55238673 | 694794 | 56192022 | 28.3170 | 8 |
| task-03-rename-interface | 120 | 8 | 16201134 | 76331 | 16287871 | 9.8477 | 0 |
| task-04-insert-public | 369 | 82 | 51933364 | 325882 | 52382237 | 33.7948 | 18 |
| task-05-insert-method | 118 | 8 | 15192024 | 51008 | 15253303 | 8.7393 | 0 |
| task-06-imports-cleanup | 113 | 8 | 15381031 | 65515 | 15472949 | 9.2999 | 0 |
| task-07-generate-template-main | 462 | 194 | 76836514 | 780271 | 78017293 | 55.4606 | 48 |
| task-07-imports-qualify | 120 | 8 | 21625167 | 81945 | 21728279 | 11.7647 | 0 |
| task-08-delta-recovery | 120 | 8 | 20480961 | 94634 | 20604821 | 11.7060 | 0 |
| task-09-composite-refactor | 409 | 91 | 89192933 | 616488 | 90048863 | 49.9754 | 22 |
| task-10-disambiguate-var | 120 | 8 | 13702509 | 56029 | 13770867 | 7.9794 | 0 |
| task-11-mixed-sink-api-migration | 876 | 170 | 831576595 | 5817789 | 839968251 | 444.4411 | 42 |
| task-12-rename-scale-01 | 80 | 6 | 10195712 | 34811 | 10234013 | 5.7798 | 0 |
| task-12-rename-scale-02 | 80 | 6 | 11311566 | 43897 | 11359584 | 6.2746 | 0 |
| task-12-rename-scale-04 | 80 | 6 | 11279200 | 52255 | 11337116 | 6.4603 | 0 |
| task-12-rename-scale-05-multifile | 80 | 6 | 13320390 | 75265 | 13410702 | 8.1165 | 0 |
| task-12-rename-scale-08 | 80 | 6 | 11238882 | 59859 | 11304702 | 6.6990 | 0 |
| task-12-rename-scale-16 | 80 | 6 | 13011270 | 80785 | 13099831 | 8.2198 | 0 |
| task-12-request-identity-migration | 242 | 22 | 111554645 | 860990 | 112587917 | 45.0060 | 5 |
| task-13-rename-followup-scale | 100 | 6 | 200229120 | 681607 | 200963100 | 90.9316 | 0 |
| zeromicro__go-zero-2463 | 21 | 13 | 28186497 | 241934 | 28540023 | 15.4600 | 0 |
| zeromicro__go-zero-964 | 14 | 8 | 8225116 | 81707 | 8335517 | 4.7964 | 1 |