Stats
Does correcting the drafts make them right? · eval 25/10/2026, 01:24:52
Held-out emails that depend on house rules
| Setup | All fields right | Field accuracy | Median latency |
|---|---|---|---|
| GPT, no correctionsfrozen prompt, email only | 63.3%60–65% · 3 runs | 93.9% | 1856 ms |
| GPT + coordinator correctionswhat the app sends today · 6.0 notes per email | 83.3%75–90% · 3 runs | 96.8% | 2605 ms |
| GPT + rules written in the promptbaseline: someone writes the rules down | 81.7%80–85% · 3 runs | 97.7% | 1605 ms |
| Mubit adapter v1 · Qwen3-4B-Instructtrained by Mubit on calls up to ~16:30 Sat (25 of 38 corrections in by then) · no notesMubit grade 73.3% matches GPT on 15 test casespass 85.0%Below target · silent test on 10% of traffic | not tried yet | not tried yet | — |
| Mubit adapter v2next training run · first to learn from calls since v1, which carry 13 more corrections as notesNot trained yet · waiting on Mubit's next training run | not tried yet | not tried yet | — |
n=20 · guests within ±5% · eval calls are never recorded as training data · adapter rows from data/adapter-results.json
Corrections
… · latest 20
No corrections yet. Edit a spec on an enquiry to log one.