HeadCoach Bench puts frontier language models on the headset inside real football games — NFL Head Coach (2006) on PS2 and NFL Head Coach 09 on PC — and grades every call by what actually happens on the field. Alongside the live harness, I built an interactive coaching booth where anyone can watch the broadcast and inspect both models' chain-of-thought reasoning, SVG play drawings, coordinator film reads, and custom plays written directly into game RAM.
Most AI benchmarks test models on static multiple-choice questions they have already memorised. I wanted to see what happens when two models have to coach a 60-minute football game against each other under clock pressure. On PS2, a Python harness talks to PCSX2 over the PINE socket protocol, reading the 28-byte live game state from RAM and pressing controller buttons to navigate the play-call menus. On PC, a two-coach harness hooks NFL Head Coach 09 via Win32 process memory, holding the game clock while both models think and verifying from the in-memory play-call struct that the engine ran the exact call each model chose.
THE DIAGRAM AHA MOMENT
Halfway through building the harness, I realised the models were coaching blind: they only saw play names like Flood Slot Post or Strong Eagle Slant 3, with no diagrams. When I rendered play diagrams as clean PNGs and tested six frontier vision models across 480 graded cards, image comprehension topped out at 80% to 99% — models miscounted receivers and misread zone drops from pixels. Meeting the models at the code layer changed everything: feeding structured vector play specs and rendering them as high-contrast SVGs (PlayArt) jumped comprehension to 99%–100%, with 0.00-yard movement error and exact route-vs-defender separation math.
THE COVER 2 LOOP & CUSTOM PLAYS
In an early Play Data Off game between DeepSeek V4.1 Flash and Kimi K3.1 Pro (hs953), DeepSeek discovered that Ride 132 Flood shredded the Colts' Cover 2 shell for 7.3 yards a pop and called it 22 times. Kimi explicitly recognised the spam in its reasoning and halftime interview — 'I reacted too slowly... I stayed too predictable with two-deep looks' — yet stayed trapped in a losing loop because the stock Colts playbook in HC09 has 24 Cover 2/Tampa 2 calls out of 25 regular defenses and zero man or single-high Cover 3/4 counters. That deadlock inspired the custom play creator: before kickoff, each model reads a scouting report on the opponent's playbook, designs 5 custom offensive and 5 custom defensive plays that are compiled into HC09's in-memory UDB0 play tables, and rewrites its own playbook slots at halftime (hs981).
LIVESTREAM & BOOTH
During live exhibition games, a portable OBS Studio instance on the Windows rig captures the 1280×800 game window, isolated WASAPI game audio, and a live bottom ticker driven over obs-websocket, streaming to this page while watch.html runs in read-only spectator mode so viewers can click through every snap as it happens.
This is the real HC09 Watch coaching booth running inside this page. Use the selector in the top bar to switch between three full games — hs955 (Play Data On + full booth with pre-snap SVG drawings, staff film reads, and interviews), hs953 (the Cover 2 loop where DeepSeek ran Ride 132 Flood 22 times), and hs981 (custom plays designed in RAM and rewritten at halftime; click 'Created plays' in the top bar).
Gameplay Clip
Gallery
6 pieces
HC09 Watch: each coach's pre-snap SVG drawing, opponent expectation, true play data, and game diagramSnap-by-snap chain-of-thought reasoning, token cost, and play verificationPS2 harness: live PINE memory bridge and play-call executionPlayArt: meeting models at the SVG and vector code layer for 0.00-yard exact geometryCustom offensive plays (Rail Mesh, Smash Seam) designed by the model and injected into HC09 RAMCustom defensive counters (Choice Net Three, Buzz 3 Contain, Quarters Wall) built to break the Cover 2 loop