$ What a Timed AI Interview Actually Measures

Two attempted AI-conducted technical interviews — one cut short around ten minutes in, one completed — surfaced something concrete: fast, verbal recall of syntax-level terminology (const vs. var, the spread operator, interface vs. type) is a real weak spot under a cold, timed format. Worth saying plainly rather than only showcasing wins.

What didn't falter, in the same sessions, was the actual reasoning: spotting an off-by-one bug in a loop, and correctly explaining why it produced the wrong output. The gap wasn't understanding — it was retrieving the right word for something already understood, fast, out loud, on the clock. Farpost — 39 backend models, dozens of spec-tracked changes, real production bugs found by reasoning about spec-vs-code mismatches, not by naming vocabulary quickly — is concrete counter-evidence sitting right next to the gap.

There isn't a fix here in the usual sense — the honest response was recognizing the format mismatch for what it is: a timed vocabulary quiz measures something real, but narrower than it presents itself as measuring, and it's a different skill than the reasoning that actually ships production code.

A portfolio that's all technical wins reads as curated. Naming a real limitation plainly — paired with hard evidence of the specific capability the format failed to measure — is a more credible signal than either overclaiming or staying silent about it. It's also a quiet argument to anyone evaluating candidates the same way: a take-home or portfolio review surfaces real capability that a cold timed quiz can miss entirely.