Notes from building a verification-first coding-agent harness — what the architecture makes measurable, and what the measurements surface.https://sarthak-blog.sarthak-xceed.workers.dev/https://sarthak-blog.sarthak-xceed.workers.dev/blog/02-dont-judge-by-pass-at-1/https://sarthak-blog.sarthak-xceed.workers.dev/blog/02-dont-judge-by-pass-at-1/A single pass@1 score collapses runs that differ on three axes it can't show: how a failure was shaped, whether a success repeats, and what either one cost. One model concluded a secret-safety task in 4 turns; two others avoided the leak but never concluded, burning ~77× the tokens. Then the reliability metric I reached for turned out to move in coarse sixths, and inverted when I reran the identical matrix. What to log, and how far to trust it.Sat, 11 Jul 2026 00:00:00 GMThttps://sarthak-blog.sarthak-xceed.workers.dev/blog/01-scaffold-not-model/https://sarthak-blog.sarthak-xceed.workers.dev/blog/01-scaffold-not-model/One model went from a 0.10 to a 0.75 success rate without anyone touching the model. The fault was a tool-schema bug in the harness, and the only reason I caught it was reading the agent's trajectory instead of the scoreboard. A case study on why agent evals score the whole system, and how to tell which part you're actually measuring.Wed, 01 Jul 2026 00:00:00 GMThttps://sarthak-blog.sarthak-xceed.workers.dev/blog/00-verification-first-harness/https://sarthak-blog.sarthak-xceed.workers.dev/blog/00-verification-first-harness/Many agent loops still treat a model's completion message as terminal, but stopping isn't the same as being correct. On moving the authority for 'done' out of the model and into a verifier, and what that one move makes measurable.Thu, 18 Jun 2026 00:00:00 GMT