engineering2 min de lecture
Friday Five #5: Small enough to check
How I hand work to coding agents: a five-field envelope, an eight-field report, and a small program that checks both.
Rob Vasquez·
Five notes from the week. Short on purpose.
- Sent: Every handoff I give a coding agent starts as an envelope. It has 5 fields: goal, task, scope, done-when and constraints. Scope is a list of paths. Not a paragraph. Done-when is something I can observe, like "the formatter tests pass". The agent hands back a report with 8 fields, where status is complete, partial or blocked and confidence is high, medium or low, and I don't accept anything outside those words.
- Checked: A small program reads both blocks and prints JSON. A report marked complete that also lists blockers fails. So does a duplicate field, an absolute path or a wildcard in the scope. No partial credit. It doesn't guess, it rejects the whole document, and anything over 64 KiB is out. I didn't want a checker that fills in the blanks. The checks and demos are in a public repo, and they don't need any accounts: https://github.com/robvasquez/agent-harness
- Limited: Strict, and still not enough. It can't prove the tests ran. A plausible false report can pass, and I don't trust one just because it's tidy. So I read the diff myself, and a separate review pass checks the work. In September that pass caught a save that waited on cache invalidation while every test was green, and one regression test now holds invalidation open.
- Gated: I've got a gate for outbound text in the same repo. It blocks a commit message or pull request body with a local path or an internal marker, and it tells me why. `make demo` blocks a sample body, then accepts the fixed one. Anyone can skip it. The README says so.
- Asking: How do you check what an agent tells you it did? Do you read the diff, the report, or both?