skill_issue

AI coding agents: write principles, not prompts (AGENTS.md template)

Issue #1 · · by skill issue

a principles file you can paste tonight, why the harness beats the model, and 4 checks before you call it done

TL;DR (20 Oct 2026): write your agent's rules down in AGENTS.md, treat the harness as the part you own, and require 4 checks before "done". One preprint (arXiv 2610.07832, not yet replicated) reports the same model going from 6.5% to 31.0% on 20 repo-migration tasks after a harness swap.

thats one preprint, on 20 whole-repository migration tasks, and nobody has independently replicated it yet. (the paper calls its harness HERMES, its not my hermes agent) but it points where i think agent work is going: when your agent fails, the model often isnt the only problem, and its the part you control least. the rules it runs under and the checks around it are the part you own (my take)

so this issue gives you three things to copy today:

  1. a principles block your agent reads before every task
  2. why the code around the model is the part worth owning
  3. a 4-check list for the gap between "tests passed" and "works"

1. write your agent's rules down: a principles file for AGENTS.md

you know the loop. the agent overbuilds, you switch models, the new one overbuilds differently

my take: the model isnt ignoring you. you never wrote down the rules that matter, so it falls back to its defaults. in my experience those defaults lean toward more code, more abstraction, more "just in case"

its not only me saying this. Anthropic's write-up on rebuilding its source control says its engineers very often found their role was to tell Claude "no". agents would sometimes violate core system constraints to satisfy a pattern that should never have been allowed, and adversarial review could push Claude toward over-engineering against hypothetical concerns. thats a lab with access to its own frontier models, still steering with constraints

Coinbase's takeaway from post-training its own fraud model points the same way. one of its three disciplines: version the whole agent, meaning model weights, prompts and decision policy together. the prompt is part of the system, not a vibe you retype every session

so stop retyping vibes. write the principles once, put them where the agent loads them every session, and make every decision answer to them

paste this into AGENTS.md at your repo root (or CLAUDE.md, if thats the only file your tool reads):

## principles (apply to every decision)
1. smallest change that solves the stated problem. no new abstractions, layers or config unless asked
2. read before you write. check that a file, function or api exists before you use it
3. a failed read is unknown, not empty. report the failure, never act as if nothing is there
4. stay in scope. if the fix needs more than was asked, stop and propose it first
5. follow this repo's existing patterns over your own defaults
6. never weaken a test, check or guard to make something pass
7. "done" needs evidence: say what you ran, what it showed and what you didnt check

before and after, same task (an illustration, not a benchmark):

before: add retry logic to the fetch client

after:  add retry logic to the fetch client. principles in AGENTS.md apply.
        smallest diff. tell me which check proves it works and what you skipped

the after version costs you one line, and the principles file loads every session, on every model you point at it

the honest caveat: a February preprint, "Evaluating AGENTS.md", found that context files "do not generally improve task success rates, while increasing inference cost by over 20%." the same preprint says agents follow the instructions in those files well, and that repository overviews are what didnt help. its conclusion: context files are useful for specifying non-standard coding practices, and anything you add for performance should be tested. so keep the file to rules, not a tour of your repo, and check it earns its tokens on your own tasks

more: context engineering · prompting · patterns

2. why does the agent harness matter more than the model?

the last two weeks of outside evidence point at the code around the model:

heads up on the name: the preprint's HERMES is a research harness from university authors, not the open-source Hermes agent. what it shows is the general point: same weights, different code around them, very different results

thats why i treat the harness as the thing worth investing in, and the model as fuel you can swap (my take)

one practical check before you paste anything: find out which rules file your tool actually loads. tools differ, and some load only one project file per session. Claude Code for example, per its changelog, since mid-September (v2.1.277) reads AGENTS.md when a project has no CLAUDE.md. if both exist, CLAUDE.md wins (switchable under "Project instructions" in /config). one AGENTS.md can feed several tools, as long as you dont also keep a CLAUDE.md that says something different

one more from Meta Superintelligence Labs and co-authors, who studied agents that build their own persistent notes, skills and tools. they found that "the agent that ultimately performs best need not be the one that improves most efficiently." and in one GPT-5.5 run they describe, the agent's own reflection step kept telling it to load its saved move-picking program right away, and it still spent its first steps listing and reading files

writing a rule down isnt the same as following it. thats what the next section is for

3. why "tests passed" doesn't mean "works"

green tests prove the script behaves. they dont prove it works for whoever actually uses it

the research backs a narrower point. a February preprint on SWE-bench Verified found that pushing agents to write more or fewer of their own tests barely moved resolve rates: current agent-written testing "reshape[s] process and cost more than final task outcomes". and in the HERMES preprint, the diagnosis step isnt allowed to mark a task passed while tests or the build visibly fail. in the worked example they walk through, the authors say a pass meant the patch was consistent with the issue with no observed regression, not that the behaviour was certified

paste this under the principles in your AGENTS.md:

## verification (before you say done)
1. size the check to the risk. run the check that would actually fail if this change is wrong. name what you skipped and why
2. review the exact candidate. any edit after a review voids it. re-review what changed
3. change files safely. back up the exact version you read before overwriting it. if the read failed, stop. after writing, read the target back
4. qualify at runtime. run the real path the user or downstream system uses. if that runtime isnt available, report a blocker. never simulate a pass

check 1 keeps you out of test theatre. check 4 is the one that catches "it said done" when it isnt

see evaluation

who holds the power here

Anthropic writes that it went from about 2,500 merged code changes a day in May to over 30,000 a day in September. it built its own version control system and moved its monorepo off git in five months. those are the company's own figures

thats what owning every layer looks like: when the agents outgrow the tooling, the lab rebuilds the tooling

you probably arent rebuilding git. you dont need to

my take: what decides whether an agent is useful is moving out of the weights and into the files and checks around them. the weights you mostly rent. the principles, the checks and a harness that lets you swap the model underneath are yours. Coinbase showed one version of that with an open 9B model and its own data. the principles block and the 4 checks are the version you can run tonight

keep your rules in your repo and run a harness that takes any model, and switching providers becomes a one-command model switch instead of a rebuild (my take)

one ask

if you run agents and something keeps breaking, DM me on X (the "DM on X" link at the bottom of this page) with your setup: harness, model, and what your rules file says. ill reply with where id look first

sources

All editions

Plain-text Markdown copy