AI Coding Tip 041 - Turn Every Review Finding Into a Rule
A human should never address the same issue twice.
I’m a senior software engineer loving clean code, and declarative designs. S.O.L.I.D. and agile methodologies fan.
TL;DR: Turn every human review finding into a harness rule, so the next review starts where this one ended.
Common Mistake ❌
You review the AI's pull request and find a problem.
You fix it, leave a comment, and merge.
Next week the AI (or another human) makes the same mistake, and you write the same comment again.
The comment lived in a closed pull request, and the harness never heard about it.
You're the only memory your harness has.
And you're the most expensive and scarcest part of it.
Problems Addressed 😔
Your most expensive reviewer spends their attention on findings a linter or a hook could catch.
The same finding comes back in the next pull request, because the comment lived in the diff and never reached the harness.
Two reviewers carry two different rule sets in their heads, so they catch different things in the same code without standards.
The AI writes faster than you read, so reviewing every line becomes the bottleneck of the whole pipeline.
A big pull request buries the one decision that needed a human.
The harness stays frozen at the rules you guessed on day one, while the real mistakes of your codebase go unrecorded.
Design decisions made in a review stay undocumented, and the next reviewer has to rediscover them.
You can't tell whether your team improves, because nobody counts what reviews find.
Continuous improvement, the cycle Deming described, never starts when reviews only fix code and never fix the process.
How to Do It 🛠️
Treat every human review comment as a defect in the harness, not only in the code.
Sort the finding by who could have caught it: a linter or hook, a judge skill, a written decision, or only a human.
Turn mechanical findings, like naming and formatting, into a hook or a lint rule.
Turn semantic findings into a rule inside a validator skill that the AI reads fresh every session.
Turn design findings into a written decision, and write down the reason next to it.
Add the rejected code as a failing example in the pitfalls file.
Start a rule suite in the harness, and write every human finding as a failing test case first: the rejected code is the input, and the test fails because nothing detects it yet.
Write the rule that makes that test pass, then run the harness against the rejected diff with a criteria check, and keep the rule only if it blocks that diff.
Ship the test and the rule in the same pull request as the fix, so the lesson merges together with the change.
Send the next diffs through your pipeline first, so you only see what survives it.
Tag every finding as nit, defect, design, or business rule, and count them per review.
Run the contradiction audit every few weeks, since every new rule can disagree with an old one.
Reserve your attention for the decisions the harness can't make: design trade-offs, business rules, and risk.
Treat the whole loop as continuous improvement: measure the findings, change the harness, and measure again.
Benefits 🎯
Protect your scarcest resource: You read what the harness can't judge, and nothing else.
Never pay twice: A finding that became a rule is a finding you never write again.
Grow from real pain: The harness collects the mistakes your codebase actually produces, instead of the ones you imagined on day one.
Review the same way every time: Every diff meets the same rules, no matter which reviewer opens it.
Move up: Your reviewers spend their time on design trade-offs and business rules.
Merge faster: Fewer findings reach you, so the review of a small pull request ends sooner.
Read your maturity: The mix of findings tells you how mature your team is, without a survey.
Onboard people and agents: A new teammate or a new agent inherits every decision a human ever made, like a company brain.
Keep the rules honest: Auditing for hoarded rules removes the ones the model has outgrown.
Context 🧠
You generate code faster than anyone can read it, and you don't scale as a reviewer.
Mäntylä and Lassenius found that 60% to 75% of review defects were about evolvability: naming, structure, and documentation.
Every comment you leave in a pull request is a rule living in the wrong place.
The right place is the harness: a linter, a hook, a skill, or a written decision.
Fool me once, shame on you; fool me twice, shame on me.
Traditional proverb, first printed in 1651 by Anthony Weldon
Standards gates block the mechanical findings, so you climb toward design trade-offs and risk.
This is the loop in loop engineering: the evaluation checks every finding you move into it, no human needed.
The harness grows, and rules start to disagree, so you should audit your contradictions every few weeks.
Prompt Reference 📝
Bad Prompt 🚫
Review this pull request.
Fix every problem you find.
Tell me when it's ready to merge.
Good prompt 👉
You're reviewing this pull request as the harness, not as a person.
Read AGENTS.md, the standards, and the design decisions first.
Review the diff against them.
List every finding in a numbered table.
For each finding, say who should catch it next time.
Pick one: linter, hook, skill, or human.
Then propose the smallest harness change that catches it.
Write the rule, the file where it belongs, and a failing example.
Don't edit any code.
Don't edit any rule.
Ask me to approve each rule, one at a time.
Last, list the findings only a human should decide.
Explain why the harness can't.
A human reviewer found five issues with different severities.
Take the review comments on the pull request.
Write a failing test that represents each finding.
Watch it fail on the rejected code.
Write a rule that fixes the problem.
Add it to the code standards, style, architecture, and design suite.
That way, the finding never reaches a human reviewer again.
Considerations ⚠️
Not every finding deserves a rule.
A one-off judgment call about a strange edge case stays a comment.
A rule written after one comment can be wrong.
You should test it on the rejected diff and on a clean one, so it blocks the first and lets the second pass.
Rules are code too, so they need review like any other change.
You don't leave the review.
You move to the decisions that matter, and that's a different job, not a smaller one.
I don't know yet how many findings per review a healthy team should see.
Zero would mean nobody is looking.
Type 📝
[X] Semi-Automatic
Limitations ⚠️
The first weeks cost more time, because you write the rule on top of the fix.
A judge skill costs tokens on every diff, so you should keep the cheap checks in linters.
A finding that depends on context nobody wrote down can't become a rule until you write that context.
Tags 🏷️
- Knowledge Management
Level 🔋
[X] Intermediate
Related Tips 🔗
https://maximilianocontieri.com/ai-coding-tip-040-make-the-ai-audit-your-own-contradictions
https://maximilianocontieri.com/ai-coding-tip-016-feed-your-pr-lessons-into-the-ai-brain
https://maximilianocontieri.com/ai-coding-tip-027-force-code-standards
https://maximilianocontieri.com/ai-coding-tip-022-give-ai-a-harness-to-work-with
https://maximilianocontieri.com/ai-coding-tip-032-build-a-dark-factory-pipeline
https://maximilianocontieri.com/ai-coding-tip-024-force-a-criteria-check-before-the-task-ends
https://maximilianocontieri.com/ai-coding-tip-006-review-every-line-before-commit
https://maximilianocontieri.com/ai-coding-tip-025-pair-every-skill-with-a-pitfalls-file
https://maximilianocontieri.com/ai-coding-tip-034-stop-hoarding-rules-in-your-agents-md
https://maximilianocontieri.com/ai-coding-tip-023-shrink-your-ai-s-pull-request
https://maximilianocontieri.com/ai-coding-tip-019-tell-the-ai-why-not-just-what
Conclusion 🏁
A review is the cheapest way to learn what your harness is missing.
You pay for it once, with a human hour.
If the finding doesn't become a rule, you pay again next week.
When you turn every finding into a rule, you move up.
More Information ℹ️
Expectations, Outcomes, and Challenges of Modern Code Review, Bacchelli and Bird
Modern Code Review: A Case Study at Google, Sadowski et al.
What Types of Defects Are Really Discovered in Code Reviews?, Mäntylä and Lassenius
Fool Me Once, Shame on You; Fool Me Twice, Shame on Me, Grammarist
Also Known As 🎭
Findings-to-Rules
Review-Fed-Harness
Never-Twice-Review
Tools 🧰
Git hooks and linters hold the mechanical rules.
A validator skill holds the semantic ones.
It lives in an assistant that reads AGENTS.md, like Claude Code.
A decisions file holds the design ones.
Disclaimer 📢
The views expressed here are my own.
I am a human who writes as best as possible for other humans.
I use AI proofreading tools to improve some texts.
Most AI detectors will flag this article as AI-generated. That's expected. It's a technical article. It has a rigid format and clear steps to follow.
That's exactly the pattern those tools are trained to catch. I've apparently been "writing like an AI" for decades, long before AI existed. This is a technical article, not a novel.
I welcome constructive criticism and dialogue.
I shape these insights through 30 years in the software industry, 25 years of teaching, and writing over 500 articles and a book.
Credits 🙏
This article is part of the AI Coding Tip series.
