AI Coding Tip 037 - Stop Patching Blind
Old code, new AI, same unwritten test suite.
I’m a senior software engineer loving clean code, and declarative designs. S.O.L.I.D. and agile methodologies fan.
TL;DR: Patch code that never had a test written for it, and every quick fix becomes tomorrow's outage.
Common Mistake ❌
You assume a decades old, stable feature must have some kind of safety net, because it has been shipping without complaints for almost twenty years.
Automated testing wasn't standard practice when much of that code was written.
Nobody ever wrote a test for it, not because a team decided to skip that step.
The step simply didn't exist yet as an expectation.
In September 2026, Microsoft shipped a security update for Excel, documented on its own KB5002914 support page, and paste operations started failing silently across Excel 2016, Office 2019, Office 2021 LTSC, Office 2024 LTSC, and Microsoft 365 Apps.
AutoFill broke too, presumably out of solidarity 🙂.
Microsoft's follow-up fix, KB5002665, later confirmed the trigger in its own words: "If the workbook contains conditional formatting, paste operations might continue to fail."
Users found the pattern before any release note did, flooding Microsoft's own Q&A forum with reports of cells stuck mid-selection, borders that kept blinking, and an Escape key that stopped working, more than a hundred replies deep before a real fix shipped.
Nobody on that patch team sat down and chose to skip testing the conditional formatting code path.
There was no test to run, because there had never been one, and a security hotfix isn't the moment anyone stops to write the first test a decades old module has ever had.
This isn't a Microsoft problem.
It's what happens anywhere a stable product ships a fix into a module nobody has ever verified with an actual test, at the speed an AI can now generate a diff, against code old enough that testing it was never part of anyone's job.
Problems Addressed 😔
- A patch to code with no tests ships on faith, not on evidence, and faith isn't a regression suite.
- AI writes changes far faster than a team can manually verify every code path they touch, which widens the gap between how fast you can edit and how fast you can confirm nothing broke.
- Changes shipped without coverage hide their damage until a user reports it, and users report the damage as lost trust, not as a filed issue.
- A codebase with no seams forces every change, human or AI, to touch the original untested logic directly, so a narrow fix for a security hole can reach into unrelated behavior like clipboard handling.
- Vanity coverage numbers convince a team it's safe to patch fast, while the actual paths that break in production were never exercised by any test, a green dashboard lying straight to your face.
- Rolling back becomes the only realistic fix, because nobody wrote down what the old, untested behavior actually was before the patch replaced it. But you don't want to leave the security hole.
- Every unverified patch that reaches production teaches users the product is less stable than it was yesterday, and that lesson compounds with every release.
How to Do It 🛠️
Treat any code without tests as legacy code, using Michael Feathers' definition from Working Effectively with Legacy Code, no matter how recently it was last edited.
Write a characterization test that records the module's current behavior, including the parts you find ugly or confusing, before the AI changes a single line near it.
Find a seam, a place where you can insert a change without editing the original code path, and route the AI's patch through that seam instead of through the untested core.
Ask the AI to draft the missing tests for the exact module you're about to patch, then review every assertion before you commit to it.
Run the full regression suite and any acceptance tests tied to the feature before and after the patch, and reject any AI-generated fix that doesn't leave every existing test passing.
Scope the change to the smallest edit that closes the actual security hole, and stop the AI from cleaning up nearby legacy code in the same commit.
Don't authorize the AI to change functional behavior while it closes a security hole, and if the fix genuinely can't avoid touching behavior, make it stop and ask a human before it does.
Add every new characterization test to the pipeline, so the same regression under the same conditions never again depends on a user noticing it first.
Ship the patch to a small canary segment before a full rollout, so a blind spot in decades old code surfaces on a fraction of your users instead of on all of them at once.
A canary isn't always possible with a security hole.
When the vulnerability is already public or actively exploited, leaving any segment unpatched just hands the remaining users to whoever is exploiting it, so ship to everyone at once and lean on the characterization tests instead of a staged rollout to catch a regression.
Benefits 🎯
Ship with evidence: A passing characterization suite replaces a shrug and a hope with a specific, checkable claim about what still works.
Shrink the blast radius: A canary rollout turns a legacy blind spot into a contained incident instead of a headline about broken spreadsheets.
Turn legacy code into safe territory: Characterization tests convert unknown behavior into known behavior, so the AI can extend it without guessing.
Stop the firefighting cycle: A tested seam absorbs the next patch too, instead of forcing another blind edit into the same fragile module.
Protect a stable product's reputation: Users forgive a slow fix; they don't forgive a security update that quietly breaks a feature they use every day.
Make review possible: A reviewer can check a diff against a test far faster than they can trace decades old logic by eye.
Context 🧠
Michael Feathers opened Working Effectively with Legacy Code with a blunt definition: legacy code is simply code without tests.
Not old code, not ugly code, code you can't safely change because nothing tells you when you broke it.
Excel's conditional formatting and clipboard logic fits that definition perfectly, and so does most of the code every long-lived product still runs in production.
Age alone doesn't make that inevitable, no matter how much a rushed patch team would like to blame the calendar.
Meanwhile, over at SQLite, somebody has clearly never let a deadline win an argument: running since August 2000, sitting inside an estimated one trillion database files worldwide, and likely the most widely deployed software library on the planet after zlib.
Its own testing page states the project carries roughly 590 times as much test code as product code: fuzzing that runs about a billion mutations a day, out-of-memory and I/O fault injection, crash recovery checks, and a claimed 100% branch and MC/DC coverage on the core engine, verified under Valgrind and multiple sanitizers before every release.
Twenty-five years old and drowning in tests is a choice a team made and kept making.
Twenty-five years old and never tested once is a different choice, made by omission a long time ago and repeated every time nobody goes back to fix it.
AI changes the economics of that second choice in a way Feathers never had to plan for.
Writing a patch used to be the slow part, and testing it thoroughly, even manually, could roughly keep pace.
Now an AI assistant can draft a fix for a fifteen year old code path in the time it takes to read the CVE, and the pace of change outruns the pace of verification unless the test suite is automated and runs on every change.
A characterization test doesn't ask whether the old behavior was correct.
It asks whether the new code still does what the old code did, on purpose or not, so a patch can prove it didn't change anything it wasn't supposed to touch.
That's the seam Feathers describes: a point in the code where you can alter behavior without editing the class or function that currently owns it, which is exactly where an AI-generated fix belongs when the surrounding code has no coverage.
Feathers built his whole book around getting legacy code under test through seams like this one, not around rewriting the module first and hoping the tests catch up later.
A security patch through a seam only touches the one behavior the CVE is about.
Only a passing regression suite in the exit criteria counts, not just code that compiles and looks plausible.
Code standards you force on every change should include this one explicitly for any file that touches a stable, shipped feature.
And when an AI reports that tests pass, verify it the way you'd verify any other AI claim you can't fully see: run the suite yourself, don't take the summary as proof.
The KB5002914 incident didn't need a smarter model to avoid.
It needed a characterization test on the conditional formatting code path, run automatically before the patch shipped to hundreds of millions of installs.
Users get frustrated by new features that don't work, but they never tolerate breaking changes on functionality they've been using for years.
Prompt Reference 📝
Bad Prompt 🚫
Fix the security hole
in the conditional formatting module and ship it today.
This code has never had a test.
It has worked fine for years without one.
Ship it the same way.
Good prompt 👉
Write characterization tests
For the conditional formatting module.
It never had tests.
You must capture its real clipboard and AutoFill behavior.
Show me the failing diff
Only after every existing and new test passes.
I will also validate it manually.
Scope the fix to
The smallest change that closes the security hole.
You aren't authorized to change functional behavior.
If you must, stop and ask me first.
Considerations ⚠️
A characterization test doesn't judge whether old behavior was good design.
It only proves the patch didn't silently change it, which is a different and more urgent question during an emergency fix.
Writing that first test on truly untested legacy code takes real hours you don't have during an active incident, so the honest move is budgeting for it before the next CVE, not during it, while everyone is yelling in the incident channel.
An AI can draft the test scaffolding quickly, but someone still has to confirm the assertions describe the behavior you actually want preserved, not the behavior the AI assumed from the function name.
A security patch authorizes closing the hole, nothing else.
If the AI decides the smallest fix still requires changing what the feature does, that decision belongs to a human, not to whichever model happened to draft the diff.
A canary rollout only limits damage if someone is watching it, so pair it with the same kind of monitoring you'd want on any privileged system change.
Type 📝
[X] Semi-Automatic
Limitations ⚠️
Characterization tests describe current behavior, not correct behavior, so a defect baked into the legacy code gets preserved right alongside everything that works, immortalized by the very tests meant to protect you.
Retrofitting tests onto a codebase with none takes longer than the emergency patch itself, which is exactly why teams skip it under deadline pressure, and exactly why the skipping keeps happening, generation after generation of engineers inheriting the same untested pile and pretending it's fine.
Tags 🏷️
- Safety
Level 🔋
[X] Intermediate
Related Tips 🔗
https://maximilianocontieri.com/ai-coding-tip-006-review-every-line-before-commit
https://maximilianocontieri.com/ai-coding-tip-015-force-the-ai-to-obey-you
https://maximilianocontieri.com/ai-coding-tip-021-avoid-comprehension-debt
https://maximilianocontieri.com/ai-coding-tip-022-give-ai-a-harness-to-work-with
https://maximilianocontieri.com/ai-coding-tip-023-shrink-your-ai-s-pull-request
https://maximilianocontieri.com/ai-coding-tip-024-force-a-criteria-check-before-the-task-ends
https://maximilianocontieri.com/ai-coding-tip-027-force-code-standards
https://maximilianocontieri.com/ai-coding-tip-032-build-a-dark-factory-pipeline
https://maximilianocontieri.com/ai-coding-tip-033-protect-yourself-against-ai-cheating
Conclusion 🏁
Every stable product is one untested module away from becoming a headline about broken copy and paste.
AI didn't create that risk, but it lets a team reach it faster than ever, one confidently drafted patch at a time, cheerfully unaware it just walked into a minefield nobody mapped.
Write the characterization test first, find the seam, and let the AI work inside walls a human actually verified.
A quick fix that breaks a feature everyone relies on isn't quick.
It's just a slower disaster with better timing.
More Information ℹ️
KB5002914 Support Page, Microsoft
KB5002665 Support Page, Microsoft
We Are Facing Issue in Excel 2026, 2019 and 2021, Microsoft Q&A
Microsoft Office September 8, 2026 Update Causes Excel Copy Paste Issue, Born City
Most Widely Deployed and Used Database Engine, SQLite
Characterization Test, Wikipedia
Also Known As 🎭
- Untested-Legacy-Patching
- Characterization-Test-First
- Seam-Driven-AI-Fixes
- Blind-Patch-Syndrome
Tools 🧰
Any unit testing framework already in your stack, a feature flag or canary deployment tool, and a coverage reporter wired into continuous integration so a missing test on a changed file blocks the merge.
Disclaimer 📢
The views expressed here are my own.
I am a human who writes as best as possible for other humans.
I use AI proofreading tools to improve some texts.
Most AI detectors will flag this article as AI-generated. That's expected. It's a technical article. It has a rigid format and clear steps to follow.
That's exactly the pattern those tools are trained to catch. I've apparently been "writing like an AI" for decades, long before AI existed. This is a technical article, not a novel.
I welcome constructive criticism and dialogue.
I shape these insights through 30 years in the software industry, 25 years of teaching, and writing over 500 articles and a book.
This article is part of the AI Coding Tip series.
