# Code Smell 320 - Vanity Coverage

> TL;DR: You write tests that touch every line but verify nothing, creating false confidence in a broken system.

# Problems 😔

*   False confidence
    
*   Hidden production [defects](https://maximilianocontieri.com/stop-calling-them-bugs)
    
*   Misleading metrics
    
*   Wasted test effort
    
*   Untested edge cases
    

# Solutions 😃

1.  Use mutation testing
    
2.  Test real behaviors
    
3.  Write assertive tests
    
4.  Delete coverage-only tests
    

# Refactorings ⚙️

%[https://maximilianocontieri.com/refactoring-011-replace-comments-with-tests] 

# Context 💬

Many teams set a coverage threshold: 80%, 90%, or even 100%.

When you chase that number, you write tests that call methods without checking the actual results.

A test that calls `calculateTax()` but only asserts `result is not None` executes the line.

It doesn't verify the tax calculation is correct.

The dashboard turns green.

Defects survive in production.

This is vanity coverage: cosmetic metrics that hide real problems. Like brushes that make surfaces look smooth while the rot stays underneath.

Mutation testing reveals the truth.

When you mutate production code and no tests fail, your coverage numbers lied.

# Sample Code 💻

## Wrong 🚫

```javascript
describe('BankAccount', () => {
  test('deposit', () => {
    const account = new BankAccount(100);
    account.deposit(50);
    // Only checking it didn't crash
    expect(account).toBeDefined();
  });

  test('withdraw', () => {
    const account = new BankAccount(100);
    const result = account.withdraw(30);
    // No assertion about the result!
  });

  test('transfer', () => {
    const source = new BankAccount(200);
    const target = new BankAccount(0);
    // Just calling the method to "cover" the line
    source.transfer(50, target);
  });
});
```

## Right 👉

```javascript
describe('BankAccount', () => {
  test('deposit increases balance', () => {
    const account = new BankAccount(100);
    account.deposit(50);
    expect(account.balance()).toBe(150);
  });

  test('withdraw decreases balance', () => {
    const account = new BankAccount(100);
    account.withdraw(30);
    expect(account.balance()).toBe(70);
  });

  test('withdraw raises on insufficient funds', () => {
    const account = new BankAccount(50);
    expect(() => account.withdraw(100))
      .toThrow(InsufficientFundsError);
  });

  test('transfer moves money between accounts', () => {
    const source = new BankAccount(200);
    const target = new BankAccount(0);
    source.transfer(50, target);
    expect(source.balance()).toBe(150);
    expect(target.balance()).toBe(50);
  });
});
```

# Detection 🔍

\[X\] Semi-Automatic

Run a mutation testing tool ([PIT](https://pitest.org/) for Java, [Stryker](https://stryker-mutator.io/) for JavaScript, [mutmut](https://github.com/boxed/mutmut) for Python).

Count the surviving mutants.

If coverage is high but mutants survive, you have vanity coverage.

You can also search for assertion-free tests, single `assertNotNull()` assertions, or tests that still pass after you delete the entire production method body.

# Exceptions 🛑

Smoke tests that call endpoints to verify the system starts are acceptable without detailed assertions.

These work when they complement a real test suite, not replace it.

# Tags 🏷️

*   Testing
    

# Level 🔋

\[x\] Intermediate

# Why the Bijection Is Important 🗺️

Your test suite must map each test to a real behavior in the [MAPPER](https://maximilianocontieri.com/what-is-wrong-with-software).

When a test covers a line without verifying observable behavior, you break that [bijection](https://maximilianocontieri.com/the-one-and-only-software-design-principle).

Coverage tools only measure "lines executed."

Chase the number without [bijection](https://maximilianocontieri.com/the-one-and-only-software-design-principle) and your suite looks complete while missing real requirements entirely.

# AI Generation 🤖

AI code generators sometimes produce vanity coverage.

Ask one to "add tests to reach 80% coverage," and it writes tests that call methods and assert trivially true facts.

The metric goes up.

Nothing gets verified.

# AI Detection 🧲

AI can detect vanity coverage, but only if you ask the right questions.

Try: "Find tests with no real assertions" or "Find tests that pass when I delete the production method body."

Without those prompts, most AI tools see a green test and call it good.

## Try Them! 🛠

*Remember: AI Assistants make lots of mistakes*

> Suggested Prompt: Replace vanity coverage tests with tests that verify real behaviors and fail when production code is wrong

| Without Proper Instructions | With Specific Instructions |
| --- | --- |
| [ChatGPT](https://chat.openai.com/?q=Rewrite+these+tests+with+meaningful+behavioral+assertions%3A+%60%60%60javascript%0D%0Adescribe%28%27BankAccount%27%2C+%28%29+%3D%3E+%7B%0D%0A++test%28%27deposit%27%2C+%28%29+%3D%3E+%7B%0D%0A++++const+account+%3D+new+BankAccount%28100%29%3B%0D%0A++++account.deposit%2850%29%3B%0D%0A++++%2F%2F+Only+checking+it+didn%27t+crash%0D%0A++++expect%28account%29.toBeDefined%28%29%3B%0D%0A++%7D%29%3B%0D%0A%0D%0A++test%28%27withdraw%27%2C+%28%29+%3D%3E+%7B%0D%0A++++const+account+%3D+new+BankAccount%28100%29%3B%0D%0A++++const+result+%3D+account.withdraw%2830%29%3B%0D%0A++++%2F%2F+No+assertion+about+the+result%21%0D%0A++%7D%29%3B%0D%0A%0D%0A++test%28%27transfer%27%2C+%28%29+%3D%3E+%7B%0D%0A++++const+source+%3D+new+BankAccount%28200%29%3B%0D%0A++++const+target+%3D+new+BankAccount%280%29%3B%0D%0A++++%2F%2F+Just+calling+the+method+to+%22cover%22+the+line%0D%0A++++source.transfer%2850%2C+target%29%3B%0D%0A++%7D%29%3B%0D%0A%7D%29%3B%0D%0A%60%60%60) | [ChatGPT](https://chat.openai.com/?q=Replace+vanity+coverage+tests+with+tests+that+verify+real+behaviors+and+fail+when+production+code+is+wrong%3A+%60%60%60javascript%0D%0Adescribe%28%27BankAccount%27%2C+%28%29+%3D%3E+%7B%0D%0A++test%28%27deposit%27%2C+%28%29+%3D%3E+%7B%0D%0A++++const+account+%3D+new+BankAccount%28100%29%3B%0D%0A++++account.deposit%2850%29%3B%0D%0A++++%2F%2F+Only+checking+it+didn%27t+crash%0D%0A++++expect%28account%29.toBeDefined%28%29%3B%0D%0A++%7D%29%3B%0D%0A%0D%0A++test%28%27withdraw%27%2C+%28%29+%3D%3E+%7B%0D%0A++++const+account+%3D+new+BankAccount%28100%29%3B%0D%0A++++const+result+%3D+account.withdraw%2830%29%3B%0D%0A++++%2F%2F+No+assertion+about+the+result%21%0D%0A++%7D%29%3B%0D%0A%0D%0A++test%28%27transfer%27%2C+%28%29+%3D%3E+%7B%0D%0A++++const+source+%3D+new+BankAccount%28200%29%3B%0D%0A++++const+target+%3D+new+BankAccount%280%29%3B%0D%0A++++%2F%2F+Just+calling+the+method+to+%22cover%22+the+line%0D%0A++++source.transfer%2850%2C+target%29%3B%0D%0A++%7D%29%3B%0D%0A%7D%29%3B%0D%0A%60%60%60) |
| [Claude](https://claude.ai/new?q=Rewrite+these+tests+with+meaningful+behavioral+assertions%3A+%60%60%60javascript%0D%0Adescribe%28%27BankAccount%27%2C+%28%29+%3D%3E+%7B%0D%0A++test%28%27deposit%27%2C+%28%29+%3D%3E+%7B%0D%0A++++const+account+%3D+new+BankAccount%28100%29%3B%0D%0A++++account.deposit%2850%29%3B%0D%0A++++%2F%2F+Only+checking+it+didn%27t+crash%0D%0A++++expect%28account%29.toBeDefined%28%29%3B%0D%0A++%7D%29%3B%0D%0A%0D%0A++test%28%27withdraw%27%2C+%28%29+%3D%3E+%7B%0D%0A++++const+account+%3D+new+BankAccount%28100%29%3B%0D%0A++++const+result+%3D+account.withdraw%2830%29%3B%0D%0A++++%2F%2F+No+assertion+about+the+result%21%0D%0A++%7D%29%3B%0D%0A%0D%0A++test%28%27transfer%27%2C+%28%29+%3D%3E+%7B%0D%0A++++const+source+%3D+new+BankAccount%28200%29%3B%0D%0A++++const+target+%3D+new+BankAccount%280%29%3B%0D%0A++++%2F%2F+Just+calling+the+method+to+%22cover%22+the+line%0D%0A++++source.transfer%2850%2C+target%29%3B%0D%0A++%7D%29%3B%0D%0A%7D%29%3B%0D%0A%60%60%60) | [Claude](https://claude.ai/new?q=Replace+vanity+coverage+tests+with+tests+that+verify+real+behaviors+and+fail+when+production+code+is+wrong%3A+%60%60%60javascript%0D%0Adescribe%28%27BankAccount%27%2C+%28%29+%3D%3E+%7B%0D%0A++test%28%27deposit%27%2C+%28%29+%3D%3E+%7B%0D%0A++++const+account+%3D+new+BankAccount%28100%29%3B%0D%0A++++account.deposit%2850%29%3B%0D%0A++++%2F%2F+Only+checking+it+didn%27t+crash%0D%0A++++expect%28account%29.toBeDefined%28%29%3B%0D%0A++%7D%29%3B%0D%0A%0D%0A++test%28%27withdraw%27%2C+%28%29+%3D%3E+%7B%0D%0A++++const+account+%3D+new+BankAccount%28100%29%3B%0D%0A++++const+result+%3D+account.withdraw%2830%29%3B%0D%0A++++%2F%2F+No+assertion+about+the+result%21%0D%0A++%7D%29%3B%0D%0A%0D%0A++test%28%27transfer%27%2C+%28%29+%3D%3E+%7B%0D%0A++++const+source+%3D+new+BankAccount%28200%29%3B%0D%0A++++const+target+%3D+new+BankAccount%280%29%3B%0D%0A++++%2F%2F+Just+calling+the+method+to+%22cover%22+the+line%0D%0A++++source.transfer%2850%2C+target%29%3B%0D%0A++%7D%29%3B%0D%0A%7D%29%3B%0D%0A%60%60%60) |
| [Perplexity](https://www.perplexity.ai/?q=Rewrite+these+tests+with+meaningful+behavioral+assertions%3A+%60%60%60javascript%0D%0Adescribe%28%27BankAccount%27%2C+%28%29+%3D%3E+%7B%0D%0A++test%28%27deposit%27%2C+%28%29+%3D%3E+%7B%0D%0A++++const+account+%3D+new+BankAccount%28100%29%3B%0D%0A++++account.deposit%2850%29%3B%0D%0A++++%2F%2F+Only+checking+it+didn%27t+crash%0D%0A++++expect%28account%29.toBeDefined%28%29%3B%0D%0A++%7D%29%3B%0D%0A%0D%0A++test%28%27withdraw%27%2C+%28%29+%3D%3E+%7B%0D%0A++++const+account+%3D+new+BankAccount%28100%29%3B%0D%0A++++const+result+%3D+account.withdraw%2830%29%3B%0D%0A++++%2F%2F+No+assertion+about+the+result%21%0D%0A++%7D%29%3B%0D%0A%0D%0A++test%28%27transfer%27%2C+%28%29+%3D%3E+%7B%0D%0A++++const+source+%3D+new+BankAccount%28200%29%3B%0D%0A++++const+target+%3D+new+BankAccount%280%29%3B%0D%0A++++%2F%2F+Just+calling+the+method+to+%22cover%22+the+line%0D%0A++++source.transfer%2850%2C+target%29%3B%0D%0A++%7D%29%3B%0D%0A%7D%29%3B%0D%0A%60%60%60) | [Perplexity](https://www.perplexity.ai/?q=Replace+vanity+coverage+tests+with+tests+that+verify+real+behaviors+and+fail+when+production+code+is+wrong%3A+%60%60%60javascript%0D%0Adescribe%28%27BankAccount%27%2C+%28%29+%3D%3E+%7B%0D%0A++test%28%27deposit%27%2C+%28%29+%3D%3E+%7B%0D%0A++++const+account+%3D+new+BankAccount%28100%29%3B%0D%0A++++account.deposit%2850%29%3B%0D%0A++++%2F%2F+Only+checking+it+didn%27t+crash%0D%0A++++expect%28account%29.toBeDefined%28%29%3B%0D%0A++%7D%29%3B%0D%0A%0D%0A++test%28%27withdraw%27%2C+%28%29+%3D%3E+%7B%0D%0A++++const+account+%3D+new+BankAccount%28100%29%3B%0D%0A++++const+result+%3D+account.withdraw%2830%29%3B%0D%0A++++%2F%2F+No+assertion+about+the+result%21%0D%0A++%7D%29%3B%0D%0A%0D%0A++test%28%27transfer%27%2C+%28%29+%3D%3E+%7B%0D%0A++++const+source+%3D+new+BankAccount%28200%29%3B%0D%0A++++const+target+%3D+new+BankAccount%280%29%3B%0D%0A++++%2F%2F+Just+calling+the+method+to+%22cover%22+the+line%0D%0A++++source.transfer%2850%2C+target%29%3B%0D%0A++%7D%29%3B%0D%0A%7D%29%3B%0D%0A%60%60%60) |
| [Copilot](https://www.bing.com/chat?showconv=1&sendquery=1&q=Rewrite+these+tests+with+meaningful+behavioral+assertions%3A+%60%60%60javascript%0D%0Adescribe%28%27BankAccount%27%2C+%28%29+%3D%3E+%7B%0D%0A++test%28%27deposit%27%2C+%28%29+%3D%3E+%7B%0D%0A++++const+account+%3D+new+BankAccount%28100%29%3B%0D%0A++++account.deposit%2850%29%3B%0D%0A++++%2F%2F+Only+checking+it+didn%27t+crash%0D%0A++++expect%28account%29.toBeDefined%28%29%3B%0D%0A++%7D%29%3B%0D%0A%0D%0A++test%28%27withdraw%27%2C+%28%29+%3D%3E+%7B%0D%0A++++const+account+%3D+new+BankAccount%28100%29%3B%0D%0A++++const+result+%3D+account.withdraw%2830%29%3B%0D%0A++++%2F%2F+No+assertion+about+the+result%21%0D%0A++%7D%29%3B%0D%0A%0D%0A++test%28%27transfer%27%2C+%28%29+%3D%3E+%7B%0D%0A++++const+source+%3D+new+BankAccount%28200%29%3B%0D%0A++++const+target+%3D+new+BankAccount%280%29%3B%0D%0A++++%2F%2F+Just+calling+the+method+to+%22cover%22+the+line%0D%0A++++source.transfer%2850%2C+target%29%3B%0D%0A++%7D%29%3B%0D%0A%7D%29%3B%0D%0A%60%60%60) | [Copilot](https://www.bing.com/chat?showconv=1&sendquery=1&q=Replace+vanity+coverage+tests+with+tests+that+verify+real+behaviors+and+fail+when+production+code+is+wrong%3A+%60%60%60javascript%0D%0Adescribe%28%27BankAccount%27%2C+%28%29+%3D%3E+%7B%0D%0A++test%28%27deposit%27%2C+%28%29+%3D%3E+%7B%0D%0A++++const+account+%3D+new+BankAccount%28100%29%3B%0D%0A++++account.deposit%2850%29%3B%0D%0A++++%2F%2F+Only+checking+it+didn%27t+crash%0D%0A++++expect%28account%29.toBeDefined%28%29%3B%0D%0A++%7D%29%3B%0D%0A%0D%0A++test%28%27withdraw%27%2C+%28%29+%3D%3E+%7B%0D%0A++++const+account+%3D+new+BankAccount%28100%29%3B%0D%0A++++const+result+%3D+account.withdraw%2830%29%3B%0D%0A++++%2F%2F+No+assertion+about+the+result%21%0D%0A++%7D%29%3B%0D%0A%0D%0A++test%28%27transfer%27%2C+%28%29+%3D%3E+%7B%0D%0A++++const+source+%3D+new+BankAccount%28200%29%3B%0D%0A++++const+target+%3D+new+BankAccount%280%29%3B%0D%0A++++%2F%2F+Just+calling+the+method+to+%22cover%22+the+line%0D%0A++++source.transfer%2850%2C+target%29%3B%0D%0A++%7D%29%3B%0D%0A%7D%29%3B%0D%0A%60%60%60) |
| [You](https://you.com/search?q=Rewrite+these+tests+with+meaningful+behavioral+assertions%3A+%60%60%60javascript%0D%0Adescribe%28%27BankAccount%27%2C+%28%29+%3D%3E+%7B%0D%0A++test%28%27deposit%27%2C+%28%29+%3D%3E+%7B%0D%0A++++const+account+%3D+new+BankAccount%28100%29%3B%0D%0A++++account.deposit%2850%29%3B%0D%0A++++%2F%2F+Only+checking+it+didn%27t+crash%0D%0A++++expect%28account%29.toBeDefined%28%29%3B%0D%0A++%7D%29%3B%0D%0A%0D%0A++test%28%27withdraw%27%2C+%28%29+%3D%3E+%7B%0D%0A++++const+account+%3D+new+BankAccount%28100%29%3B%0D%0A++++const+result+%3D+account.withdraw%2830%29%3B%0D%0A++++%2F%2F+No+assertion+about+the+result%21%0D%0A++%7D%29%3B%0D%0A%0D%0A++test%28%27transfer%27%2C+%28%29+%3D%3E+%7B%0D%0A++++const+source+%3D+new+BankAccount%28200%29%3B%0D%0A++++const+target+%3D+new+BankAccount%280%29%3B%0D%0A++++%2F%2F+Just+calling+the+method+to+%22cover%22+the+line%0D%0A++++source.transfer%2850%2C+target%29%3B%0D%0A++%7D%29%3B%0D%0A%7D%29%3B%0D%0A%60%60%60) | [You](https://you.com/search?q=Replace+vanity+coverage+tests+with+tests+that+verify+real+behaviors+and+fail+when+production+code+is+wrong%3A+%60%60%60javascript%0D%0Adescribe%28%27BankAccount%27%2C+%28%29+%3D%3E+%7B%0D%0A++test%28%27deposit%27%2C+%28%29+%3D%3E+%7B%0D%0A++++const+account+%3D+new+BankAccount%28100%29%3B%0D%0A++++account.deposit%2850%29%3B%0D%0A++++%2F%2F+Only+checking+it+didn%27t+crash%0D%0A++++expect%28account%29.toBeDefined%28%29%3B%0D%0A++%7D%29%3B%0D%0A%0D%0A++test%28%27withdraw%27%2C+%28%29+%3D%3E+%7B%0D%0A++++const+account+%3D+new+BankAccount%28100%29%3B%0D%0A++++const+result+%3D+account.withdraw%2830%29%3B%0D%0A++++%2F%2F+No+assertion+about+the+result%21%0D%0A++%7D%29%3B%0D%0A%0D%0A++test%28%27transfer%27%2C+%28%29+%3D%3E+%7B%0D%0A++++const+source+%3D+new+BankAccount%28200%29%3B%0D%0A++++const+target+%3D+new+BankAccount%280%29%3B%0D%0A++++%2F%2F+Just+calling+the+method+to+%22cover%22+the+line%0D%0A++++source.transfer%2850%2C+target%29%3B%0D%0A++%7D%29%3B%0D%0A%7D%29%3B%0D%0A%60%60%60) |
| [Gemini](https://gemini.google.com/) | [Gemini](https://gemini.google.com/) |
| [DeepSeek](https://chat.deepseek.com/) | [DeepSeek](https://chat.deepseek.com/) |
| [Meta AI](https://www.meta.ai/chat) | [Meta AI](https://www.meta.ai/) |
| [Grok](https://grok.com/) | [Grok](https://grok.com/) |
| [Qwen](https://chat.qwen.ai/) | [Qwen](https://chat.qwen.ai/) |

# Conclusion 🏁

Coverage is a signal, not a goal.

When you treat it as a goal, you create vanity coverage that hides real [defects](https://maximilianocontieri.com/stop-calling-them-bugs).

Use mutation testing to discover what your suite actually verifies.

Write tests that describe real behaviors, not tests that execute lines.

# Relations 👩‍❤️‍💋‍👨

%[https://maximilianocontieri.com/code-smell-104-assert-true] 

%[https://maximilianocontieri.com/code-smell-76-generic-assertions] 

%[https://maximilianocontieri.com/code-smell-175-changes-without-coverage] 

%[https://maximilianocontieri.com/code-smell-30-mocking-business] 

%[https://maximilianocontieri.com/code-smell-203-irrelevant-test-information] 

# More Information 📕

[Mutation Testing](https://en.wikipedia.org/wiki/Mutation_testing)

[Test Coverage Is Not Enough](https://martinfowler.com/bliki/TestCoverage.html)

[Stryker Mutation Testing](https://stryker-mutator.io/)

# Quote

> The most dangerous kind of waste is the waste we don't recognize.

*Shigeo Shingo*

# Disclaimer 📘

Code Smells are my [opinion](https://maximilianocontieri.com/i-wrote-more-than-90-articles-on-2021-here-is-what-i-learned).

# Credits 🙏

Photo by [Jamie Street](https://unsplash.com/es/@jamie452) on [Unsplash](https://unsplash.com/es/fotos/foto-macro-de-tres-brochas-de-maquillaje-marrones-JBQdeLezIxQ)

* * *

This article is part of the CodeSmell Series.

%[https://maximilianocontieri.com/how-to-find-the-stinky-parts-of-your-code]