Using Mutation Testing to Improve a Test Suite

Understand what a surviving mutant means for test quality and how mutation testing results should drive concrete test improvements, with a pointer to the dedicated Mutation Testing lesson for full Infection/operator depth.

Code coverage tells you which lines executed during a test run — it says nothing about whether the test actually checked the result. Mutation testing closes that gap: a tool like Infection deliberately introduces small bugs ("mutants") into your code — flipping a > to >=, changing a + to -, deleting a method call — then reruns your test suite against each mutant. If a test fails, the mutant was "killed" — good, your tests caught the bug. If every test still passes, the mutant "survived" — your tests didn't actually verify that piece of logic, even if coverage says the line ran. This lesson is a short refresher on interpreting mutation results and turning them into better tests; for the full lesson on mutation operators, mutation score, and Infection configuration, see the dedicated "Mutation Testing" lesson elsewhere in this module.

A surviving mutant is a to-do list, not a failure

When Infection reports a surviving mutant, it's pointing at a specific line and a specific change that no test noticed. The fix is almost never to change the production code — it's to look at what assertion is missing. If a mutant changes return $total >= $threshold; to return $total > $threshold; and every test still passes, it means no test exercises the boundary case where $total exactly equals $threshold — a classic off-by-one gap that line coverage would never reveal, because the line ran either way.

Not every surviving mutant is worth chasing. An "equivalent mutant" changes code in a way that can never produce an observably different result (e.g. mutating an unused private constant), and killing it would mean writing a test for something that doesn't matter. Mutation testing tools can't always tell equivalent mutants apart from genuinely weak-tested ones, so triaging survivors is a human judgment call — prioritize the mutants that touch business-critical logic (pricing, permissions, calculations) over incidental code.

• Killed mutant = a test failed when the bug was introduced = good coverage of that logic. • Surviving mutant = no test failed = that logic isn't actually verified, regardless of line coverage. • Equivalent mutants (no observable behavior change) can be safely ignored. • Prioritize surviving mutants in business-critical code over incidental code.

Turning survivors into stronger assertions

The fix for a surviving mutant is usually a missing or weak assertion, not a missing test method. If a test calls a method and only asserts that no exception was thrown, a mutant that changes the return value will survive every time — the test needs to assert on the actual returned value. Boundary mutants (> vs >=, < vs <=) are killed by adding a test case exactly at the boundary; arithmetic mutants (+ vs -) are killed by asserting an exact expected value rather than just "a number was returned".

For the complete treatment — Infection's mutation operators in detail, reading and interpreting the mutation score, tuning min-msi/min-covered-msi thresholds, and wiring mutation testing into CI — see the full "Mutation Testing" lesson in this course.

• A surviving mutant almost always means "add or tighten an assertion", not "change the code". • Test boundary values explicitly to kill > vs >= style mutants. • Assert exact expected values, not just "it returned something", to kill arithmetic mutants. • See the "Mutation Testing" lesson for full operator/mutation-score/CI detail.

Code Examples

The first test only checks that a value came back, so any mutant that changes the discount math without producing null still passes — it survives. The second test asserts the exact expected number and explicitly tests the boundary case, so a mutation to the multiplier or the comparison operator produces a wrong number and the test fails, killing the mutant.

$this->assertSame(135.0, $result);

Common mistake: Asserting only that a result is 'not null' or 'truthy' passes for almost any mutation to the underlying calculation — mutation testing exists precisely to expose this kind of weak assertion.

Reacting to a surviving mutant by changing the production code to make the mutant 'go away'.

Treat a surviving mutant as evidence of a missing or weak assertion, and strengthen the test instead of touching working production code.

The production code that spawned the mutant is usually correct — the mutant is a hypothetical bug variant. The real defect is in the test suite's ability to detect that variant, so the fix belongs in the test, not in already-working logic.

A mutation testing tool changes $total >= $threshold to $total > $threshold and all tests still pass. What does this most likely indicate? A. The production code has a bug that must be fixed immediately B. No test exercises the exact boundary case where $total equals $threshold, so that boundary isn't actually verified C. The mutation testing tool is broken and should be reconfigured D. Line coverage for that file is too low and needs a coverage report

B. No test exercises the exact boundary case where $total equals $threshold, so that boundary isn't actually verified — A surviving boundary mutant is a classic signal that no test case sits exactly at the boundary value, so the >= vs > distinction is never actually exercised by the suite even though the line runs in existing tests.

<?php
declare(strict_types=1);

final class DiscountTest extends \PHPUnit\Framework\TestCase
{
    public function testAppliesDiscountAboveThreshold(): void
    {
        $result = (new Discount())->apply(total: 150.0, threshold: 100.0);

        // Weak: only checks truthiness. A mutant that changes the
        // discount calculation from *0.9 to *0.99 still returns a
        // truthy float and this test still passes — the mutant survives.
        $this->assertNotNull($result);
    }

    public function testAppliesDiscountAboveThresholdStrong(): void
    {
        $result = (new Discount())->apply(total: 150.0, threshold: 100.0);

        // Strong: asserts the exact expected value, killing arithmetic
        // mutants (e.g. *0.9 mutated to *0.99, or - mutated to +).
        $this->assertSame(135.0, $result);

        // Also test the boundary explicitly to kill >= vs > mutants.
        $atThreshold = (new Discount())->apply(total: 100.0, threshold: 100.0);
        $this->assertSame(90.0, $atThreshold);
    }
}