Mutation testing

Mutation testing is a way to measure how well a test suite catches bugs. A tool makes many small changes to the code, called mutants, and runs the tests against each one. A good suite fails for most of them.

What it is

In the words of the Stryker docs: bugs, or mutants, are inserted into your code automatically. If your tests fail, the mutant is killed. If they pass, the mutant survived, which points to a gap in the tests. The higher the share of mutants killed, the more effective the tests.

Typical mutations flip a comparison (< to <=), swap true and false, remove a function call, or replace a return value. Stryker covers JavaScript and TypeScript, C# and Scala; other ecosystems have their own tools.

Mutation testing is slow, because it runs the suite once per mutant, so teams often run it on changed files only or on a schedule.

Why it matters

Coverage tells you which code ran during the tests. Mutation testing tells you whether the tests would notice that code being wrong. The difference matters with AI-written tests, which can run every line and still check very little.

In AI CodeOps

AI CodeOps doesn't run mutation testing. Its release gate uses your test results and a coverage floor. If you run a mutation tool in CI, treat its score as a separate check.