mutation-adequacy

Mutation Adequacy Skill

Overview

The mutation-adequacy review dimension is the adequacy backstop for the relaxed verification mix (verification ladder slice 3, R5). When the planner ships the cheaper testing strategy — strict types + inline invariants + one PBT + one acceptance test instead of granular per-behavior red-green — mutation adequacy is the machine guard against the vacuous tests (and vacuous PBT properties) that omission could otherwise admit. It scores whether the test suite actually kills injected mutants: the strongest signal that tests fail for the right reason.

This dimension gates the HIGH risk tier only, at the /review boundary only — it is the review-phase counterpart to the delegation-time check_test_adequacy (R3) gate. The coupling to the high tier is policy data in review-contract.ts (the dimension name = this skill's folder name); you never decide tier membership in this skill. The gate is a required review dimension of the high tier — a structural participation fact, distinct from severity (below). The action table registers it as [MUTATION_GATE_NAME]: adaptLadderGate(handleMutationAdequacy), and the severity wrapper withGateSeverity reads its gate name and dimension from the gate metadata of its registration.

Two orthogonal axes, two default severities. The mutation score stays advisory by default: a sub-threshold mutationScore surfaces survivor follow-ups and warns, but does not block, unless an explicit review.gates['mutation-adequacy'] config override raises its severity. A 100% score is neither expected nor required (equivalent mutants exist).

The NoCoverage axis is a second, independent check DR-6 added, with its own real, functioning enforcement path: under review.mutationEnforcement: block, the block-mode guard (workflow/guards.ts Check 4b) fails the review → synthesize transition whenever the diff-scoped noCoverage count exceeds the configured maxNoCoverage budget (default 0 — zero uncovered changed lines tolerated), independent of the score. This is live code, not a deferred plan — but it is opt-in, the same way the score's severity is: review.mutationEnforcement defaults to advisory, and mutation-adequacy's per-gate severity default is deliberately advisory too (so a project's dimension-level defaults never silently re-block a demoted gate). A project flips NoCoverage enforcement on with one config line (review.mutation-enforcement: block); honor whichever way it's resolved, never hardcode "advisory" when recording this dimension's result.

The CI side is separate and narrower: a diff-scoped mutation gate runs on pull-request events, but it runs in observe mode unconditionally — it logs its verdict and always exits 0, regardless of any .exarchos.yml setting. It is not yet a blocking CI backstop; the flip is deferred, pending a clean runner verdict on the full server suite. Never represent the CI gate as a merge blocker — where NoCoverage enforcement is live, it is on the review path above, not in CI.

Note: advisory (where it applies) refers to whether a sub-threshold score or an over-budget NoCoverage count actually stops the transition. The dimension entry is required on the high tier regardless of either axis's resolved severity (the all-reviews-passed guard fails if the mutation-adequacy review is absent), so run and record it every time.

Triggers

Activate this skill when:

• review reaches the quality stage on a HIGH-tier feature
• The review contract lists mutation-adequacy in the required reviews for this workflow
• You need to assess whether the (possibly relaxed) test mix actually kills mutants

Do not activate for medium/low-tier work, or outside the /review boundary — those paths do not require this dimension.

Execution

Step 1: Run the diff-scoped mutation gate

Invoke the action against the review/PR base ref. It runs the resolved mutation command diff-scoped (Stryker --since, cargo-mutants --in-diff, mutmut path restriction — resolved from the toolchains SoT, never composed by hand), so the run completes in < minutes, not the full-tree time budget.

exarchos:exarchos_orchestrate({
  action: "mutation-adequacy",
  featureId: "<featureId>",
  base: "<review/PR base ref>",      // e.g. "main" — reuse the same base the review diff uses
  worktreePath: "<optional worktree>", // 'auto' resolves the calling delegation's worktree
  operationId: "<optional idempotency key>"
  // scope defaults to "diff"; do NOT pass scope:"full" here (see Anti-Patterns)
})

The action emits mutation.executing_started / mutation.executed (liveness, INV-10) and a foldable gate.executed carrying mutationScore (INV-1) automatically. Do not hand-emit these events.

Step 2: Read the carrier

The action returns the fixed carrier (data). The shape is stable regardless of pass/fail/degrade:

Field	Meaning
passed	mutationScore >= threshold && noCoverage <= maxNoCoverage — two axes, two default severities (see Step 4)
mutationScore	killed / (total − noCoverage) — the Stryker convention; noCoverage is excluded from the denominator
killed	mutants a test caught (good)
survived	mutants that escaped — tests ran but did not catch them
noCoverage	mutants in code no test exercises at all
total	total mutants generated within the diff scope
threshold	the effective advisory threshold (override > config > soft default)
report	the parsed Stryker mutation-testing-report-schema
next_actions	"write a test that kills <file>:<line>" follow-ups (Step 3)

Degrade signals to recognize (each returns passed: true so the gate never blocks closed-with-error):

• skipped: true + reason — no mutation runner resolved. Report the remediation; do not treat as a failure.
• warning + a warnings[] entry — the runner produced no parseable report. Note it; do not throw.
• deferred: true + scope: 'full' — you (incorrectly) requested full scope; re-run with the default diff scope.

See references/reading-the-carrier.md for the full carrier and report shape.

Step 3: Turn survivors into kill-this-mutant follow-ups (INV-12)

The action already maps each surviving and NoCoverage mutant to a next_actions entry of the form "write a test that kills <file>:<line>". Surface these as concrete, actionable review findings — each one names a specific assertion the suite is missing:

• A survived mutant means a test exercises that line but asserts nothing strong enough to detect the mutation. The follow-up: add an assertion that distinguishes the mutated behavior.
• A NoCoverage mutant means no test touches that line at all. The follow-up: add a test that exercises it, then assert on the observable behavior.

Record these as issues with category: "test-quality" so they ride the same review-report contract as the other dimensions. Do not invent generic "improve coverage" advice — quote the file:line the action surfaced.

Step 4: Apply the verdict — two axes, each with its own resolved severity

The score axis stays advisory (warning) by default. A sub-threshold mutationScore:

• surfaces the survivor follow-ups (Step 3),
• warns in the review report,
• does not block the review → synthesize transition — unless an explicit review.gates['mutation-adequacy'] config override raises its severity. Honor the resolved severity, do not hardcode it. See references/advisory-threshold.md for why the default is a soft threshold and how to calibrate it from the score trend.

The NoCoverage axis is independent, with its own real block-mode enforcement path: under review.mutationEnforcement: block, if the folded noCoverage count exceeds the configured maxNoCoverage budget (default 0), the block-mode guard fails the transition regardless of the score. Like the score, it is opt-in (review.mutationEnforcement defaults to advisory) — resolve and honor the actual configured mode, don't assume either axis is on or off. Record both axes honestly in the review report — do not fold a blocking NoCoverage failure into advisory-score language, and do not describe either axis as enforced by CI (the diff-scoped CI wiring runs in observe mode; see Anti-Patterns).

Required Output Format

Record the dimension result on the review state. The review key MUST be the kebab-case dimension name (it equals this skill's folder name):

exarchos:exarchos_workflow({ action: "update", featureId: "<id>", updates: {
  reviews: { "mutation-adequacy": {
    status: "pass",            // "pass" is honest when both axes are advisory (the default); if either
                                // axis's severity is configured to block, report the real result instead
    summary: "mutationScore 0.62 (threshold 0.40); 3 survivors surfaced as kill-test follow-ups",
    issues: [
      { severity: "MEDIUM", category: "test-quality", file: "src/foo.ts", line: 42,
        description: "surviving mutant — no assertion distinguishes the mutated branch",
        required_fix: "write a test that kills src/foo.ts:42" }
    ]
  } }
}})

A passing-value status (pass | passed | approved | fixes-applied, case-insensitive) is required for the all-reviews-passed guard — a flat string is silently ignored and blocks the transition.

Anti-Patterns

Don't	Do Instead
Run scope: "full" inline	Full-tree mutation is the long-running op deferred to R10/v2.12 — it returns a deferred advisory, never an inline run
Compose --since / --in-diff by hand	Let the action resolve the diff scope from the toolchains SoT
Block the merge on a sub-threshold score with no config override	The score axis is advisory by default — surface follow-ups, honor the resolved severity
Assume NoCoverage is always advisory, or always blocking	It has its own real block-mode path (review.mutationEnforcement: block, default budget 0) — opt-in, just like the score's severity; resolve and honor the actual configured mode
Claim the CI mutation gate blocks merges	It runs diff-scoped in observe mode unconditionally (logs its verdict, always exits 0, regardless of config) — never represent it as a merge blocker
Treat a Skipped/Warning carrier as a hard failure	Both return passed: true — report the reason, never throw
Run this dimension for medium/low tier	It gates the HIGH tier at the /review boundary only
Emit gate.executed manually	The action auto-emits liveness + the foldable gate event
Give generic "add more tests" advice	Quote the <file>:<line> the action surfaced in next_actions

References

• references/reading-the-carrier.md — reading the carrier and the Stryker mutation-testing-report-schema.
• references/advisory-threshold.md — why the threshold is a soft default, and how to calibrate it.