acceptance-pipeline-catalog

Robert C. Martin Acceptance Pipeline Best Practices

Language-neutral specification for a portable acceptance-test pipeline: Gherkin feature files to JSON IR to generated acceptance tests to mutation testing. Based on Robert C. Martin's Acceptance Pipeline Specification. Contains ~50 rules across 14 categories, prioritized by impact.

When to Apply

Reference these rules when:

• Building a Gherkin parser that outputs JSON IR
• Implementing an acceptance test generator from JSON IR
• Writing an acceptance runtime that expands scenarios and dispatches steps
• Implementing mutation testing for acceptance test example values
• Setting up the full pipeline (parser, generator, runner, mutator) in a new project
• Debugging pipeline failures (parse errors, generation issues, mutation classification)

Pipeline Overview

The pipeline has two modes:

Normal acceptance run:

feature file -> gherkin parser -> JSON IR -> acceptance generator -> generated tests -> test runner

Mutation run:

feature file -> gherkin parser -> base JSON IR -> mutator (one changed IR per mutation)
  -> generator (tests per mutation) -> test runner (evaluate each) -> mutation report

The normal run proves the project satisfies the feature. The mutation run probes whether tests are strong enough to fail when example data changes.

Rule Categories by Priority

Priority	Category	Impact	Prefix	Rules
1	Parser	CRITICAL	parser-	9
2	JSON IR	CRITICAL	ir-	4
3	Generator	CRITICAL	gen-	2
4	Runtime	HIGH	rt-	3
5	Step Handlers	HIGH	handler-	4
6	Test Runner	HIGH	runner-	2
7	Mutator Core	HIGH	mut-	4
8	Value Mutation Rules	HIGH	val-	10
9	Result Classification	HIGH	result-	2
10	Conformance	HIGH	conform-	1
11	Agent Setup	HIGH	setup-	1
12	Mutation Execution	MEDIUM	exec-	4
13	Reports	MEDIUM	report-	3
14	Project Layout	MEDIUM	layout-	3

Quick Reference

1. Parser (CRITICAL)

• parser-command-interface - Two positional args, exit codes 0/1/2
• parser-feature-declaration - Feature: keyword required, trimmed name
• parser-background - Optional Background: section with Given/And steps
• parser-scenarios - Scenario: and Scenario Outline: both supported
• parser-steps - Given/When/Then/And keywords, keyword stored separately
• parser-parameters - Angle-bracket placeholders, not expanded by parser
• parser-examples-tables - Pipe-delimited tables, header row first
• parser-general-rules - Blank lines, comments, whitespace, ordering
• parser-unsupported-syntax - Tags, rules, localized keywords, doc strings

2. JSON IR (CRITICAL)

• ir-feature-object - name, scenarios, optional background
• ir-scenario-object - name, steps, examples arrays
• ir-step-object - keyword, text, optional parameters
• ir-example-object - String-keyed, string-valued maps

3. Generator (CRITICAL)

• gen-command-interface - Two positional args, exit codes 0/1/2
• gen-requirements - Embed IR, no Gherkin parsing, deterministic output

4. Runtime (HIGH)

• rt-responsibilities - Load IR, expand, dispatch, report
• rt-scenario-expansion - One execution per example row, background prepended
• rt-execution-naming - Scenario name / example index naming convention

5. Step Handlers (HIGH)

• handler-matching - Match by exact text value, not keyword
• handler-world-state - Fresh world/state per scenario execution
• handler-value-handling - Fetch, parse, fail on missing/malformed
• handler-unsupported - Unsupported step text must fail the test

6. Test Runner (HIGH)

• runner-interface - Input/output contract for the test runner adapter
• runner-classification - Three-way: failure, success, infrastructure error

7. Mutator Core (HIGH)

• mut-command-interface - CLI options, exit codes 0/1/2
• mut-scope - Only example cell values mutated
• mut-identity - Stable deterministic IDs, paths, descriptions
• mut-deep-copy - Original IR never modified in place

8. Value Mutation Rules (HIGH)

• val-rule-order - 8 rules applied in priority order
• val-comma-list - Comma-delimited list mutation
• val-boolean - true/false toggle
• val-null - null/nil/none to dithered string
• val-integer - Integer plus pseudo-random delta
• val-float - Float plus pseudo-random delta
• val-datetime - ISO-8601 date/time shift
• val-duration - Duration shift preserving syntax
• val-string-dither - Character-level string edits
• val-determinism - Pseudo-random, deterministic for fixed input

9. Result Classification (HIGH)

• result-statuses - killed, survived, error
• result-classification-rules - Mapping from test outcomes to statuses

10. Conformance (HIGH)

• conform-checklist - All 21 validation items

11. Agent Setup (HIGH)

• setup-checklist - 15-step installation guide

12. Mutation Execution (MEDIUM)

• exec-work-directory - Per-mutation directory structure
• exec-workflow - Write IR, generate, run, classify
• exec-parallelism - Concurrent workers, isolated directories
• exec-timeout - Full-run timeout, unfinished = error

13. Reports (MEDIUM)

• report-text-format - Summary line + per-result lines
• report-json-format - JSON object with summary and results array
• report-field-requirements - Required fields for summary and results

14. Project Layout (MEDIUM)

• layout-required-paths - features/, build/, acceptance/ directories
• layout-commands - gherkin-parser, acceptance-generator, gherkin-mutator
• layout-scripts - Normal acceptance and mutation scripts

How to Use

Read individual reference files for detailed spec requirements and rationale:

• Start with the category relevant to the component you are building
• Each rule file is self-contained with WHY explanations, spec requirements, and examples
• For a new project setup, read setup-checklist first
• For validation, use conform-checklist
• Check gotchas.md for known failure points

Reference Files

File	Description
metadata.json	Version and reference information
gotchas.md	Known failure points (append-only)
references/	All rule files organized by prefix