Validation Pipeline — Epic Design Doc#

Status: 🔄 In Refinement (Step 0) Authors: Dan Hannah & Clay Created: March 23, 2026

Overview#

What Is This Epic?#

The Validation Pipeline is Routr's quality assurance backbone — a multi-layered testing system that ensures the app produces dimensionally accurate, safe G-code for real CNC machines. It bridges the gap between "the app works in the browser" and "this G-code is safe to run on wood."

This is Routr's first forward-looking design doc. Every previous doc was retroactive — reading existing code and documenting what was already built. This one designs before building. That's the CSDLC leveling up.

Problem Statement#

Routr has solid unit test coverage across toolpath generators, coordinate transforms, and SVG parsing. But there is no automated way to verify that the full pipeline — from board configuration to final G-code output — produces correct results. And beyond digital verification, there's no formal process for validating that G-code performs correctly on a physical CNC machine.

What's broken today:

• A change to the toolpath generator could silently alter G-code output for every operation type, and no test would catch it
• Coordinate system bugs (like the edge treatment flip and kerf line flip) made it to real cuts before being discovered
• The design tab, simulator, and G-code can each show different things — a chamfer placed on the top edge in the design tab showed up on the bottom in the sim tab but cut correctly in real life. No automated test catches cross-layer visual inconsistencies.
• There's no "known good" G-code baseline to compare against
• Physical validation cuts have been done ad hoc — no formal protocol, no documented results, no traceability from design dimensions to measured dimensions
• There's no way for AI sub-agents to programmatically interact with the app for testing — all interaction requires UI automation through a human-designed interface

The deeper problem:

In an AI-driven development workflow, the AI writes the code — but how does the AI prove to the human that the code works? Unit tests are invisible to most humans. The validation pipeline is the mechanism by which the AI demonstrates correctness: "Here's the G-code output, here's what the design tab looks like, here's what the sim looks like, here are the dimensional measurements, and here's how all of these match." The human reviews evidence, not code. This shifts the human role from writing the proof to reviewing the proof — the AI-driven equivalent of test-driven development.

What triggered this work:

The coordinate flip bug class — the same category of bug appeared in edge treatments and table saw kerf lines. Both were caught during physical cuts, not by tests. The chamfer top/bottom flip between design and sim tabs proved that G-code correctness alone isn't sufficient — visual consistency across the app's layers matters too. With launch approaching, the question is: what else might be wrong that we haven't cut yet?

Goals#

• Define the complete validation pipeline from automated tests to physical verification
• Establish integration tests driven by JSON test fixtures that validate G-code output, 2D canvas rendering, AND 3D simulator rendering
• Create a formal physical validation protocol with documented results
• Build an in-app measurement tool for fast dimensional comparison between design and physical cuts — available in both 2D design tab and 3D sim tab for cross-layer verification
• Define AI-accessible hooks that let sub-agents programmatically drive and test the app
• Build an MCP server that wraps the app's capabilities for AI-driven exploratory testing and future product features
• Make "is this operation validated?" answerable for any feature at any time
• Enable fast, confident iteration on the codebase without fear of silent regressions
• Establish a testing strategy pattern that applies to all future epics/features

Non-Goals#

• Not fixing the kerf line flip bug — that's a standalone ticket
• Not adding roundover validation — roundover is feature-flagged off for launch
• Not automating physical cuts — the human runs the CNC machine; this epic defines the protocol

Context#

Current State#

What exists today:

The unit tests are solid but test functions in isolation. None of them test the full pipeline: "given this board with these shapes and these tool settings, the G-code output should be exactly X and the 2D canvas should look exactly like Y and the sim should look exactly like Z."

Affected Systems#

Dependencies#

• Coordinate flip fix (SHIPPED) — centralized edgeMapping.ts must be in place before establishing baselines
• Kerf line flip bug (OPEN) — should be fixed before table saw baselines are established
• Playwright — already available; needed for screenshot capture and AI-driven app interaction

Dependents#

• Production launch — launch requires minimum validation confidence per operation type
• Future epics/features — every new feature enters the pipeline through this testing framework
• PDF-to-G-code pipeline — AI hooks built for testing become the foundation for AI-driven app features
• Epic design template — testing strategy pattern from this epic gets extracted into the template for all future epics

Design#

Three-Layer Testing Architecture#

graph TD
    subgraph L1["Layer 1: Unit Tests (Existing)"]
        UT["Individual function tests<br/>Math, transforms, parsing"]
    end

    subgraph L2["Layer 2: Functional / Integration Tests (NEW)"]
        JSON["JSON Test Fixture<br/>(board + shapes + tools)"]
        JSON --> HOOKS["AI-Accessible Hooks<br/>Load fixture → Zustand store"]
        HOOKS --> GCODE["G-code Output<br/>→ assert matches expected .nc"]
        HOOKS --> CANVAS2D["2D Design Tab Screenshot<br/>→ pixel-diff against golden .png"]
        HOOKS --> SIM3D["3D Sim Tab Screenshot<br/>→ pixel-diff against golden .png"]
        HOOKS --> MEASURE_CROSS["Cross-layer measurement<br/>Design tab dims = Sim tab dims?"]
        GCODE --> CROSS["Cross-layer consistency<br/>All outputs agree?"]
        CANVAS2D --> CROSS
        SIM3D --> CROSS
        MEASURE_CROSS --> CROSS
    end

    subgraph L3["Layer 3: E2E Tests (NEW)"]
        FIXTURE["Test fixture with<br/>known dimensions"]
        FIXTURE --> MEASURE_APP["In-app measurement tool<br/>→ expected dimensions"]
        FIXTURE --> CUT["Cut on CNC machine"]
        CUT --> MEASURE_REAL["Measure with calipers<br/>→ actual dimensions"]
        MEASURE_APP --> COMPARE["Compare:<br/>design vs actual"]
        MEASURE_REAL --> COMPARE
        COMPARE --> RECORD["Validation record<br/>±0.5mm tolerance"]
        EYEBALL["Simulator eyeball check<br/>Does the sim look right?"]
        EYEBALL --> CUT
    end

    L1 --> |"Functions correct"| L2
    L2 --> |"Pipeline + visuals correct"| L3
    L3 --> |"Real-world confirmed"| LAUNCH["Launch Confidence ✅"]

Each layer catches different categories of bugs:

Quality Gates#

Quality gates define WHERE in the CSDLC pipeline each check lives. This maps the testing layers to the development process:

graph LR
    subgraph DEV["Development (Step 3)"]
        UNIT_G["🚦 Gate 1: Unit Tests<br/>Must pass before PR"]
        INT_G["🚦 Gate 2: Integration Tests<br/>G-code assertions must match"]
    end

    subgraph CI["CI / PR (Automated)"]
        CI_G["🚦 Gate 3: GitHub Actions<br/>Unit + G-code integration<br/>BLOCKS merge"]
    end

    subgraph REVIEW["Review (Steps 4-6)"]
        VIS_G["🚦 Gate 4: Visual Regression<br/>AI Lead reviews screenshot diffs"]
        LEAD_G["🚦 Gate 5: AI Lead Review<br/>Output matches ticket scope"]
        HUMAN_G["🚦 Gate 6: Human Review<br/>Intent met, feels right"]
    end

    subgraph LAUNCH["Pre-Launch"]
        E2E_G["🚦 Gate 7: Physical Validation<br/>Dimensional accuracy on CNC"]
    end

    DEV --> CI --> REVIEW --> LAUNCH

The key insight: Gates 1-3 are cheap and automated — they run on every change. Gates 4-5 are AI-assisted — the AI Lead surfaces evidence for review. Gate 6 is human judgment — the irreplaceable "does this feel right?" check. Gate 7 is physical — rare but critical for real-world trust.

CI/CD Strategy#

Split approach: text-based tests run locally as part of sub-agent verification. Visual regression tests run in GitHub Actions CI.

Why the split: Text-based tests (unit, G-code integration, cross-layer) are deterministic and fast — sub-agents run them locally before creating PRs. No PR has shipped with broken tests to date. Visual tests have a different workflow problem: they produce screenshot artifacts that need to be attached to PRs, and rendering consistency matters (same browser/OS). GitHub Actions solves both — Playwright runs in a consistent environment, diff images upload as artifacts, and the action auto-posts a PR comment with the visual report.

Future: Expand CI. Unit and G-code integration tests can be added to CI when team size or failure rate justifies it. The infrastructure is CI-ready (Vitest, no browser needed).

Cross-Cutting Concern: AI-Accessible Interface#

Since this app is designed and written by AI, it makes sense to build first-class interfaces for AI to drive and test it. This isn't just about testing — it's about making the app AI-native. The hooks built here for validation become the foundation for future AI features like PDF-to-G-code.

This may be a core architectural pattern for AI-designed apps. Every modern app has a UI layer (for humans), an API layer (for services), and a data layer (for persistence). AI-designed apps need an AI layer — a structured interface purpose-built for LLMs to understand, drive, and extend the app. Not a REST API (too low-level, no semantic context). Not the UI (too fragile, designed for humans). An MCP-style interface that gives AI agents the same fluency with the app that a senior developer has. This pattern should be considered for every CSDLC project going forward.

Dev-mode API (window.__routr):

Exposed in development/test builds, and on staging behind a feature flag for AI feature experimentation. NOT in production until a product feature requires it.

// Proposed dev-mode API surface
window.__routr = {
  // Fixture loading
  loadFixture(fixture: TestFixture): void,      // Load JSON fixture into Zustand store
  getState(): ProjectState,                       // Read current store state

  // G-code
  generateGcode(): string,                        // Trigger pipeline, return G-code string
  generateToolpaths(): ToolpathOperation[],        // Get intermediate toolpath data

  // Measurement (same engine used by in-app measurement tool)
  measureDistance(pointA: Point, pointB: Point): number,  // Distance between two points (mm)
  measureFromEdge(shapeId: string, edge: 'top' | 'bottom' | 'left' | 'right'): number,

  // Canvas capture
  captureDesignCanvas(): Promise<ImageData>,       // 2D canvas screenshot
  captureSimCanvas(): Promise<ImageData>,          // 3D sim screenshot

  // Store actions
  addBoard(config: BoardConfig): string,           // Returns board ID
  addShape(boardId: string, shape: ShapeConfig): string,
  setToolSettings(settings: ToolSettings): void,
  addEdgeTreatment(boardId: string, treatment: EdgeTreatmentConfig): void,
};

Why this matters:

• Testing: Sub-agents call loadFixture() + generateGcode() instead of clicking through UI
• Reliability: No flaky CSS selectors, no waiting for animations, no "button didn't render yet"
• Speed: Direct function calls are orders of magnitude faster than Playwright UI automation
• Future features: PDF-to-G-code AI reads a plan, calls addBoard() + addShape() to build the design programmatically
• Validation: measureDistance() and measureFromEdge() make dimensional verification instant — in both 2D and 3D contexts

MCP Server: AI-Native Tool Interface (Separate Package)#

The window.__routr API gives programmatic access from within the browser. But to let AI agents reason about and drive the app from outside — connecting from Claude, OpenClaw sub-agents, or any MCP-compatible client — we wrap the same capabilities in an MCP (Model Context Protocol) server.

The MCP server lives as a separate package in the repo (packages/routr-mcp or similar) with its own versioning. This keeps concerns clean — the app doesn't depend on MCP, and MCP can evolve independently. It also positions the MCP server for distribution beyond just testing (future product features, marketplace integrations).

What MCP adds over a raw API:

MCP doesn't just expose endpoints — it gives each tool rich descriptions, parameter schemas, and contextual guidance so the LLM knows when, why, and how to use each tool. The AI doesn't just know "there's a loadFixture function." It knows "use loadFixture when you want to set up a test scenario — it takes a fixture JSON describing a board with shapes and tool settings, and after loading you can generate G-code or capture screenshots to verify the result."

Proposed MCP tool surface:

MCP Server: routr-tools
├── create_board          — Create a board with dimensions and material
├── add_shape             — Add a cut, pocket, hole, path, or slot to a board
├── add_edge_treatment    — Apply chamfer, dado, or rabbet to a board edge
├── set_tool_settings     — Configure bit, feeds, speeds
├── import_svg            — Import an SVG file for engrave/pocket
├── generate_toolpaths    — Generate toolpath operations from current state
├── generate_gcode        — Generate G-code and return as string
├── capture_design_view   — Screenshot the 2D design canvas
├── capture_sim_view      — Screenshot the 3D simulator at a standard angle
├── measure_distance      — Measure between two points or shape-to-edge
├── load_fixture          — Load a test fixture JSON into the app
├── get_project_state     — Read the current Zustand store state
└── validate_gcode        — Compare generated G-code against an expected file

AI-Driven Exploratory Testing & the Protagonist/Antagonist Pattern#

Scripted integration tests (fixtures + assertions) catch regressions. But an AI agent connected via MCP can do something far more powerful — creative, exploratory testing:

• "Design a board with a 2-inch pocket centered on a 6×12 board, generate the G-code, and verify the pocket coordinates are correct"
• "Try every edge treatment on every edge and check for visual inconsistencies between the design and sim tabs"
• "Create a stress test — maximum shapes, weird dimensions, overlapping cuts — and report what breaks"
• "Verify that changing the bit diameter correctly adjusts all toolpath offsets"

The AI understands woodworking concepts, understands the app's capabilities, and can creatively find edge cases that no human would think to write a fixture for. This is a fundamentally different testing paradigm — not "did it change?" (regression) but "is it correct?" (verification).

Protagonist/Antagonist Agent Pattern:

This maps naturally to the CSDLC pipeline:

graph LR
    PROTAGONIST["🟢 Protagonist Agent<br/>(Step 3: Implementation)<br/>Writes the code,<br/>creates the PR"] --> ANTAGONIST["🔴 Antagonist Agent<br/>(Step 4-5: Review + QA)<br/>Connects via MCP,<br/>tries to break it"]
    ANTAGONIST --> |"Finds issues"| PROTAGONIST
    ANTAGONIST --> |"No issues found"| HUMAN["👤 Human Review<br/>(Step 6)"]

• The protagonist is the sub-agent executing the ticket (Step 3). It writes code, runs scripted tests, creates the PR.
• The antagonist is a separate AI agent that reviews the protagonist's work (Steps 4-5). It connects via MCP, loads the affected fixtures, runs exploratory tests, tries creative edge cases, and reports findings.
• The human reviews the evidence from both agents (Step 6) and makes the final call.

This adversarial pattern drives quality because the antagonist has different incentives than the protagonist — its job is to find problems, not ship features. It's the AI equivalent of a dedicated QA engineer who didn't write the code.

The product vision connection:

The MCP server built for testing IS the integration layer for Routr's AI product features:

Architecture:

graph TD
    subgraph CLIENTS["MCP Clients"]
        PROTAGONIST["🟢 Protagonist Agent"]
        ANTAGONIST["🔴 Antagonist Agent"]
        CLAUDE["Claude / AI Agents"]
        OPENCLAW["OpenClaw Sub-agents"]
    end

    subgraph MCP["MCP Server: routr-tools<br/>(separate package)"]
        TOOLS["Tool definitions<br/>+ descriptions + schemas"]
    end

    subgraph APP["Routr App (Browser)"]
        API["window.__routr API"]
        STORE["Zustand Store"]
        ENGINE["Engine Pipeline"]
        CANVAS["Canvas Renderers"]
        MEASURE["Measurement Engine"]
    end

    CLIENTS --> |"MCP protocol"| MCP
    MCP --> |"Playwright bridge<br/>or direct API"| APP
    API --> STORE
    API --> ENGINE
    API --> CANVAS
    API --> MEASURE

The MCP server connects to the running Routr app (via Playwright for browser-based interaction, or directly to the engine for headless G-code generation). This means we can run both:

• Headed mode: MCP drives the actual app UI (for visual testing)
• Headless mode: MCP calls engine functions directly (for fast G-code testing)

Layer 2: Functional / Integration Tests (Detail)#

This is the core of the epic. Integration tests tie together multiple outputs from a single input.

Integration Test Architecture#

graph TD
    subgraph INPUT["Test Input"]
        FIXTURE["JSON Test Fixture<br/>📄 fixture.json"]
    end

    subgraph LOAD["Loading"]
        HOOKS["window.__routr.loadFixture()<br/>or MCP load_fixture"]
    end

    subgraph OUTPUTS["Test Outputs (all from same fixture)"]
        GCODE["G-code Output<br/>📄 expected.nc"]
        DESIGN["2D Design Tab<br/>📸 design.png"]
        SIMTAB["3D Sim Tab<br/>📸 sim.png"]
        MEASURE_2D["2D Measurements<br/>shape-to-edge distances"]
        MEASURE_3D["3D Measurements<br/>toolpath-to-edge distances"]
    end

    subgraph ASSERTIONS["Assertions"]
        A1["G-code matches expected file"]
        A2["2D screenshot matches golden image"]
        A3["3D screenshot matches golden image"]
        A4["2D measurements = 3D measurements<br/>(cross-layer consistency)"]
    end

    FIXTURE --> HOOKS
    HOOKS --> GCODE --> A1
    HOOKS --> DESIGN --> A2
    HOOKS --> SIMTAB --> A3
    HOOKS --> MEASURE_2D --> A4
    HOOKS --> MEASURE_3D --> A4

How the test runner works:

1. Load — Read fixture JSON, call loadFixture() to hydrate Zustand store
2. Generate — Call generateGcode(), capture the output string
3. Capture — Trigger Playwright screenshots of design tab and sim tab at locked viewport/camera/theme
4. Measure — Call measureFromEdge() for key shapes in both 2D design context and 3D sim context
5. Assert — Compare all outputs against stored baselines:
   • G-code: character-by-character diff against .expected.nc
   • Screenshots: pixel-diff with tolerance threshold against golden .png files
   • Measurements: 2D design dimensions must equal 3D sim dimensions (cross-layer consistency)
6. Report — On failure, output the diffs for human review

Test Report: How AI Proves Changes Are Valid#

The integration test runner generates an HTML report (or markdown summary) on every run. This is the primary mechanism by which the AI demonstrates correctness to the human:

📊 Integration Test Report — 2026-03-24 14:32

Fixture: pocket-rectangle
├── G-code:      ✅ Match (no diff)
├── Design 2D:   ✅ Match (0.1% pixel diff — within tolerance)
├── Sim 3D:      ✅ Match (0.0% pixel diff)
└── Measurements: ✅ 2D pocket-to-left-edge: 47.0mm | 3D: 47.0mm

Fixture: edge-chamfer-top
├── G-code:      ⚠️ Changed (diff below)
│   - Line 42: G1 Z-3.175 → G1 Z-3.200
│   - [Review: depth calculation adjusted]
├── Design 2D:   ✅ Match
├── Sim 3D:      ✅ Match
└── Measurements: ✅ Consistent

Overall: 14/16 fixtures pass | 2 changed (review required)

For visual diffs, the report includes side-by-side before/after images with pixel differences highlighted. The human scans the gallery, verifies the changes look correct, and approves. No manual screenshot capturing needed — the test infrastructure generates the evidence.

The protagonist sub-agent includes this report summary in the PR description. The human reviews evidence, not code.

Baseline management:

• First run with a new fixture: npm run test:integration -- --update saves current outputs as baselines
• Subsequent runs compare against baselines
• When a baseline changes intentionally: review the diff, run --update to accept
• Fixture updates require human review — the diff must be inspected, not rubber-stamped

Test commands:

# Run G-code integration tests only (fast, CI-friendly)
npm run test:integration:gcode

# Run visual regression tests (requires browser, slower)
npm run test:integration:visual

# Run all integration tests
npm run test:integration

# Update baselines after intentional changes
npm run test:integration -- --update

The JSON Test Fixture#

A fixture file fully describes a project state — everything needed to reproduce a specific board configuration deterministically:

{
  "name": "straight-cut-basic",
  "description": "Single vertical table saw cut through a board",
  "board": {
    "width": 200,
    "height": 100,
    "thickness": 19
  },
  "shapes": [
    {
      "type": "rectangle",
      "cutType": "table-saw",
      "position": { "x": 100, "y": 0 },
      "params": { "width": 0, "height": 100 },
      "depth": 19
    }
  ],
  "toolSettings": {
    "bitDiameter": 6.35,
    "feedRate": 1000,
    "plungeRate": 500,
    "stepDown": 3,
    "safeHeight": 5,
    "spindleSpeed": 18000
  },
  "edgeTreatments": [],
  "units": "metric"
}

Fixture design principles:

• Each fixture tests one concern — a straight cut fixture, a pocket fixture, etc.
• Use known, simple dimensions — easy to mentally verify
• Specify ALL settings explicitly — never rely on defaults (defaults change, fixtures shouldn't)
• A comprehensive fixture combines multiple operations to test ordering and tool changes

Three Assertion Types + Cross-Layer Measurement#

1. G-code assertion — Load fixture via loadFixture(), call generateGcode(), compare against stored .expected.nc file. Character-by-character match. If it differs → developer reviews diff → update expected file (intentional) or fix regression.

2. 2D canvas screenshot — Load fixture, capture design tab canvas via Playwright at locked viewport + theme. Compare against golden .design.png using pixel-diff with tolerance threshold (absorbs anti-aliasing). Catches: shapes in wrong positions, cuts on wrong side, visual artifacts.

3. 3D sim screenshot — Load fixture, generate toolpaths, capture sim tab at a fixed camera angle. Compare against golden .sim.png. Catches the chamfer-on-wrong-edge class of bugs — where design tab shows one thing and sim shows another.

4. Cross-layer measurement — Use the measurement API to check the same dimensions in both the 2D design tab and 3D sim tab. "This pocket is 47mm from the left edge" should be true in BOTH contexts. If the numbers disagree, we've found a coordinate bug. This is the most precise way to catch the kerf line flip class of bugs — not just "does it look right?" but "do the numbers match?"

Standard Camera Angles for 3D Sim Screenshots#

Consistent camera angles ensure deterministic screenshots. We use orthographic views only — perspective projection introduces angle distortion that makes pixel-diff unreliable for position verification (a 5-pixel shift in perspective could be correct or a bug — impossible to tell without angle correction math).

Every fixture captures both angles. Between top-down and front, all three dimensions are covered without any angle correction:

• X position — verified by both views
• Y position — verified by top-down
• Z depth — verified by front view + G-code assertions (exact Z values) + measurement API

Pixel-diff on orthographic views is clean and reliable — if something shifts, it's a real change, not a projection artifact. Additional angles can be added later if needed, but these two plus the measurement API provide complete dimensional coverage.

File Structure#

cncmill-app/src/__fixtures__/
├── straight-cut-basic/
│   ├── fixture.json          # Input: board + shapes + tools
│   ├── expected.nc           # Expected G-code output
│   ├── design.png            # Golden 2D canvas screenshot
│   ├── sim-top.png           # Golden 3D sim screenshot (top-down orthographic)
│   └── sim-front.png         # Golden 3D sim screenshot (front orthographic)
├── pocket-rectangle/
│   └── ...
├── drill-basic/
│   └── ...
├── edge-chamfer-top/
│   ├── fixture.json
│   ├── expected.nc
│   ├── design.png
│   ├── sim-top.png           # top-down (verify X/Y position)
│   └── sim-front.png         # front (verify which edge the chamfer is on)
├── svg-engrave/
│   └── ...
└── multi-operation/          # Comprehensive: multiple ops on one board
    └── ...

Fixture Inventory#

Layer 3: E2E Tests (Detail)#

E2E testing is the full loop: design in app → verify in simulator → cut on CNC → measure physical piece → compare against design dimensions.

Simulator Eyeball Check#

Before cutting anything, the human reviews the simulator render: "Does this look like what I'd expect this cut to produce?" This is NOT automated — it's a human judgment call. The simulator already exists and works; this just formalizes reviewing sim output before committing material.

When to do it:

• After establishing new fixture baselines
• After any coordinate system changes
• Before physical validation cuts (preview what you're about to cut)

Physical Validation Protocol#

For each validation cut:

1. Design — Load test fixture in Routr (or create the design matching the fixture)
2. In-app measurement — Use the measurement tool in the design tab to record expected dimensions (e.g., "hole center is 47.5mm from left edge, 32mm from top edge"). Verify the same measurements in the sim tab.
3. Eyeball sim — Check sim tab — does it look right?
4. Export — Generate G-code
5. Setup — Load G-code on CNC, set origin, verify material dimensions with calipers
6. Cut — Run the program
7. Measure — Use calipers to measure the same dimensions recorded in step 2
8. Record — Document: expected → actual → delta → pass/fail with comments

Tolerance target: ±0.5mm (±0.020")

Starting point. Tighter than most hobby CNC work but achievable with proper setup. Will adjust based on real measurement data.

Validation Matrix#

Validation Record Template#

Physical validation is pass/fail with comments. Photos are optional — useful for documentation but not required.

## [Fixture Name] — Physical Validation Record

**Date:** YYYY-MM-DD
**Fixture:** `__fixtures__/[name]/fixture.json`
**Material:** [species, dimensions]
**Machine:** [CNC model]
**Bit:** [type, diameter]

### Measurements

| Dimension | Expected (app) | Actual (calipers) | Delta | Pass/Fail |
|-----------|:-:|:-:|:-:|:-:|
| Cut position from left edge | 100.0mm | 99.8mm | -0.2mm | ✅ |
| Cut depth | 19.0mm | 18.7mm | -0.3mm | ✅ |

### Checks
- [ ] Sim visual matches design tab
- [ ] Cut positions look correct
- [ ] Edge treatments on correct edges
- [ ] Cross-layer measurements match (design tab = sim tab)

### Result: ✅ PASS / ❌ FAIL
**Comments:** [observations, anomalies, lessons, anything worth noting]

In-App Measurement Tool#

Purpose: Surface the dimensional data the app already knows internally in a way that's useful for both validation and everyday use. Available in both the 2D design tab and the 3D sim tab for cross-layer verification.

How it works:

The app already stores exact positions, dimensions, and relationships between shapes and toolpaths. The measurement tool exposes this data:

• Point-to-point distance — Click two points on the canvas, see the distance in current units
• Shape-to-edge distance — Select a shape, see its distance from each board edge
• Shape dimensions — Select a shape, see its width, height, depth, position
• Toolpath-to-edge distance (sim tab) — Same measurements but against generated toolpaths

Cross-layer validation use case:

The measurement tool in the design tab shows "this pocket is 47mm from the left edge." The measurement tool in the sim tab should show the toolpath for that pocket starts 47mm from the left edge. If they disagree, we've found a coordinate bug. This is how we'd catch the kerf line flip — the design tab says the kerf line is at X=100, the sim tab says the toolpath is at X=100... but wait, the G-code says X=0 (board width minus 100 after flipY). The measurement tool makes the discrepancy visible without needing to read G-code.

UX approach:

Top toolbar "Measure" mode — similar to Fusion 360's approach. A tools dropdown in the top bar that activates measurement mode. When active:

• Clicking on the canvas shows dimension overlays
• Works in both 2D design tab (measures against shapes/board) and 3D sim tab (measures against toolpaths/board)
• Same visual language in both contexts for consistency
• Deactivate by clicking the mode again or pressing Escape

Implementation approach:

The measurement engine is shared between the UI tool and the window.__routr API. measureDistance() and measureFromEdge() serve both the in-app overlay and automated testing. This means integration tests can assert measurements programmatically — the same data the human sees in the overlay.

Testing Strategy for New Features#

When a new epic or feature is built, it enters the validation pipeline through a standard process:

graph TD
    NEW["New Feature / Epic"] --> FIXTURE["Create test fixture(s)<br/>covering the feature"]
    FIXTURE --> UNIT["Write unit tests<br/>for new engine functions"]
    FIXTURE --> INTEGRATION["Establish integration baselines<br/>G-code + 2D + 3D screenshots<br/>+ cross-layer measurements"]
    UNIT --> REVIEW["AI Lead reviews<br/>all baselines"]
    INTEGRATION --> REVIEW
    REVIEW --> ANTAGONIST{"Run antagonist<br/>agent? (optional)"}
    ANTAGONIST --> |Yes| EXPLORE["🔴 Antagonist explores<br/>via MCP, tries to break it"]
    ANTAGONIST --> |No| PHYSICAL
    EXPLORE --> PHYSICAL{"New operation type?"}
    PHYSICAL --> |Yes| CUT["Physical validation cut<br/>following E2E protocol"]
    PHYSICAL --> |No| DONE["✅ Feature validated"]
    CUT --> DONE

Rules:

1. Every new feature must have at least one fixture. No exceptions. The fixture is created during story breakdown (Step 1).
2. G-code baselines are mandatory. If the feature affects G-code output, integration test baselines must be established before the PR is merged.
3. Visual baselines are mandatory for UI-affecting changes. If it changes what the design tab or sim tab renders, screenshot baselines must be established.
4. Cross-layer measurement assertions are mandatory for spatial features. If the feature places or moves things on the canvas, measurement assertions must verify consistency between design and sim tabs.
5. Physical validation is required only for new operation types. New cut type, new edge treatment, new toolpath algorithm → needs a real cut. Bug fixes and UI changes do not.
6. Fixture updates require human review. When a fixture baseline changes, the diff must be reviewed by the human before accepting. No rubber-stamping.
7. Antagonist testing is encouraged for complex features. Connect an antagonist agent via MCP to exploratory-test the new feature. Not mandatory for every PR, but strongly recommended for new operation types and coordinate-affecting changes.

📝 PROCESS NOTE: This testing strategy should be extracted into the epic design doc template after this epic is finalized. Every future epic should include a "Testing Strategy" section that references these rules and specifies which fixtures will be created.

Integration Test Strategy#

How integration tests are structured and maintained as the codebase grows:

Test organization:

cncmill-app/
├── src/
│   ├── __fixtures__/              # JSON fixtures + golden files
│   └── __tests__/
│       ├── integration/
│       │   ├── gcode.test.ts      # G-code assertion tests (all fixtures)
│       │   ├── visual.test.ts     # Screenshot comparison tests (all fixtures)
│       │   └── measurement.test.ts # Cross-layer measurement tests
│       └── ...                    # Existing unit tests
├── playwright.config.ts           # Visual test configuration
└── package.json                   # Test scripts

Writing a new integration test:

1. Create a fixture directory under __fixtures__/ with fixture.json
2. Run npm run test:integration -- --update to generate initial baselines
3. Review the baselines manually — do the G-code, screenshots, and measurements look correct?
4. Commit the fixture + baselines together
5. From this point forward, any change that affects this fixture's output will cause a test failure

When tests fail:

graph TD
    FAIL["❌ Integration test fails"] --> REVIEW["Review the diff"]
    REVIEW --> INTENTIONAL{"Intentional change?"}
    INTENTIONAL --> |Yes| INSPECT["Inspect: does new output<br/>look correct?"]
    INSPECT --> |Yes| UPDATE["npm run test:integration -- --update<br/>Commit updated baselines"]
    INSPECT --> |No| BUG["Bug in the change — fix it"]
    INTENTIONAL --> |No| REGRESSION["Regression — investigate<br/>and fix before merging"]

Avoiding flaky visual tests:

• Lock viewport size: 1280×720 for all Playwright screenshots
• Lock theme: light mode
• Lock camera angles: standard angles defined in fixture or test config
• Pixel-diff tolerance: allow ~1-2% pixel difference for anti-aliasing
• Run on consistent environment (same browser version)
• If a test is persistently flaky, increase tolerance or switch to structural assertion

Mass baseline updates (UI changes):

When a UI change (CSS restyle, canvas rendering update, theme tweak) breaks multiple visual baselines at once:

1. Run npm run test:integration:visual — see which fixtures fail
2. Run npm run test:integration:visual -- --update — regenerate all screenshot baselines
3. Review the batch diff — a gallery of before/after screenshots across all fixtures
4. Verify: do the new visuals look correct? Is anything unexpectedly shifted or missing?
5. Commit the updated baselines with the UI change PR

This is working as designed — a UI change SHOULD force review of all visual baselines. The test breakage surfaces the full visual impact of the change. G-code and measurement tests are unaffected by pure UI changes, so those still pass and confirm the engine is untouched.

Agent prompt template — integration test section:

Every sub-agent prompt for implementation work should include:

## Verification
After implementation:
1. Run `npm test -- --run` — all unit tests must pass
2. Run `npm run test:integration:gcode` — all G-code assertions must pass
3. If your change affects G-code output:
   - Review the diffs carefully
   - Update baselines with `npm run test:integration:gcode -- --update`
   - Include baseline diffs in PR description for human review
4. If your change affects UI rendering:
   - Run `npm run test:integration:visual`
   - Update baselines with `npm run test:integration:visual -- --update`
   - Include before/after screenshot comparisons in PR description
5. Do NOT rubber-stamp baseline updates — review every diff

Antagonist agent (optional, for complex changes):

The antagonist is NOT a sub-agent that runs on every PR. It's a separate agent spawned by the AI Lead when extra confidence is needed — typically for new operation types, coordinate-affecting changes, or complex multi-system features. The antagonist connects via MCP and runs exploratory tests. Its findings are advisory — the human makes the final call.

Edge Cases & Gotchas#

Risks#

Features#

Features extracted from this epic. Each becomes a set of implementable stories during Step 1.

graph TD
    F1["F1: AI-Accessible Interface<br/>(window.__routr)"] --> F2["F2: MCP Server<br/>(routr-tools, separate pkg)"]
    F1 --> F3["F3: Test Fixture Library<br/>(incl. comprehensive fixture)"]
    F1 --> F6["F6: In-App Measurement Tool<br/>(2D + 3D, Fusion 360 style)"]
    F3 --> F4["F4: G-code Integration Tests<br/>+ CI Pipeline + Test Report"]
    F3 --> F5["F5: Visual Regression +<br/>Cross-Layer Coordinate Tests"]
    F3 --> F7["F7: Physical Validation<br/>Protocol (lightweight)"]
    F6 --> F7
    F2 --> |"Enables protagonist/<br/>antagonist pattern"| F4
    F2 --> |"Enables exploratory<br/>testing"| F5

Features are broken down into implementable stories during Step 1 (Story Breakdown). This table is the feature index.

Story Breakdown#

Implementation Batches#

Stories are organized into dependency-ordered batches:

Batch 1: F1 (AI Interface) → F3 (Fixtures) → F4 (G-code Integration Tests)
  → First real regression protection, fast feedback loop

Batch 2: F5 (Visual Regression + Cross-Layer) → F2 (MCP Server)
  → Screenshot baselines, cross-layer assertions, GitHub Actions visual CI, then external AI access

Batch 3: F7 (Physical Validation Protocol)
  → Human on the CNC, protocol + recording templates

F1: AI-Accessible Interface — Stories#

Dependency graph:

graph TD
    S1["F1-S1: Foundation & Types"] --> S2["F1-S2: Fixture Loading<br/>& State Access"]
    S1 --> S4["F1-S4: Programmatic<br/>Store Actions"]
    S1 --> S5["F1-S5: Canvas Capture"]
    S1 --> S6["F1-S6: Measurement Engine"]
    S2 --> S3["F1-S3: G-code<br/>Generation Hooks"]

    S3 -.-> F3["F3: Test Fixture Library"]
    S3 -.-> F4["F4: G-code Integration Tests"]
    S4 -.-> F2["F2: MCP Server"]
    S5 -.-> F5["F5: Visual Regression"]
    S6 -.-> F5

    style S1 fill:#e67e22,color:#fff
    style S2 fill:#e67e22,color:#fff
    style S3 fill:#e67e22,color:#fff
    style S4 fill:#3498db,color:#fff
    style S5 fill:#3498db,color:#fff
    style S6 fill:#3498db,color:#fff
    style F3 fill:#555,color:#fff
    style F4 fill:#555,color:#fff
    style F2 fill:#555,color:#fff
    style F5 fill:#555,color:#fff

🟠 Orange = critical path (serial). 🔵 Blue = parallelizable after S1.

Code Investigation Notes#

These findings informed story scoping:

• Feature flags: Two patterns in use — featureFlags.ts (compile-time booleans) and hostname checks (runtime). Neither is a proper env-driven system. For window.__routr, we'll use import.meta.env.VITE_EXPOSE_DEV_API so Vite tree-shakes the entire module out of production builds. This is scoped to S1 — we're not overhauling the existing feature flag patterns.
• 2D canvas (BoardCanvas): Raw HTML <canvas> via useRef<HTMLCanvasElement>. Capture is trivial — canvasRef.current.toDataURL() gives a PNG directly.
• 3D canvas (Preview3D/Scene3D): @react-three/fiber wrapping Three.js. WebGL renderer (gl) is accessible inside the R3F <Canvas> context via useThree(). Capture via gl.domElement.toDataURL(), but requires calling from inside R3F context. Needs camera positioning to standard orthographic angles before capture, then restore.
• Measurement engine (F1-S6): Scope is the programmatic API only (measureDistance, measureFromEdge). The in-app measurement tool UI (toolbar mode, click-to-measure, visual overlays) is deferred to a separate epic.

F3: Test Fixture Library — Stories#

Dependency graph:

graph TD
    S1["F3-S1: Fixture Schema,<br/>Types & Validation"] --> S2["F3-S2: Core Operation<br/>Fixtures + Baselines"]
    S1 --> S3["F3-S3: Edge Treatment<br/>Fixtures + Baselines"]
    S1 --> S4["F3-S4: Multi-Operation<br/>Fixture + Baseline"]
    S2 --> S5["F3-S5: Fixture<br/>Validation Script"]
    S3 --> S5
    S4 --> S5

    S5 -.-> F4["F4: G-code Integration Tests"]
    S5 -.-> F5["F5: Visual Regression"]
    S5 -.-> F7["F7: Physical Validation"]

    style S1 fill:#e67e22,color:#fff
    style S2 fill:#3498db,color:#fff
    style S3 fill:#3498db,color:#fff
    style S4 fill:#3498db,color:#fff
    style S5 fill:#e67e22,color:#fff
    style F4 fill:#555,color:#fff
    style F5 fill:#555,color:#fff
    style F7 fill:#555,color:#fff

🟠 Orange = serial (must happen in order). 🔵 Blue = parallelizable after S1.

Design Notes#

• Fixture format: Each fixture is a standalone JSON file in src/__fixtures__/ containing board config, shapes, tool settings, and optionally edge treatments. The JSON schema matches the TestFixture type from F1-S1.
• Baseline convention: Each fixture {name}.fixture.json has a corresponding {name}.expected.nc in the same directory. Fixtures without baselines (e.g., blocked on a bug) have no .expected.nc file — the validation script skips baseline assertions for these.
• Baseline generation: Baselines are generated via window.__routr — load fixture → generateGcode() → write output as .expected.nc. This means baselines reflect the current G-code pipeline output. When the pipeline changes intentionally, baselines are regenerated. When it changes unintentionally, F4's integration tests catch the drift.
• straight-cut-basic is blocked: The table saw kerf line flip bug means this fixture's baseline would encode incorrect behavior. Create the fixture JSON (the board config is valid), skip the baseline. Revisit after the kerf fix ships.

F4: G-code Integration Tests — Stories#

Dependency graph:

graph TD
    S1["F4-S1: G-code Test Harness<br/>(Vitest, auto-discovery)"] --> S2["F4-S2: Test Report &<br/>Dev Workflow Integration"]

    F3_S5["F3-S5: Fixture Validation<br/>Script (baseline mgmt)"] -.-> S1
    F1_S3["F1-S3: G-code Generation<br/>Hooks"] -.-> S1

    S2 -.-> WORKFLOW["WORKFLOW.md<br/>Agent Template Update"]

    style S1 fill:#e67e22,color:#fff
    style S2 fill:#e67e22,color:#fff
    style F3_S5 fill:#555,color:#fff
    style F1_S3 fill:#555,color:#fff
    style WORKFLOW fill:#555,color:#fff

Both stories are serial — S2 builds on S1. Only two stories because the scope is focused: no CI, baseline management lives in F3.

Design Notes#

• No GitHub Actions CI. All tests run locally as part of sub-agent verification. CI deferred to a Future ticket (see CI/CD Strategy section and Decisions Log). The test infrastructure is CI-ready if/when we add it — Vitest, no browser, deterministic.
• Baseline management is F3's responsibility. F4 reads .expected.nc baselines but does not create or update them. To regenerate baselines after intentional changes: npm run validate-fixtures -- --generate (F3-S5's script). Clean separation: F3 owns fixtures + baselines, F4 owns assertions + reporting.
• Dynamic fixture discovery. The test harness auto-discovers *.fixture.json files in src/__fixtures__/. Adding a new fixture + baseline automatically adds a test case. Zero boilerplate.
• Fixtures without baselines are skipped, not failed. straight-cut-basic (and any future fixture blocked on a bug) appears in the report as ⏭️ SKIPPED — No baseline rather than pass or fail. When the baseline is generated, the test automatically activates.

F5: Visual Regression + Cross-Layer Tests — Stories#

Dependency graph:

graph TD
    S1["F5-S1: Playwright Infrastructure<br/>& Screenshot Capture"] --> S2["F5-S2: Visual Regression<br/>Test Suite"]
    S3["F5-S3: Cross-Layer<br/>Coordinate Assertions"] --> S4["F5-S4: GitHub Actions<br/>Visual CI"]
    S2 --> S4

    F1_S5["F1-S5: Canvas Capture"] -.-> S1
    F1_S6["F1-S6: Measurement Engine"] -.-> S3
    F3["F3: Fixture Library"] -.-> S2
    F3 -.-> S3
    F4_S1["F4-S1: G-code Test Harness"] -.-> S3

    style S1 fill:#e67e22,color:#fff
    style S2 fill:#e67e22,color:#fff
    style S3 fill:#3498db,color:#fff
    style S4 fill:#e67e22,color:#fff
    style F1_S5 fill:#555,color:#fff
    style F1_S6 fill:#555,color:#fff
    style F3 fill:#555,color:#fff
    style F4_S1 fill:#555,color:#fff

🟠 Orange = Playwright chain (serial — S1 → S2 → S4). 🔵 Blue = S3 is independent (Vitest, no browser) and can run in parallel with S1/S2.

Design Notes#

• F6 dependency removed. The measurement engine (measureFromEdge, measureDistance) shipped in F1-S6. F5's cross-layer assertions use the programmatic API, not the measurement tool UI. F6 (UI) is fully decoupled — separate epic.
• Cross-layer assertion model: Three sources of truth that must agree: ① measureFromEdge() returns the design intent (shape position relative to board edge), ② G-code output contains the execution positions (parsed X/Y/Z coordinates), ③ Screenshots show the visual position in both design and sim tabs. F5-S3 asserts ① vs ② programmatically. F5-S2 captures ③ for visual verification. A coordinate flip bug (like the kerf line flip) would cause ① and ② to disagree — the shape is at X=100 from the left, but the G-code moves to X=100 from the right.
• No dedicated toolpath measurement API. Parsing G-code X/Y positions and comparing against measureFromEdge() achieves cross-layer detection without a new API. If this proves too brittle after F5 ships, a measureToolpathFromEdge() API can be added as a follow-up.
• Visual tests run on every PR via GitHub Actions. Even non-UI changes can shift rendering (toolpath algorithm changes, coordinate fixes). The whole point is catching unexpected visual changes. 30-60 seconds is acceptable overhead.
• Golden baselines committed to git. ~21 PNG files (~5-10MB total) in src/__fixtures__/ alongside fixture JSON and .expected.nc files. Manageable at current fixture count. Revisit with Git LFS if fixture count grows significantly. Diff images are ephemeral GitHub Actions artifacts (auto-expire, no repo bloat).
• Pixel-diff tolerance. Anti-aliasing causes 1-2% pixel variation across renders. Tests use a tolerance threshold (configurable per fixture if needed). pixelmatch library handles this — same library Playwright uses internally.

F2: MCP Server (routr-tools) — Stories#

Dependency graph:

graph TD
    S1["F2-S1: Package Scaffold<br/>& MCP Server Foundation"] --> S2["F2-S2: Playwright<br/>Browser Bridge"]
    S2 --> S3["F2-S3: Project & Design<br/>Tools (4 tools)"]
    S2 --> S4["F2-S4: Generation, State<br/>& Validation Tools (5 tools)"]
    S2 --> S5["F2-S5: Capture &<br/>Measurement Tools (3 tools)"]

    F1["F1: AI-Accessible Interface<br/>(window.__routr)"] -.-> S2

    S3 --> TEST["All tools registered<br/>& testable"]
    S4 --> TEST
    S5 --> TEST

    style S1 fill:#e67e22,color:#fff
    style S2 fill:#e67e22,color:#fff
    style S3 fill:#3498db,color:#fff
    style S4 fill:#3498db,color:#fff
    style S5 fill:#3498db,color:#fff
    style F1 fill:#555,color:#fff
    style TEST fill:#555,color:#fff

🟠 Orange = serial (S1 → S2 must happen in order). 🔵 Blue = S3/S4/S5 parallelizable after S2 (with worktrees!).

Design Notes#

• Separate package: packages/routr-mcp/ with its own package.json, tsconfig.json, build config. Monorepo structure — the app doesn't depend on this package. Independent versioning.
• MCP SDK: @modelcontextprotocol/sdk — official TypeScript SDK. Handles protocol plumbing (JSON-RPC, tool registration, transport). We write tool definitions + handlers, SDK does the rest.
• Transport: stdio only for v1. Server is spawned as a child process by MCP clients (Claude Desktop, OpenClaw). Config entry is just { "command": "node", "args": ["packages/routr-mcp/dist/index.js"] }.
• Playwright bridge: The MCP server launches a Chromium browser via Playwright, navigates to the running Routr dev server, and executes tool handlers via page.evaluate(() => window.__routr.someMethod(...)). All 13 tools go through the browser — headed-only for v1.
• Dev-only: MCP server is a dev dependency. Not in staging, not in production. Same category as Vitest and Playwright. Does not affect app builds or deployment.
• Tool descriptions are the documentation: Each tool carries rich descriptions and parameter schemas as part of the MCP protocol. When a client connects, it receives the full tool catalog with context — the LLM knows when, why, and how to use each tool without external docs.
• Lightweight testing: Tools are thin wrappers over window.__routr (already tested in F1). MCP layer tests verify: tool registration, parameter validation/schema correctness, and that handlers call the right window.__routr methods. Don't re-test engine logic.
• Tool grouping rationale: Stories split by conceptual domain rather than dependency chain because: (a) all tools depend equally on the Playwright bridge (S2), (b) domain grouping keeps related tool descriptions and schemas together for coherent review, (c) enables parallel execution with worktrees after S2 ships.

Tool Surface Reference#

Decisions Log#

Future Work: MCP Tool Surface Expansion#

Captured March 25, 2026 — context: protagonist/antagonist agentic testing pattern.

The current 12-tool surface is write-heavy (create, set, generate). For the antagonist agent pattern to work effectively, we need read-back, mutation, and chaos tools. The protagonist needs tools to build; the antagonist needs tools to inspect, mutate, and break.

Proposed Additional Tools#

Guiding Principle#

Protagonist = write tools (build projects, generate output). Antagonist = read + chaos tools (inspect state, mutate, break things). The current surface is ~80% protagonist. Expanding read-back and mutation tools enables the adversarial testing pattern described in the epic design notes.

Implementation Notes#

• Each new tool needs a corresponding window.__routr method in the devApi
• Read-back tools (get_shape, list_shapes, get_tool_settings) are thin — just expose existing Zustand state
• Mutation tools (resize_board, move_shape, delete_shape) need new store actions
• Scope as a follow-up epic or add incrementally as the antagonist pattern is validated

Known Issues / Tech Debt#

This epic doc is refined collaboratively (Step 0) before stories are broken down (Step 1). Once refined, the AI Lead extracts context from this doc to craft sub-agent prompts (Step 2). Update this doc as implementation reveals new information — design docs are living documents.