Skip to main content

Add critical-path CI gate to block unverified merges

Status

DECIDED

Context

On 2026-04-14, an agent shipped a Stripe harness refactor that passed build, passed type checks, and merged cleanly. The webhook started returning HTTP 200 on silent provisioning failures — a live paying customer's church was never provisioned.

The agent feedback memory already said: "NEVER claim done without Playwright- verified interaction on the deployed URL." It had no enforcement mechanism. Agents could acknowledge the rule and still ship unverified code. The memory was advisory; merge was not blocked.

Multiple flows exist where "build passes" is a false signal. TypeScript is satisfied by code that returns 200 from a handler that swallows exceptions. The only signal that counts for customer-money and customer-data flows is an observed, end-to-end, verified interaction.

Decision

Install a critical-path-gate workflow in churchwiseai-web's GitHub Actions.

On every PR, the gate:

  1. Reads knowledge/tests/registry.yaml (source of truth for critical paths)
  2. Identifies entries with critical_path: true for this repo
  3. Compares PR changed files against each entry's code_files list
  4. If the PR touches a critical-path file, requires one of:
    • A CI artifact named <spec_name>-playwright-run.zip showing a green run
    • The PR label critical-path-override with a comment explaining why
  5. Blocks merge if neither is present

Critical paths are declared in the registry, not hardcoded in the workflow. Adding a new critical path is a registry edit, not a workflow change.

Initial gated path (2026-04-14): stripe-live-checkout — live Stripe checkout + webhook provisioning in churchwiseai-web.

Demo PR #3 confirmed the gate blocks without the artifact.

Rationale

Enforcement must be mechanical, not memorial. Agents have the feedback, follow it in some sessions, forget it in others. A gate that blocks merge is the only reliable enforcement for a rule that must be followed every time.

The override mechanism exists for genuinely safe changes to critical-path files (updating a code comment, renaming an unused variable). Overrides are logged and auditable. The expected use is rare.

The registry pattern keeps the gate extensible: when voice-live-call and user-login-cwa have their Playwright specs passing, flipping critical_path: true in the registry is sufficient to gate them — no workflow changes needed.

Consequences

  • Good: Agents cannot merge unverified critical-path changes. The incident that triggered this decision cannot recur through the same mechanism. Overrides are visible and logged.
  • Bad: PRs that touch critical-path files require a Playwright run against the deployed URL before merge. This adds time — the spec must run, upload an artifact, and the gate must read it. Not instantaneous.
  • Reversible? Yes — the workflow can be removed or disabled. Not recommended while critical-path flows handle live money.

Alternatives considered

  • Branch protection rules only — rejected; branch protection can be overridden by the founder and doesn't enforce the specific evidence requirement (Playwright artifact vs. just "CI green").
  • Require PR reviews for critical-path files — rejected; the founder does not review PRs before merging (per explicit founder preference). A human review gate would be bypassed by the founder and wouldn't verify end-user interaction anyway.
  • Post-merge monitoring only — rejected; by the time monitoring catches a failure, a customer has already been harmed. The 2026-04-14 incident is the proof.
  • Process: knowledge/processes/critical-path-definition.md
  • Registry: knowledge/tests/registry.yaml
  • Code: churchwiseai-web/.github/workflows/critical-path-gate.yml
  • Related decision: 2026-04-14-stripe-webhook-inbox
  • Memory: ~/.claude/projects/C--dev/memory/feedback_no_half_assed_work.md