Skip to main content

Phase 3 Handoff — Fix Specs + Generate E2E Tests

Date: 2026-03-28 Status: Complete (2026-04-16) Previous session: Completed Phase 1 (write all specs) + Phase 2 (expert review) + founder decisions on all flagged items.

What Was Accomplished​

Phase 1: 11 Acceptance Specs Written​

SpecFileSource
Starter Chatacceptance/starter-chat.mdFounder interview (2026-03-26)
Pro + Suite Chatacceptance/pro-suite-chat.mdFounder interview (2026-03-27)
Starter Voiceacceptance/starter-voice.mdFounder interview (2026-03-28)
Starter Bothacceptance/starter-both.mdFounder (Q1-12) + agent (Q13-19)
Pro Bothacceptance/pro-both.mdAgent-derived
Suite Bothacceptance/suite-both.mdAgent-derived
Trial Expiredacceptance/trial-expired.mdAgent-derived
Cancelledacceptance/cancelled.mdAgent-derived
ITW Premiumacceptance/itw-premium.mdAgent from code analysis
PewSearch Premiumacceptance/pewsearch-premium.mdAgent from code analysis
PewSearch Pro Websiteacceptance/pewsearch-pro-website.mdAgent from code analysis (delta spec)
SermonWise Proacceptance/sermonwise-pro.mdAgent from code analysis

Phase 2: 3 Expert Review Reports​

ReportFileKey Findings
QA Consistencydecisions/phase2-qa-consistency-report.md8 critical, 12 warnings
DBA + Securitydecisions/phase2-dba-security-report.md5 schema mismatches, 7 migrations, 6 security concerns
Voice Engineeringdecisions/phase2-voice-review-report.md7 inaccuracies, 6 unverifiable

Founder Decisions (All Resolved)​

#DecisionAnswer
1Phone number retention after cancellation30 days — approved
2Bundle cancellation suggests standalone plansYes
3"Export My Data" button on expired dashboardBuild now, then show
4Win-back email suppression after 2+ cancelsYes, suppress
5Embed widget for ALL tiers including StarterYes, confirmed
64 agents or 2?Chatbot=4 (built). Voice=2 (Care + Coordinator). Stewardship voice config fix needed (change to chat-only). Coordinator handles giving for voice.
7ITW Favorites/CollectionsGated to Premium (code fix needed)
8PewSearch overlapping subscriptionsEnd Premium billing at period end, CWA Pro absorbs it
9SermonWise "sermon series planning"Remove from pricing page, backlog the build

Phase 3: What Needs to Happen​

3A: Fix Specs Based on Review Findings​

Use Sonnet subagents for each spec fix batch. Main agent (Opus) orchestrates.

Critical Fixes from QA Report (8 items)​

  1. C1: features.yaml agent count — Says 2 for Pro/Suite, should say 4 (chatbot). Voice stays at 2. Update features.yaml.
  2. C2: Role count — Specs say 9 roles, features.yaml says 7. Verify actual code in premium-shared.ts and align.
  3. C3: Starter Chat Training sub-tabs — Spec says 6, code has 5. Fix starter-chat.md to say 5 (Church Knowledge, This Week, Theology, Agents, Safety). Remove "Training Progress" as a separate sub-tab if it's not one.
  4. C4: Starter Chat Getting Started checklist — Has 3 items but omits theological lens step that Pro has. Verify if Starter should also have it (founder added it for Starter Voice).
  5. C5: features.yaml Pro persona_label — "Full Voice + Care Automation" is wrong for chat plans. Fix per-channel labels.
  6. C6: features.yaml Pro features — Incorrectly says "Pro Website included." Pro gets Premium Listing ($9.95 value). Suite gets Premium Listing + Pro Website. Fix.
  7. C7/C8: Tab ordering — Care and Training swap positions between Starter and Pro specs. Standardize order across all specs. The correct order (from live Zewdei dashboard): Overview, Calls, Requests, Care, Training, Settings, Upgrade.

Critical Fixes from DBA Report (5 schema mismatches → 7 migrations)​

These don't change specs, but the migration SQL needs to be created and run:

  1. Add source column to voice_prayer_requests, voice_callback_requests, voice_visitor_contacts
  2. Add phone_provider column to premium_churches (or organization_settings)
  3. Add cancellation_reason column to premium_churches
  4. Add cancel_at_period_end and current_period_end columns to premium_churches
  5. Add is_hidden (soft delete) column to voice_prayer_requests, voice_callback_requests, voice_visitor_contacts, voice_call_logs

Migration SQL is already drafted in the DBA report. Review and apply via Supabase MCP.

Critical Fixes from Voice Report (7 inaccuracies)​

  1. Agent count in voice specs — Pro/Suite voice specs say 4 agents. Should say 2 for voice (Care + Coordinator). Chatbot gets 4. Fix all voice and bundle specs.
  2. Stewardship channel config — Change channels: ['chat', 'voice'] to ['chat'] in agent-type-config.ts.
  3. Subscription status check missing — Voice agent load_church_data() doesn't check if subscription is active. Critical pre-launch fix.
  4. "Line is live" Email 2 — Not built yet. Flag as pre-launch blocker (already in action items).
  5. Tool counts not enforced in voice — All voice tools available at all tiers. Voice specs should note this or tier-gating should be added.
  6. Stale knowledge docs — add-voice-agent.md and tools.md reference old Cartesia/Twilio patterns. Update.
  7. Voice specs reference non-existent voice-specific settings — Verify what actually exists in the admin UI for voice config.

Fixes from Founder Decisions​

  1. Embed widget: Update starter-chat.md to show embed as available (was marked HIDDEN/Pro+). Update tier-config.ts if needed.
  2. SermonWise: Remove "sermon series planning" from pricing page.
  3. ITW: Gate Favorites/Collections to Premium in code.
  4. PewSearch: Design overlapping subscription reconciliation logic.

Warnings from QA Report (12 items — should fix)​

  • features.yaml missing conversation limits for Starter (200) and Pro (500)
  • Embed widget tier conflict between specs
  • Suite Both savings claim math
  • Care broadcast limits missing from features.yaml
  • Starter Chat Touchpoint 13 PewSearch bonus contradiction
  • (See full list in phase2-qa-consistency-report.md)

3B: Generate E2E Playwright Tests​

After spec fixes are applied, generate Playwright tests from each spec.

Each spec's touchpoints include "Playwright Test Hints" — use these as the test skeleton.

Test Organization​

churchwiseai-web/e2e/journeys/
  starter-chat.spec.ts
  pro-suite-chat.spec.ts
  starter-voice.spec.ts
  starter-both.spec.ts
  pro-both.spec.ts
  suite-both.spec.ts
  trial-expired.spec.ts
  cancelled.spec.ts

sermon-illustrations/e2e/journeys/
  itw-premium.spec.ts

pewsearch/web/e2e/journeys/
  pewsearch-premium.spec.ts
  pewsearch-pro-website.spec.ts

churchwiseai-web/e2e/journeys/
  sermonwise-pro.spec.ts

Test Strategy​

  • Tests run against production URLs (founder's preference — see feedback_test_against_production.md)
  • Use test accounts from C:\dev\TEST_URLS.md
  • Never write test data to production DB (see feedback_no_qa_writes_to_prod.md)
  • Read-only assertions: check what's visible/hidden, verify URLs, check element counts
  • For checkout flows: test up to Stripe page (don't complete purchase)

3C: Run Tests + Fix Failures​

Run all generated Playwright tests. For failures:

  • If spec is wrong → fix spec
  • If code is wrong → fix code
  • If test is flaky → fix test

Execution Plan for Next Session​

Orchestration Model​

Main agent (Opus) orchestrates. Subagents (Sonnet) execute in parallel.

Step 1: Fix Specs (3 parallel Sonnet subagents)​

  • Subagent A: Fix CWA tier specs (starter-chat, starter-voice, starter-both, pro-both, suite-both) based on QA + Voice review findings
  • Subagent B: Fix Pro+Suite Chat spec + lifecycle specs (trial-expired, cancelled) based on QA findings
  • Subagent C: Fix cross-property specs (ITW, PewSearch, SermonWise) + update features.yaml

Step 2: Apply DB Migrations (1 Sonnet subagent)​

  • Subagent D: Review migration SQL from DBA report, apply via Supabase MCP, verify columns exist after

Step 3: Code Fixes (2 parallel Sonnet subagents)​

  • Subagent E: Fix agent-type-config.ts (Stewardship channels), tier-config.ts (embed widget for Starter), SermonWise pricing page (remove sermon series planning)
  • Subagent F: Fix ITW Premium gating (Favorites/Collections), update stale knowledge docs

Step 4: Generate E2E Tests (3 parallel Sonnet subagents)​

  • Subagent G: Generate CWA tier journey tests (starter-chat, starter-voice, starter-both)
  • Subagent H: Generate CWA tier journey tests (pro-both, suite-both, pro-suite-chat, trial-expired, cancelled)
  • Subagent I: Generate cross-property journey tests (ITW, PewSearch, SermonWise)

Step 5: Run Tests + Fix (Opus orchestrates, Sonnet fixes)​

  • Main agent runs all test suites
  • Failures triaged and dispatched to Sonnet subagents for fixes
  • Re-run until green or all failures documented

Step 6: Founder Boardroom Review​

  • Present: all specs finalized, test results, remaining action items
  • Update: INTERVIEW-QUEUE.md, README.md, DECISION_LOG.md

Key Files for Next Session​

Specs (to fix)​

  • C:\dev\knowledge\acceptance\*.md (all 11 spec files)

Review Reports (to read)​

  • C:\dev\knowledge\acceptance\decisions\phase2-qa-consistency-report.md
  • C:\dev\knowledge\acceptance\decisions\phase2-dba-security-report.md
  • C:\dev\knowledge\acceptance\decisions\phase2-voice-review-report.md

Action Items​

  • C:\dev\knowledge\acceptance\decisions\interview-action-items.md (11 items with priorities)

Config Files (to fix)​

  • C:\dev\churchwiseai-web\src\lib\agent-type-config.ts (Stewardship channels)
  • C:\dev\churchwiseai-web\src\lib\tier-config.ts (embed widget gating)
  • C:\dev\knowledge\data\features.yaml (agent counts, role counts, conversation limits, Pro features)

Reference​

  • C:\dev\PRICING.md
  • C:\dev\TEST_URLS.md
  • C:\dev\knowledge\acceptance\decisions\starter-voice-interview-answers.md
  • C:\dev\knowledge\acceptance\decisions\starter-both-interview-answers.md
  • C:\dev\knowledge\acceptance\decisions\pro-chat-interview-answers.md

Prompt to Start Next Session​

Read C:\dev\knowledge\acceptance\decisions\PHASE3-HANDOFF.md and execute Phase 3.

Use Sonnet subagents for parallel execution. Main agent (Opus) orchestrates.
Start with Step 1 (fix specs) — launch 3 parallel Sonnet subagents as described.