Knowledge > Integrations > OpenAI

OpenAI Integration

OpenAI serves two distinct roles in the ChurchWiseAI portfolio: it is the sole provider for vector embeddings (text-embedding-3-small) used across all three codebases to power RAG search, and it is the automatic fallback LLM (gpt-4o-mini) for the churchwiseai-web chatbot when Anthropic is unavailable. In PewSearch and IllustrateTheWord, gpt-4o-mini also acts as the primary LLM for lightweight chat features (directory chatbot, demo chat, illustration personalization) since those codebases do not use the Anthropic SDK directly.

Account Info

Field	Value
Account email	churchwiseai@gmail.com
Console	https://platform.openai.com
Billing	Pay-as-you-go (usage-based)
Priority	P1 — important but not blocking (churchwiseai-web chatbot degrades to static FAQ; embeddings cause RAG to fail silently)

How Used Per Product

Product / Codebase	Purpose	Models Used
churchwiseai-web (chatbot)	Automatic LLM fallback when Anthropic returns 5xx or times out	gpt-4o-mini
churchwiseai-web (RAG)	Embedding generation for vector search in unified_rag_content and church knowledge base	text-embedding-3-small
pewsearch/web (chatbot)	Deprecated — church chatbot now unified through CWA /api/chatbot/stream (Claude Haiku 4.5 primary)	N/A (was gpt-4o-mini)
pewsearch/web (AI search)	Church directory natural-language search	gpt-4o-mini
pewsearch/web (support chat)	Support chat + embedding search for churches in directory	gpt-4o-mini + text-embedding-3-small
pewsearch/web (demo chat)	Demo chatbot for unauthenticated visitors	gpt-4o-mini
sermon-illustrations (RAG)	Embedding generation for illustration vector search	text-embedding-3-small
sermon-illustrations (support chat)	Illustration support + personalize endpoints (with Claude Haiku as primary fallback)	gpt-4o-mini
sermon-illustrations (personalize)	Personalize illustration copy for a pastor's context	gpt-4o-mini

LLM Routing Detail — churchwiseai-web Chatbot

The chatbot uses a dual-provider architecture defined in churchwiseai-web/src/lib/llm-provider.ts:

• Primary: Anthropic Claude Haiku 4.5 (claude-haiku-4-5-20251001)
• Escalation: Anthropic Claude Sonnet 4.6 (claude-sonnet-4-6) when escalate: true
• Fallback: OpenAI gpt-4o-mini — activates automatically only when Anthropic returns a 5xx error or times out

gpt-4o-mini is NOT a co-primary for churchwiseai-web chatbot; it is a safety net. When fallback activates, an email alert fires (rate-limited to 1 per 15 minutes) to notify the founder that Anthropic is degraded.

Embedding Pipeline

All three codebases call /v1/embeddings directly via fetch. The embedding model is always text-embedding-3-small. Generated vectors are stored in the unified_rag_content Supabase table (327K records) and queried via pgvector similarity search (match_rag_content RPC). If OPENAI_API_KEY is missing, generateEmbedding() returns null and RAG search returns an empty array — the system degrades gracefully without throwing.

Key Environment Variables

Variable	Scope	Used By
OPENAI_API_KEY	Server-side only	churchwiseai-web (chatbot, RAG), pewsearch/web (chatbot, AI search, support, demo, RAG), sermon-illustrations (support chat, personalize, RAG backfill scripts)

The key is server-side in all cases. It is never exposed to the browser. In Next.js, it is used exclusively inside API routes and server-side lib modules.

CLI Patterns & Gotchas

Embedding backfill scripts:

• ITW: sermon-illustrations/scripts/backfill-embeddings.mjs — generates embeddings for new illustration records
• These scripts call the OpenAI REST API directly (not via the SDK) using node-fetch or native fetch

No OpenAI SDK installed in churchwiseai-web:

• The LLM fallback in llm-provider.ts calls the OpenAI REST API via fetch directly — the openai npm package is NOT in use in churchwiseai-web
• PewSearch and ITW also use raw fetch to the OpenAI completions endpoint

gpt-4o-mini is NOT gpt-4o:

• All codebases consistently use gpt-4o-mini (not gpt-4o or gpt-4.1) for cost control
• The one exception: PewSearch pewsearch/web/src/app/api/demo/chat/route.ts uses gpt-4o-mini; the AI search route uses gpt-4o-mini as well
• ITW chat/support/route.ts uses gpt-4.1-nano (the most compact model) for the chat completion step but falls back to Claude Haiku for quality-sensitive paths

Missing key behavior:

• If OPENAI_API_KEY is not set, churchwiseai-web chatbot still works (falls back to Anthropic is primary)
• If both OPENAI_API_KEY and ANTHROPIC_API_KEY are missing from churchwiseai-web, the chatbot route returns HTTP 503 immediately
• PewSearch chatbot and AI search return an error if OPENAI_API_KEY is missing (no fallback configured)

Failure Modes & Recovery

Failure	Symptom	Recovery
API key missing (churchwiseai-web)	Chatbot still works via Anthropic primary; embeddings return null, RAG returns empty	Add OPENAI_API_KEY via Vercel env add
API key missing (PewSearch)	Chatbot widget returns error message; AI search fails	Add OPENAI_API_KEY to PewSearch Vercel env
API key missing (ITW)	Support chat and personalize return errors; backfill scripts fail	Add OPENAI_API_KEY to ITW Vercel env
Rate limit / 429	Embedding generation returns null; RAG degrades to no-context reply	Temporary; no action needed. Monitor usage dashboard.
Anthropic 5xx in churchwiseai-web	Auto-fallback to gpt-4o-mini; founder receives email alert	Investigate Anthropic status page; fallback runs automatically
OpenAI 5xx in churchwiseai-web	LLM fallback fails; chatbot returns generic error	Rare; check https://status.openai.com

Monitoring: The founder dashboard (churchwiseai-web/src/app/founder/[token]/components/ProductionMonitoring.tsx) includes an OpenAI status card with a link to the OpenAI status page.

Cost Model / Usage Limits

Usage Type	Model	Approximate Cost
Embeddings	text-embedding-3-small	$0.02 per million tokens ($0.00002/query)
LLM fallback (rare)	gpt-4o-mini	~$0.15 / $0.60 per M input/output tokens
PewSearch support/demo/search chat	gpt-4o-mini	Per conversation; low volume at launch
ITW personalize	gpt-4o-mini	Per personalization request

Key cost note: OpenAI costs are expected to remain very low because:

1. The churchwiseai-web chatbot primary LLM is Anthropic — OpenAI only runs on failover
2. Batch content generation (sermon illustrations, scrapers) uses the Claude CLI (claude -p), NOT the OpenAI API — zero OpenAI cost for batch work
3. text-embedding-3-small is one of the cheapest embedding models available

Supabase pgvector stores all vectors; re-embedding only happens when new content is added. The 327K records in unified_rag_content were embedded once during content ingestion.

See Also

• Anthropic Integration — primary chatbot LLM; used when OpenAI is fallback
• Google AI Integration — Gemini 2.5 Flash for voice agent
• Infrastructure Reference — full env var matrix and service dependencies
• Supabase Integration — pgvector storage for embeddings (unified_rag_content)