Skip to main content

Knowledge > Runbooks > Voice Ops > Voice Call Quality Review

Voice Call Quality Review

Systematically review voice call quality for a church, identify agent performance gaps, and implement improvements through prompt tuning or code updates.

Prerequisites​

  • Supabase MCP or direct DB access
  • WSL terminal and Cartesia deploy access (if code changes needed)
  • At least 10 recent calls to review for meaningful patterns

Steps​

  1. Pull recent calls for the church:

    SELECT
      id,
      created_at,
      duration_seconds,
      call_outcome,
      sentiment_score,
      urgency_level,
      gemini_summary,
      transcript
    FROM voice_call_logs
    WHERE church_id = '[uuid]'
      AND created_at >= now() - interval '30 days'
    ORDER BY created_at DESC
    LIMIT 20;

  2. Review Gemini post-call classification fields for patterns:

    • sentiment_score — negative scores indicate caller frustration
    • urgency_level — frequent high urgency may indicate the agent is missing important context
    • gemini_summary — quick summary of what the caller needed and whether it was resolved

  3. Read transcripts for calls with negative sentiment or poor outcomes: Focus on:

    • Questions the agent could not answer (missing knowledge)
    • Incorrect tool usage (prayer request tool triggered for non-prayer topics)
    • Unhelpful loops or repetitive responses
    • Calls where the caller gave up and hung up
    • Audio issues noted in the transcript (e.g., silence periods, repeated prompts)

  4. Identify the improvement category:

    Issue TypeFix Location
    Church-specific info wrong (hours, pastor name, address)church_voice_agents.system_prompt_override
    General agent behaviorverticals/church/prompts.py
    Tool not triggering correctlyverticals/church/tools.py
    Noise/silence filtering too aggressiveturn_processor.py thresholds
    Portfolio-wide prompt improvementverticals/church/prompts.py + redeploy

  5. Fix church-specific issues (no redeploy required):

    UPDATE church_voice_agents
    SET system_prompt_override = '[Improved prompt for this church. Include accurate service times, pastor names, programs, and any caller patterns you observed.]',
        updated_at = now()
    WHERE church_id = '[uuid]';

    Wait up to 5 minutes for session cache to expire before testing.

  6. Fix portfolio-wide issues (requires redeploy):

    • Edit verticals/church/prompts.py or turn_processor.py in a feature branch.
    • Test with python -m py_compile to catch syntax errors.
    • Deploy via WSL following voice-agent-update.md.

  7. Test improvements by placing calls to the church number and evaluating:

    • The specific issue is resolved.
    • No regressions in other call types.
    • New call logs show improved sentiment_score and call_outcome.

  8. Document improvements if the change was portfolio-wide, so other churches benefit: Add a note in C:\dev\DECISION_LOG.md with the pattern identified and fix applied.

Verification​

  • Targeted call scenarios now produce correct responses.
  • voice_call_logs for new calls show improved call_outcome and sentiment_score.
  • No regressions in other church accounts (spot-check 2–3 other churches' call logs).

See Also​