Conversation quality

BMO treats conversation quality as a staged control problem, not a single prompt-engineering trick. Different features observe or shape different phases of a turn. This guide orients you across those phases so you can pick the smallest surface that answers your question.

Arena belongs here as a during-turn decision forum. It runs inside the current session, but session continuity, checkpoints, and restore decisions stay with the Sessions family.

flowchart LR
  subgraph beforeTurn [Before model]
    enhancer[Prompt Enhancer]
    stack[Prompt Stack]
  end
  subgraph duringTurn [During API turn]
    orch[Quality orchestration]
    arena[Arena]
  end
  subgraph afterTurn [After draft]
    gates[Quality Gates]
  end
  subgraph overTime [Across sessions]
    shadow[Shadow Evals]
  end
  enhancer --> stack
  stack --> orch
  orch --> gates
  arena -.->|"accept winner"| gates
  gates --> shadow
  stack -.->|"observe input"| shadow
  gates -.->|"rubric on recipes"| shadow

Lifecycle phases

Phase	Reach for…	Feature page
Before the model sees the turn	What segments fired; full trace in Context Lineage	Prompt Stack
During an API infra/lifecycle turn	Which candidates ran and which answer won	Quality orchestration
During turn (operator comparison)	Same prompt, multiple models; accept winner	Arena
After a draft exists	Rubric verdict on answers, plans, recipes, proposals	Quality Gates
Across sessions and changes	Deterministic eval compare + shadow posture	Shadow Evals

Prompt Enhancer shapes user-message context before assembly; Prompt Stack shows how the system prompt was built on top of that.

Runtime authority

Authority	Features
Read-only / metadata	Prompt Stack ring, Quality Gates posture readouts
Reshapes live API answer	Quality orchestration (one assistant message out; TUI chat unchanged)
Advisory by default	Quality Gates (send_back at most once when armed)
Never changes live response	Shadow Evals

Entry surface

If you work in…	Start with…
TUI	/prompt-stack, /quality-gates, /arena, /debugger
Open WebUI / OpenAI-compat API	Quality orchestration + run ledger
CI / maintainer	bmo eval posture, bmo quality-gate check, bmo eval compare

Walkthrough: infra answer regression

Setup: You use Open WebUI against BMO’s OpenAI-compatible API. After changing a prompt prefix for observability Q&A, answers sound like internal backend jargon instead of VictoriaMetrics-grounded responses.

Step	Phase	Action	Question answered
1	Observe input	/prompt-stack → Context Lineage	Which segments fired? Was infra route context included or shed?
2	Observe during-turn	Agent Debugger: orchestration_run_id, candidate_id, judge_decision, final_answer_origin	Did fanout run? Which candidate won?
3	Verify draft	/quality-gates + debugger quality_judge_* events	Did the answer gate pass? Was the judge unavailable vs pass?
4	Measure change	bmo eval run → compare → eval posture; optional shadow variants	Did the prompt change regress scenarios? Is shadow evidence fresh?
5	Harden artifact	bmo quality-gate check --surface prompt_recipe --path …	Does the recipe meet hardening rubric before merge?

Pick your job

Response surprised me?
  → Prompt Stack → Context Lineage → Debugger timeline

API infra/lifecycle answer wrong?
  → Debugger orchestration metadata → Quality orchestration page

Draft looks weak / policy violation?
  → Quality Gates → debugger quality_judge_* events

Changed prompts, routing, or provider?
  → eval compare (blocking) → eval posture (shadow advisory)
  → optional: quality-gate check on prompt_recipe files

Naming disambiguation

Term	Means	Not
Quality Gates	Post-draft rubric judge (/quality-gates)	Shadow maintainer evidence recipe
Maintainer evidence recipe (shadow-evals)	bmo eval run + compare + posture	bmo quality-gate check
Quality orchestration	Server multi-candidate fanout for infra/lifecycle	Adaptive orchestration, team lifecycle hooks

Related feature pages

• Prompt Stack — metadata-only assembly ring and Context Lineage handoff
• Quality orchestration — API fanout select/compose for infra and lifecycle
• Quality Gates — post-draft judge and manual plan/recipe checks
• Arena — multi-model comparison and accept into session
• Shadow Evals — capture, replay, proposals, and bmo eval posture
• Concept Map — concept-first docs navigation

Maintainer design detail: Conversation quality seams (Route → Scope → Clarify → Gather → Verify → Fallback → Observe).