How the agents learn
Six pipelines exist to make replies better over time. This page is the honest scoreboard: what each one does, whether it genuinely works today, and what stands between the broken ones and life. Audited against live data — not the design documents.
flowchart LR
EP["📼 Episodes
every bot turn recorded"] ==>|"✅"| SC["Nightly scorer
did the turn work?"]
SC ==>|"✅ fixed Aug 2026"| UI["Per-customer
behavior notes"]
UI ==>|"✅ refilling"| P["🧠 the agent's prompt"]
KB["📚 Company knowledge
authored policies"] ==>|"✅"| P
AM["🧠 Rolling summary
updated every turn"] ==>|"✅ feeds BOTH agents"| P
QB["❓ Question Bank
29 questions"] ==>|"✅ serving ON — grows with approvals"| P
CORR["📝 Staff corrections"] -.->|"⚠️ ready, never used"| P
AN["📊 Analyst signals"] -.->|"❌ frozen since Feb"| P
P ==> REPLY["💬 better replies"]
classDef ok fill:#e7f2ec,stroke:#166b4e,color:#182420
classDef idle fill:#f7edd8,stroke:#b4711a,color:#182420
classDef dead fill:#f6e3de,stroke:#a63a2b,color:#182420
class EP,SC,UI,KB,AM,QB ok
class CORR idle
class AN dead
Solid arrows flow today. Dashed ones are built but not yet feeding anything.
The scoreboard
| Pipeline | What it does | Status |
|---|---|---|
| Company knowledge | Hand-written policy pages (prices, rules, services) retrieved into the agent's prompt as the authoritative source it must never contradict. | WORKING |
| Rolling summary | A per-customer memory note refreshed after every turn — read by both the conversation agent and the outreach agent. The one true bridge between them. | WORKING |
| Episodes → outcomes | Every turn is scored at night: quick reply? repeated question? went silent? completed a profile after? Scores roll up into per-customer behavior notes injected into future conversations. | WORKING repaired Aug 2026 — was silently dead for months |
| Question Bank | Deduplicated customer questions with staff-approved answers, served on meaning-match. Full page → | SERVING ON |
| Staff corrections | When staff correct a bad bot reply, every future similar situation shows the agent "don't repeat this mistake — prefer this wording." Fully built and waiting; no correction has ever been submitted. | READY, UNUSED |
| Analyst signals | Conversation-quality metrics from the analyst service. Its writes have been silently failing since February — known root cause, queued for repair. | FROZEN |
Why the episode loop matters most
It's the only pipeline that learns from outcomes rather than from what someone wrote down. A repeated question is a reply that failed. A customer who goes silent after a message is a signal. A profile completed the day after a conversation is a win worth attributing. For months these signals were computed by a job that crashed nightly while reporting success — the fix (August 2026) drained the entire backlog in one run, and two outcome signals fired for the first time ever the same night.