PRD — The Brain Era: 8 features from the 2026-07-06 audit
Handoff target: an agent goal (/60/goal pipeline or overnight-operator). Each feature is a self-contained epic with stories, acceptance criteria, and the eval dimension that proves it. The eval battery (v3app/eval, 38 questions, strict scorer) is the acceptance instrument throughout — no feature is "done" until its named metric moves and no other dimension regresses.
Context an implementing agent must read first: gateway PR #88 (feat/brain-memory-95 — the grounded answer layer lab/brainAnswer.mjs, tryBrainAnswer, degrade-open contract), the eval README + latest results JSON, tickets USE60-APP-203–209, and the V3 experience plan (Railway sites: 60-chat-experience-plan.html). Baseline after PR #88 deploys: Memory 98.3 / Brain 96.3 / confabs 0 (local-verified).
Epic 1 — The Writer: fluent answers on grounded truth
Problem. Prod /api/chat has NO writer LLM configured (discovered 2026-07-06 — the root cause of years of boilerplate). PR #88 makes answers true but deterministic-terse. Users deserve both.
Design. Writer LLM (Anthropic, default claude-sonnet-5) composes the final reply only from the grounded layer's output: tryBrainAnswer's facts + source metadata are the entire allowed context ("evidence bundle"). System prompt forbids claims outside the bundle; the existing confabulation gate post-checks the draft (any entity in the reply absent from bundle+question → fall back to the deterministic reply). Temperature ≤0.3. Latency budget: p95 ≤ 2.5s added; timeout → deterministic reply (degrade-open, same contract as #88).
Stories.
- As a user, my answers read like a sharp colleague wrote them while containing only verifiable facts.
- As an operator, if the writer key is missing/erroring, behaviour is exactly PR #88 (never boilerplate, never an error).
- As a reviewer, I can diff writer-on vs writer-off replies for the full battery in one artifact.
Acceptance. Eval: all dimensions ≥ PR-#88 scores; confabs stay 0 across 3 consecutive runs (the writer must not reintroduce invention). New WRITER smoke: 5 battery questions' replies score 3/3 AND pass a "no uncited entity" check.
ANTHROPIC_API_KEYconfigured via Railway + documented in .env.example. Rollback = unset key. Deps: PR #88 deployed. Risk: writer re-introduces confabulation → mitigated by post-gate + eval gate in CI (Epic 2).
Epic 2 — Eval as a nightly SLO
Problem. The writer key sat missing in prod for months because nothing measured answer quality. Scores must become monitored infrastructure.
Design. Nightly job (GitHub Actions cron or Dev Hub durable cron) runs the battery against prod /api/chat with a dedicated eval service-account (long-lived refresh token stored as a secret; rotate on use like the runner does). Posts scoreboard to Slack (#eng or #brain): two scores, per-dimension deltas, confab count, 3 worst answers verbatim. Alert path (distinct @here message) when: any dimension drops >5 pts, either headline score drops >3, or confabs >0. Results appended to a brain_eval_runs table (or repo-committed JSONs) for trend charts.
Stories. 1. Nightly Slack scoreboard. 2. Regression alert within 24h of any deploy that hurts the brain. 3. A trend view (even a sparkline in the Slack message) showing 30-day movement.
Acceptance. Three consecutive nightly runs posted; a deliberately-broken staging run triggers the alert format; runbook documented (how to refresh the eval account, how to add questions). Battery versioned — score changes from battery edits are labelled as such in the post.
Deps: none (works against current prod; better after #88).
Epic 3 — Conversation memory ("what did I ask you yesterday")
Problem. No chat history persists across sessions — eval P04 fails structurally; the assistant has amnesia between conversations.
Design. Per-user rolling conversation memory: on each /api/chat turn, upsert a compact turn record (user question, answer headline, entities, ts) into copilot_conversation_memory (new table, RLS by user). The grounded layer gains a conversation_recall facet: "what did I ask yesterday / last week / about X" answers from this store with timestamps. Retention 90 days; PII stays within the org's own Supabase.
Stories. 1. "What did I ask you yesterday?" → real list with times. 2. "What did you tell me about Acme last week?" → the prior answer's headline + date, with a "want the full answer again?" offer. 3. Deleting a conversation deletes its memory rows.
Acceptance. Eval P04 scores 3/3 (add bonus terms so 3 is reachable); a new RECALL question about a prior conversation scores ≥2; migration + RLS reviewed; write path adds <50ms p95.
Deps: app-repo migration + gateway change. Risk: privacy — document retention and deletion path in the PR.
Epic 4 — Unified last-contact (meetings + email + Slack)
Problem. "When did I last speak to X" only sees analysed meetings. Real relationships live across channels.
Design. A last_contact resolver in the grounded layer that unions: analysed meetings (existing stakeholder scan), Gmail thread participants (existing google-direct tables), Slack DMs/mentions (existing slack domain tables). Returns the most recent touch with channel + date + one-line context, plus the trailing 3 touches. Answer cites each source ("Email · 3 Jul", "Meeting · 2 Jul").
Stories. 1. "When did I last speak to Angela?" → "Thursday, by email (re: DNS handover); before that the Jul 2 call." 2. Person unknown across all channels → honest absence (confab gate applies). 3. Ambiguous names → clarify with the channels each candidate appears in.
Acceptance. New eval RECALL sub-questions (3) covering email-only, slack-only, and cross-channel contacts score ≥2 each; p95 ≤ 1.5s (parallel reads, 60s cache); zero regression elsewhere.
Deps: read access to email/slack tables from the gateway service role (verify RLS posture; if blocked, expose via two new internal tools instead — note in PR).
Epic 5 — Calendar-depth freshness
Problem. FRESHNESS ceiling: "what's coming up" lists events; it should reason (prep gaps, conflicts, deal linkage). Design. Extend the freshness facet: for each upcoming event in the window join → has prep? (meeting_prep) · relates to at-risk deal? (pipeline stage + staleness) · conflicts? (overlap detection) · external vs internal. Answer leads with the judgement ("Tomorrow's Anuncia call is your only external meeting this week and it has no prep yet"), then the list. Stories. 1. "What's coming up this week?" → prioritised, judged list. 2. "Am I ready for tomorrow?" → prep-status answer with a draft-prep offer. 3. Conflicts called out unprompted when asked about the day. Acceptance. Battery F01/F03/F04 get bonus criteria (specific judgement terms) and score 3/3; FRESHNESS ≥90; add F05 "am I ready for tomorrow" scoring ≥2. Deps: none beyond existing tools; battery edit versioned per Epic 2.
Epic 6 — GAMECHANGER: The self-healing brain
Problem. 2026-07-06 proved the loop once, manually orchestrated: eval → diagnose → fix → verify → draft PR. Institutionalise it.
Design. A scheduled autonomous pipeline (Dev Hub cron → overnight-operator pattern): (1) consume the nightly eval result (Epic 2); (2) if any dimension <95 or confabs >0, spawn a diagnose-and-fix agent with the failing transcripts, the gateway repo in an isolated worktree, and the local-gateway eval loop recipe (documented in PR #88's methodology); (3) agent iterates fix→local-eval; (4) opens a DRAFT PR with before/after tables (the PR #88 template becomes docs/brain-fix-pr-template.md); (5) posts to Slack for human review. Hard rails: draft PRs only, never deploys, degrade-open changes only, one PR per night max, stops if the same dimension fails 3 nights running (escalates instead).
Stories. 1. A regression on Tuesday night is a reviewed fix PR by Wednesday morning. 2. Every fix PR carries eval evidence in a standard format. 3. Escalation, not thrash, on persistent failures.
Acceptance. One full simulated cycle: seed a deliberate regression on a staging branch → nightly eval flags → agent produces a correct draft fix PR with evidence → human merges. Runbook + kill-switch (SELFHEAL_ENABLED) documented.
Deps: Epics 1–2. Risk: battery overfitting → the agent's brief forbids battery-entity hardcoding (inherited rule), and quarterly battery refresh is part of the runbook.
Epic 7 — GAMECHANGER: Brain Score as a product surface
Problem. "How good is my 60?" is invisible to customers; integration nagging is friction with no payoff loop. Design. Per-org Brain Score computed from an org-safe subset of the eval method (auto-generated per-org battery: their real entities, standard dimensions; runs weekly server-side). Surface: a score ring in /60 settings + a "raise your score" panel mapping gaps to actions ("Connect Slack → +8 recall", "6 meetings missing transcripts → fix recording"). CS dashboard: score trend per account (churn signal). Sales: a live "audit our brain" demo mode. Stories. 1. As a customer I see my 60's score and exactly how to raise it. 2. As CS I get a weekly list of accounts whose score fell. 3. As sales I can run the score live in a demo org. Acceptance. Score computed weekly for ≥1 pilot org from real data; the "raise it" actions are real and deep-link; score persists + trends; copy reviewed (score must never shame — it invites). Feature-flagged per org. Deps: Epics 1–2 (the method), app-repo UI work (V3 design language — the credits-ring pattern fits). Risk: score embarrassment → floor the displayed granularity (bands, not decimals) and always pair with actions.
Epic 8 — GAMECHANGER: "Evidence or silence" — the universal trust contract
Problem. The confabulation gate currently guards one path. Make citability the architecture, and the market position.
Design. (1) Envelope: every /api/chat (and voice) reply carries citations[] — each claim-bearing sentence maps to a source (meeting id, record id, email thread, profile field). (2) UI: V3 source chips render per-answer citations; hovering a headline phrase highlights its source chip (the prototype's source-chip primitive is ready). (3) Contract: replies containing an uncited concrete entity are blocked server-side → regenerate grounded or answer with honest absence. (4) Writes: approval cards must cite the evidence that motivated them ("Why: 9 days quiet [Deal timeline]"). (5) Positioning: docs page + one-pager: "60 cites or stays silent."
Stories. 1. Every answer's claims are tappable to their sources. 2. An uncitable claim never reaches the user. 3. Approval cards show their evidence. 4. Voice speaks citations naturally ("from your Jul 2 call…").
Acceptance. New eval dimension CITATION (5 questions): every concrete entity in each answer maps to a valid citation — target 100; existing dimensions don't regress; contract enforced in the pipeline (test: an injected uncited claim is stripped/regenerated); one enterprise-facing doc shipped.
Deps: Epics 1, 6 recommended first. This is the largest epic — split PRs: envelope → gate → UI → writes → voice.
Sequencing & effort
| Order | Epic | Size | Gate |
|---|---|---|---|
| 1 | E1 Writer | S (2–3 days) | PR #88 deployed |
| 2 | E2 Nightly SLO | S | eval account secret |
| 3 | E3 Conversation memory | M | migration review |
| 4 | E5 Calendar freshness | S | battery edit |
| 5 | E4 Unified last-contact | M | RLS/tool decision |
| 6 | E6 Self-healing loop | M | E1+E2 live |
| 7 | E7 Brain Score | L | pilot org pick |
| 8 | E8 Evidence-or-silence | L (multi-PR) | E1 live |
Global rules for the implementing agent(s): draft PRs only; degrade-open everywhere; eval must not regress on any dimension in any PR (attach before/after tables using the PR #88 format); no battery-entity hardcoding; secrets never printed or committed; gateway deploys are always a human decision.
North-star metrics: Memory ≥95 and Brain ≥95 sustained nightly for 30 days · confabulations = 0 always · CITATION 100 after E8 · one customer-visible Brain Score live (E7).
Prepared 6 Jul 2026 · Fable · companion docs: /60 Stream Redesign plan · /60 Chat Experience plan · gateway PR #88.