feat(tui): show per-model health and throughput in the model selector - #46888
ai-dev-2024 wants to merge 1 commit into
Conversation
github-actions
Bot
added
the
needs:compliance
|
This PR doesn't fully meet our contributing guidelines and PR template. What needs to be fixed:
Please edit this PR description to address the above within 2 hours, or it will be automatically closed. If you believe this was flagged incorrectly, please let a maintainer know. |
There was a problem hiding this comment.
🟡 Changes recommended
There are a few concrete UX/runtime issues (free-model sorting with JSX footers, avg===0 treated as missing, and unhandled state-file read + unbounded seen growth) that should be fixed before merging.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
Adds a TUI-local “model health + throughput” signal to the model selector, based on recently observed message.updated events on the current machine and persisted to the TUI state directory (no protocol/schema changes).
Changes:
- Introduces a persisted per-model stats store (
model-stats.json) capturing recent tok/s samples and recent error timestamps. - Hooks
message.updatedevents in the TUI app to record throughput/errors. - Renders a colored health dot + average tok/s in the model selector footer for models with local history.
File summaries
| File | Description |
|---|---|
| packages/tui/src/util/model-stats.ts | New persistence + summarization + event recording for per-model tok/s and error health. |
| packages/tui/src/component/dialog-model.tsx | Displays per-model health/throughput footer and updates sorting/type constraints. |
| packages/tui/src/app.tsx | Initializes model stats storage and subscribes to message.updated events. |
Review details
Suppressed comments (1)
packages/tui/src/component/dialog-model.tsx:216
sortModelOptionsshould not depend on the renderedfootervalue to decide whether a model is free, since the footer may be JSX and/or include other decorations. Sorting should use a dedicated boolean field (e.g.isFree) that remains stable regardless of rendering.
export function sortModelOptions<T extends { footer?: string | JSX.Element; releaseDate: string | number; title: string }>(
options: T[],
newestFirst: boolean,
) {
if (newestFirst) return sortBy(options, [(option) => option.releaseDate, "desc"], (option) => option.title)
- Files reviewed: 3/3 changed files
- Comments generated: 4
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| const dot = <span style={{ fg: summary.healthy ? theme.diffAdded : theme.error }}>●</span> | ||
| const speed = summary.avg ? ` ${summary.avg} tok/s` : " err" | ||
| const base = existing ? ` · ${existing}` : "" |
| : undefined, | ||
| category: connected() ? provider.name : undefined, | ||
| disabled: provider.id === "opencode" && model.includes("-nano"), | ||
| footer: info.cost?.input === 0 && provider.id === "opencode" ? "Free" : undefined, | ||
| footer: statFooter(provider.id, model, info.cost?.input === 0 && provider.id === "opencode" ? "Free" : undefined), | ||
| onSelect() { |
| void Flock.withLock(LOCK, async () => { | ||
| const stored = await readJson<Record<string, ModelStat>>(file()) | ||
| if (stored && typeof stored === "object") setStats(reconcile(stored)) | ||
| }) |
| if (info.error) { | ||
| if (!seen.has(info.id)) { | ||
| seen.add(info.id) | ||
| const prev = stats[key] ?? { samples: [], errors: 0 } | ||
| setStats(key, { samples: prev.samples, errors: prev.errors + 1, lastErrorAt: Date.now() }) | ||
| persist() | ||
| } | ||
| return | ||
| } | ||
|
|
||
| if (seen.has(info.id)) return | ||
| const time = info.time ?? {} | ||
| if (typeof time.completed !== "number" || typeof time.created !== "number") return | ||
| seen.add(info.id) | ||
|
|
||
| const output = info.tokens?.output ?? 0 | ||
| const seconds = (time.completed - time.created) / 1000 | ||
| if (output <= 0 || seconds <= 0 || info.finish === "error") return |
|
This pull request has been automatically closed because it was not updated to meet our contributing guidelines within the 2-hour window. Feel free to open a new pull request that follows our guidelines. |
github-actions
Bot
removed
the
needs:compliance
What
The model selector now shows a health/throughput line for models you have actually used on this machine:
Models with no local history render exactly as before — no clutter, no fake data.
Why
#43857 and #46882 cover tok/s during and after generation. This closes the loop on the third part: choosing a model. Gateways and shared free pools mean the same model name can be fast one turn and dead the next (upstream 429s, capacity errors) — nothing in the selector reflects that today. With this, the selector answers "which of my models are actually usable right now, and how fast are they?" from observed reality.
Design disclosure
message.updatedevents it already receives (exact tok/s fromtokens.output / elapsedon completion, error counts oninfo.error), kept for the last 10 responses per model, and persisted atomically (Flock+writeJsonAtomic) tomodel-stats.jsonin the TUI state directory. No protocol, schema, or SDK changes; no network calls; nothing leaves the machine.Verification
bun turbo run typecheck --filter=@opencode-ai/tui— passesbun run devinpackages/tui— send a few messages across models (including one against a rate-limited/errored model), open the model selector: used models show● <avg> tok/s, the errored one shows red. Untried models are unchanged.Screenshot
To follow in a comment (needs an interactive session; behavior described above is testable with any two models, one forced to error).