feat(tui): show live tokens per second while a response streams - #46882
ai-dev-2024 wants to merge 1 commit into
Conversation
github-actions
Bot
added
the
needs:compliance
|
This PR doesn't fully meet our contributing guidelines and PR template. What needs to be fixed:
Please edit this PR description to address the above within 2 hours, or it will be automatically closed. If you believe this was flagged incorrectly, please let a maintainer know. |
|
The following comment was made by an LLM, it may be inaccurate: Based on my search, I found 2 potentially related PRs:
These should be reviewed to ensure PR #46882 doesn't duplicate existing work or addresses a different aspect (in this case, live streaming estimates vs. final/completed stats). |
There was a problem hiding this comment.
🟡 Changes recommended
The live ticker currently doesn’t reliably stop/hide on step completion for finish: "tool-calls" turns and can also be incorrectly hidden when the computed value rounds to 0.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
Adds a live, approximate tokens-per-second readout to the assistant message footer while a response is streaming in the TUI, then hides it once streaming ends.
Changes:
- Adds a 500ms “tick” signal to refresh a live tok/s estimate during streaming.
- Estimates tok/s from accumulated streamed
text+reasoningcharacter counts using achars / 4heuristic and elapsed time since message creation. - Renders
· ~N tok/sin the assistant footer only during the streaming window.
File summaries
| File | Description |
|---|---|
| packages/tui/src/routes/session/index.tsx | Adds a periodic live tok/s estimator/memo and displays it in the assistant message footer while streaming. |
Review details
Suppressed comments (1)
packages/tui/src/routes/session/index.tsx:1503
liveTpsonly checksfinal(), so for completed steps that end withfinish: "tool-calls"it can continue to return a value (and won’t react to completion changes) even though the message is no longer streaming. Usemessage.time.completedas the streaming boundary so the ticker reliably disappears when the step completes.
const liveTps = createMemo(() => {
if (final()) return undefined
tick()
- Files reviewed: 1/1 changed files
- Comments generated: 2
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| createEffect(() => { | ||
| if (final()) return | ||
| const id = setInterval(() => setTick((t) => t + 1), 500) | ||
| onCleanup(() => clearInterval(id)) | ||
| }) |
| <Show when={liveTps()}> | ||
| <span style={{ fg: theme.textMuted }}> · ~{liveTps()} tok/s</span> | ||
| </Show> |
|
This pull request has been automatically closed because it was not updated to meet our contributing guidelines within the 2-hour window. Feel free to open a new pull request that follows our guidelines. |
github-actions
Bot
removed
the
needs:compliance
What
Shows a live, estimated tokens-per-second figure in the assistant message footer while a response is streaming:
Build · Muse Spark 1.2 Free · ~42 tok/sWhen the message completes, the ticker disappears (the footer's final stats take over; an exact final tok/s figure is proposed separately in #46108).
Why
#43857 (throughput visibility) covers completed responses; this covers the streaming window — the part of a response where "is this model slow or just warming up?" is most felt. Same motivator: with gateways and shared free pools, per-response speed varies wildly between turns, and nothing in the UI surfaces that until it's over.
Design disclosure (the part reviewers should scrutinize)
Exact output tokens are only known at
step-finish(ctx.assistantMessage.tokens = usage.tokensin the session processor) — the client receives no token counts mid-stream. So the ticker is an estimate:text+reasoningpart characters for the streaming messagechars / 4message.time.created~prefix to mark it approximate, refreshed every 500 ms, hidden for the first second and whenever the message is final or has no output yetNo protocol/schema changes — pure TUI diff. If maintainers prefer, the tilde could become a settings-gated display, or the estimate could move server-side later (periodic usage emission during the stream) as a follow-up.
Verification
bun turbo run typecheck --filter=@opencode-ai/tui— passesbun run devinpackages/tui, send a prompt that produces a long response —~N tok/sappears while streaming and disappears at completion. Short responses may finish before the 1s warm-up elapses and show nothing (intended).