Skip to content

review: whole-repo double check — a Converse panic, the uncapped TPM window, reasoning-helper docs - #90

Merged
CMGS merged 4 commits into
mainfrom
review/double-check-2026-09-18
Sep 18, 2026
Merged

CMGS merged 4 commits into
mainfrom
review/double-check-2026-09-18

Conversation

@CMGS

@CMGS CMGS commented Sep 18, 2026

Copy link
Copy Markdown
Collaborator

A second full read of every production file (65 files, 27.6k lines) after the 09-18 batch, against the /code-rs, /simplify and loc-justify lenses. Three commits, each its own concern.

fix: a client-reachable panic in the Converse transcoder (engines/src/converse.rs)
reasoning_config read output_config["effort"] through serde_json's IndexMut, which panics on anything but an object or null. output_config arrives verbatim from a /v1/messages body (MessagesRequest.output_config: Option<Value>) and is inserted into the Messages body as-is, so on a Converse account whose id names a reasoning_config family (openai.gpt-N, xai.grok-N) a string or number there took the pipeline down to a 500 through the handler's catch_unwind. Same class as the Converse fix in #73. The read goes through get_mut now and falls back to the thinking budget, as a missing effort already did. The new test fails on the previous code with cannot access key "effort" in JSON string.

review: docs, a dead clause, a misplaced doc block, one loop (protocol/src/reasoning.rs, engines/src/converse.rs, dag/src/context.rs)

  • The 09-18 effort helpers carried three- to six-line /// blocks (about 18% comment density against the repo's 8.5%); each keeps its vendor fact in one or two lines. Comment lines +17/−34.
  • reasoning_engaged's || model.contains("gpt-6") could never fire: the function only runs for an id openai_generation parsed, and every gpt-6… id parses to a major of 6. The model parameter goes with it. (openai_effort's own contains("gpt-6") is live — it handles vendor-prefixed ids — and stays.)
  • converse::request computes reasoning_family once instead of per sampling field, and the match … _ => {} collapses into an if-let chain.
  • DagContext::effective_user_id's doc block sat above model_param.

perf: stop writing the TPM window for keys without a cap (state/src/admission.rs, governance.rs, lib.rs, views/src/lib.rs)
A key without tokens_per_minute reserved nothing, and settle_and_bill then added its total to a TPM window that only token_window_reserve reads — which runs only for capped keys. On Redis that was one EVALSHA per billed request for every uncapped key; in-process, a TokenWindow slot per key that never went away. A capped key's ungated realtime turn (a usage frame with no admitted turn) keeps accruing: it now settles against a zero reserve, which is the same increment. Governance::token_window_add and TokenWindow::add have no caller left and go. Prod −19 lines.

Everything else re-read came back consistent with the 09-09 round's adjudications; no standing rejection was re-litigated. The hygiene ledger is re-pinned to these blobs.

Validation

  • cargo fmt --all --check, cargo clippy --workspace --all-targets -- -D warnings: clean
  • cargo test --workspace: 618 passed, 0 failed; the Redis-gated gw-state governance tests ran against a scratch Redis (redis_governance_enforces_limits … ok), not skipped
  • Mutation: the new Converse test against the pre-fix code panics (cannot access key "effort" in JSON string)
  • A/B for the perf commit (fcedc10 vs 3658ccb, release, OnlineHandler::run with tenant QPS + key QPS + product QPM configured, no TPM cap, ABBA × 8 rounds in one run):
mode scenario A p50 B p50 paired median B faster
memory one hop 5458 ns 5354 ns −1.9% 8/8
memory one fallback 9208 ns 9062 ns −1.1% 8/8
redis (loopback) one hop 443.1 µs 438.6 µs −1.1% 8/8
redis (loopback) one fallback 621.4 µs 615.0 µs −1.2% 8/8

The Redis wall-clock gain is small because the settle's governance calls run under one join!; the command count is the real delta: INFO commandstats over 8,401 requests shows 58,808 EVALSHA on A and 50,407 on B — exactly one fewer per request (7.0 → 6.0).

docs: reconcile README and docs with the code (README.md, docs/{api,configuration,deployment,development,governance,multi-instance,observability,security}.md)
Every page read against the current source. Fixed: the ledger repair queue retries a batch eight times (about 13 s) and then drops it into gateway_ledger_write_failures_total — security.md and observability.md said an accepted row was never discarded; Bedrock's temperature/topP refusal covers xai.grok-<n> ids too; per-model availability counts are Redis-shared (RedisAvail) but appeared in neither the storage section nor the fleet table; the fallback paragraph now states per-hop model-QPM spend and the at-most-one request permit (a cache hit reaches no limiter); the top-level-flags snippet had trust_proxy_headers' comment continuation under max_live_streams_per_key; token estimation lives in dag, not models; the MCP OAuth highlight names both grants; the release download example moves from v0.1.9 to v0.2.0 (asset names checked against the release). Docs only, no code.

…on-object

`output_config` arrives verbatim from a /v1/messages client, and serde_json's
IndexMut panics on anything but an object or null. On a Converse account whose
id names a reasoning_config family (openai.gpt-N, xai.grok-N), a string or
number there took the pipeline down to a 500. Read the effort through get_mut
and fall back to the thinking budget, as a missing effort already did.
…a misplaced doc block

The 09-18 batch documented the effort helpers in three- to six-line blocks,
twice the repo's density; each keeps its vendor fact in one line. The
`model.contains("gpt-6")` clause in reasoning_engaged could not fire: the
function only runs for an id openai_generation parsed, and every gpt-6 id
parses to a major of 6. In converse::request the reasoning-family test moves
out of the sampling loop and the match collapses into an if-let chain. In
DagContext the doc block for effective_user_id sat above model_param.
A key without tokens_per_minute reserved nothing, and settle_and_bill then
added its total to a TPM window that only token_window_reserve reads, which
runs only for capped keys. On Redis that was one INCRBY round trip per billed
request for every uncapped key; in-process, a slot per key that never went
away. A capped key's ungated realtime turn keeps accruing: it now settles
against a zero reserve, which is the same increment, so token_window_add and
TokenWindow::add have no caller left.
Read against the current source: the ledger repair queue retries a batch eight
times (about 13 s) and then drops it into gateway_ledger_write_failures_total,
where two pages promised it was never discarded; Bedrock drops temperature/topP
for xai.grok-<n> ids as well as openai.gpt-<n>; per-model availability counts
are Redis-shared like health and governance but were listed nowhere; the
fallback paragraph now states per-hop QPM spend and the at-most-one request
permit; the top-level-flags snippet had one comment's continuation lines
under the other flag; token estimation lives in the dag crate, not models;
the MCP OAuth line names both grants; the release example points at v0.2.0.
@CMGS
CMGS merged commit c2786c6 into main Sep 18, 2026
2 checks passed
@CMGS
CMGS deleted the review/double-check-2026-09-18 branch September 18, 2026 03:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant