review: whole-repo double check — a Converse panic, the uncapped TPM window, reasoning-helper docs - #90
Merged
Conversation
…on-object `output_config` arrives verbatim from a /v1/messages client, and serde_json's IndexMut panics on anything but an object or null. On a Converse account whose id names a reasoning_config family (openai.gpt-N, xai.grok-N), a string or number there took the pipeline down to a 500. Read the effort through get_mut and fall back to the thinking budget, as a missing effort already did.
…a misplaced doc block
The 09-18 batch documented the effort helpers in three- to six-line blocks,
twice the repo's density; each keeps its vendor fact in one line. The
`model.contains("gpt-6")` clause in reasoning_engaged could not fire: the
function only runs for an id openai_generation parsed, and every gpt-6 id
parses to a major of 6. In converse::request the reasoning-family test moves
out of the sampling loop and the match collapses into an if-let chain. In
DagContext the doc block for effective_user_id sat above model_param.
A key without tokens_per_minute reserved nothing, and settle_and_bill then added its total to a TPM window that only token_window_reserve reads, which runs only for capped keys. On Redis that was one INCRBY round trip per billed request for every uncapped key; in-process, a slot per key that never went away. A capped key's ungated realtime turn keeps accruing: it now settles against a zero reserve, which is the same increment, so token_window_add and TokenWindow::add have no caller left.
Read against the current source: the ledger repair queue retries a batch eight times (about 13 s) and then drops it into gateway_ledger_write_failures_total, where two pages promised it was never discarded; Bedrock drops temperature/topP for xai.grok-<n> ids as well as openai.gpt-<n>; per-model availability counts are Redis-shared like health and governance but were listed nowhere; the fallback paragraph now states per-hop QPM spend and the at-most-one request permit; the top-level-flags snippet had one comment's continuation lines under the other flag; token estimation lives in the dag crate, not models; the MCP OAuth line names both grants; the release example points at v0.2.0.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A second full read of every production file (65 files, 27.6k lines) after the 09-18 batch, against the /code-rs, /simplify and loc-justify lenses. Three commits, each its own concern.
fix: a client-reachable panic in the Converse transcoder (
engines/src/converse.rs)reasoning_configreadoutput_config["effort"]through serde_json'sIndexMut, which panics on anything but an object or null.output_configarrives verbatim from a/v1/messagesbody (MessagesRequest.output_config: Option<Value>) and is inserted into the Messages body as-is, so on a Converse account whose id names a reasoning_config family (openai.gpt-N,xai.grok-N) a string or number there took the pipeline down to a 500 through the handler'scatch_unwind. Same class as the Converse fix in #73. The read goes throughget_mutnow and falls back to the thinking budget, as a missing effort already did. The new test fails on the previous code withcannot access key "effort" in JSON string.review: docs, a dead clause, a misplaced doc block, one loop (
protocol/src/reasoning.rs,engines/src/converse.rs,dag/src/context.rs)///blocks (about 18% comment density against the repo's 8.5%); each keeps its vendor fact in one or two lines. Comment lines +17/−34.reasoning_engaged's|| model.contains("gpt-6")could never fire: the function only runs for an idopenai_generationparsed, and everygpt-6…id parses to a major of 6. Themodelparameter goes with it. (openai_effort's owncontains("gpt-6")is live — it handles vendor-prefixed ids — and stays.)converse::requestcomputesreasoning_familyonce instead of per sampling field, and thematch … _ => {}collapses into an if-let chain.DagContext::effective_user_id's doc block sat abovemodel_param.perf: stop writing the TPM window for keys without a cap (
state/src/admission.rs,governance.rs,lib.rs,views/src/lib.rs)A key without
tokens_per_minutereserved nothing, andsettle_and_billthen added its total to a TPM window that onlytoken_window_reservereads — which runs only for capped keys. On Redis that was one EVALSHA per billed request for every uncapped key; in-process, aTokenWindowslot per key that never went away. A capped key's ungated realtime turn (a usage frame with no admitted turn) keeps accruing: it now settles against a zero reserve, which is the same increment.Governance::token_window_addandTokenWindow::addhave no caller left and go. Prod −19 lines.Everything else re-read came back consistent with the 09-09 round's adjudications; no standing rejection was re-litigated. The hygiene ledger is re-pinned to these blobs.
Validation
cargo fmt --all --check,cargo clippy --workspace --all-targets -- -D warnings: cleancargo test --workspace: 618 passed, 0 failed; the Redis-gatedgw-stategovernance tests ran against a scratch Redis (redis_governance_enforces_limits … ok), not skippedcannot access key "effort" in JSON string)OnlineHandler::runwith tenant QPS + key QPS + product QPM configured, no TPM cap, ABBA × 8 rounds in one run):The Redis wall-clock gain is small because the settle's governance calls run under one
join!; the command count is the real delta:INFO commandstatsover 8,401 requests shows 58,808 EVALSHA on A and 50,407 on B — exactly one fewer per request (7.0 → 6.0).docs: reconcile README and docs with the code (
README.md,docs/{api,configuration,deployment,development,governance,multi-instance,observability,security}.md)Every page read against the current source. Fixed: the ledger repair queue retries a batch eight times (about 13 s) and then drops it into
gateway_ledger_write_failures_total— security.md and observability.md said an accepted row was never discarded; Bedrock'stemperature/topPrefusal coversxai.grok-<n>ids too; per-model availability counts are Redis-shared (RedisAvail) but appeared in neither the storage section nor the fleet table; the fallback paragraph now states per-hop model-QPM spend and the at-most-one request permit (a cache hit reaches no limiter); the top-level-flags snippet hadtrust_proxy_headers' comment continuation undermax_live_streams_per_key; token estimation lives indag, notmodels; the MCP OAuth highlight names both grants; the release download example moves from v0.1.9 to v0.2.0 (asset names checked against the release). Docs only, no code.