views: make the request body limit configurable, default 32 MiB

[CMGS]

What breaks

axum 0.8 buffers at most 2 MiB of a request body unless the router sets a different limit, and the gateway never did. Computer-use agents send screenshots inline as base64, so one visually busy frame can push a request past 2 MiB and the gateway answers 413 request_entity_too_large_exception before any upstream sees it. The limit could not be raised from config.

Seen in production on the OSWorld node: 1 hit in 224 calls on a four-task regression, and 2 + 2 on two later runs. The driver recovered in each case, but the call was lost.

Evidence that the gateway is the only bottleneck

The same 4,002,891-byte request (one 1000×1000 image), sent two ways:

OpenRouter direct   http=200   answered "ok"
through gw 0.2.1    http=413   5.7 ms, before any upstream call

The limit, bracketed with a model name that does not exist, so nothing reaches a vendor:

2,096,101 bytes   404 unknown model   (body accepted)
2,097,201 bytes   413                 (2 MiB = 2,097,152)

The change

• listen.max_request_bytes, default 32 MiB, read once when the router is built, like host and port
• app() applies it with axum::extract::DefaultBodyLimit::max, the only mechanism axum offers for raising the extractor limit
• docs: one line in the listen: block of configuration.md

32 MiB matches Anthropic's own request ceiling. It stays a bound rather than unlimited, because an unbounded body limit is a memory-exhaustion vector.

Tests

request_body_limit_follows_listen_config, new:

• a 3 MiB body is accepted on the default config (over axum's old limit)
• a router built with max_request_bytes: 1024 rejects a 4 KiB body with request_entity_too_large_exception

It fails for the right reason. With only the DefaultBodyLimit layer removed (config field kept), the 3 MiB assertion fails because the old 2 MiB default comes back:

test request_body_limit_follows_listen_config ... FAILED
  left: 413
 right: 413          (assert_ne!)

error_contract_machine_channel already asserted that a 3 MiB body is rejected, which encoded the old default as the expected behaviour. Its oversized input moves to 33 MiB, so it still tests the rejection's error envelope and nothing else changes.

Gates at the workspace root: cargo fmt --check, cargo clippy --all-targets -- -D warnings, cargo test: 619 passed, 0 failed.

Hot path: about 140–210 ns per request, measured in isolation

The layer does one thing per request (axum-core 0.5.6, default_body_limit.rs:224):

req.extensions_mut().insert(self.kind);

http::Extensions::insert boxes the value (Box::new(val), a heap allocation freed at the end of the request) and inserts it under its TypeId; if the extensions map has not been allocated yet it allocates that too, since the map is a lazy Option<Box<AnyMap>>. The body extractor then looks the value up, which is now a hit instead of a miss.

Measured on a minimal axum 0.8.9 router with a single Bytes handler, so the layer is the only difference between arms. Nanosecond resolution, 200,000 requests per run, 8 runs interleaved with the order swapped:

                                          none            layer          delta
extensions map not yet allocated          p50 792–833 ns  p50 1000 ns    +210 ns
map already allocated (the gateway case)  p50 958–959 ns  p50 1084–1125  +140 ns

About 70 ns of the worst case is the lazy map allocation, which the gateway's own middleware already pays before this layer runs. The remaining ~140 ns is the boxed value, the insert and the lookup hit.

An earlier revision of this description said "about 1 µs". That was an artefact of the benchmark, not of the change. crates/server/tests/bench.rs records latency with as_micros(), which truncates to whole microseconds, so a real shift of ~0.15 µs that carries p50 across a microsecond boundary shows up as a full 1 µs step. The in-gateway A/B (two rounds, n=8 per arm) moved p50 from 20 to 21 µs in 7 of 8 runs, which is consistent with a sub-microsecond shift near the boundary and not evidence of a 1 µs cost.

The cost is unavoidable while the limit goes through DefaultBodyLimit. The alternative is having every body-extracting handler call axum::body::to_bytes(body, limit) with an explicit limit, which touches all of them to save ~150 ns against an upstream call measured in seconds.