feat(transport): set compression and User-Agent request defaults - #195
Merged
Merged
Conversation
The generated client left two server-observable defaults to urllib3. urllib3 puts `Accept-Encoding: identity` on every connection, which is not "no preference" but an explicit request not to compress, so every JSON response came back uncompressed. RESTClientObject.request now advertises urllib3.util.request.ACCEPT_ENCODING -- exactly the codecs the installed urllib3 can transparently decode, so a server can never negotiate an encoding that arrives as undecodable bytes. It is a setdefault, so an operation whose payload is already compressed can still pass `identity`. Requests also identified themselves as OpenAPI-Generator/1.0.0/python, which attributes traffic to neither the SDK nor a release. The generator's httpUserAgent property only bakes a literal at generation time, and regens follow spec changes rather than releases, so a baked version would go stale between them; the string is resolved at import time from installed package metadata instead and lives in the generator-ignored hotdata/_useragent.py. rest.py and api_client.py are generator output, so the edits are re-applied by scripts/patch_request_defaults.py, wired into regenerate.yml alongside the existing patch steps.
zfarrell
requested review from
shefeek-jinnah
and removed request for
a team
September 18, 2026 19:38
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
Header names are case-insensitive, so a caller passing `accept-encoding` left both keys in the dict. urllib3 emits one header line per key, so the server received the opt-out *and* the compressed set and could still compress -- silently losing the documented per-request escape hatch. Compare lowercased, the way urllib3 does before adding its own Accept-Encoding.
_call_arrow set only Accept, so the new client-wide compression default applied to Arrow fetches too. IPC record batches are frequently LZ4- or ZSTD-compressed by the writer already, making a gzip pass over the stream CPU on both ends for little size gain. Opt the path out with `Accept-Encoding: identity`, restoring the transfer behavior Arrow had before the default was introduced.
There was a problem hiding this comment.
Prior nits are resolved. The case-insensitive check in hotdata/rest.py matches urllib3 behavior and has a wire-level test. The Arrow opt-out restores the previous transfer behavior.
Checked separately: presigned storage PUTs use a separate pool and do not get the new header. No code compares Content-Length against a decoded body length. The regen clean step does not delete hotdata/_useragent.py.
CI was still queued at review time, so no claim is made about test results.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The client never asked for compressed responses — urllib3 defaults every connection to
Accept-Encoding: identity, an explicit request not to compress — and identified itself asOpenAPI-Generator/1.0.0/python, which attributes traffic to neither the SDK nor a release.Both are generated files, so the edits are re-applied by
scripts/patch_request_defaults.py, wired intoregenerate.yml. The generator'shttpUserAgentproperty was not used because it bakes a literal at generation time, and regens follow spec changes rather than releases.