Conversation
strip_newlines only replaced U+000A, so a value containing CRLF or a lone CR kept a raw carriage return inside the quoted GraphQL string literal. CR is a LineTerminator in the GraphQL grammar and is not a legal unescaped character inside a string, so the server rejects the request with 'Unterminated string' rather than running the query. Text that passed through a Windows newline is the common source.
There was a problem hiding this comment.
Orca Security Scan Summary
| Status | Check | Issues by priority | |
|---|---|---|---|
| Infrastructure as Code | View in Orca | ||
| SAST | View in Orca | ||
| Secrets | View in Orca | ||
| Vulnerabilities | View in Orca |
|
To avoid any confusion in the future about your contribution to Weaviate, we work with a Contributor License Agreement. If you agree, you can simply add a comment to this PR that you agree with the CLA so that we can merge. |
Author
|
I have read and I agree with the Contributor License Agreement. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
_sanitize_str()can emit a GraphQL string literal containing a raw carriage return, which makes the whole query fail to parse server-side.strip_newlines()only replacesU+000A:A
"foo\r\nbar"value therefore becomes"foo bar"with a stray\rleft in front of the space, and a lone"foo\rbar"is untouched. In the GraphQL grammarLineTerminatoris LF or CR, and an unescapedLineTerminatoris not a legal character inside a string literal — which is exactly why this function replaces newlines at all. CR simply never got the same treatment as LF.Reachable through the ordinary filter path: any
Whereoperand whosevalueText/valueTextList/valueStringvalue carries a carriage return goes through_sanitize_str(weaviate/gql/filter.py:692,695), as does an aggregatequery(weaviate/gql/aggregate.py:46). Text that passed through a Windows newline is the common source.Change
Collapse both line terminators, and CRLF to a single space rather than to "space + stray CR":
No new dependency, no change to any other character, and existing LF behaviour is byte-identical.
How this was verified
Regression cases added to the repo's own parametrized
test_sanitize_strintest/test_util.py(that test previously pinned only the backslash/quote cases, nothing about line terminators).Base control —
weaviate/util.pyrestored from142d798a93177215e68d44601c9653a856b3c373withgit diff --stat -- weaviate/util.pyprinting nothing, so the run measures upstream code while the new cases are kept:With the patch applied:
Whole unit suite, base and head, to show the diff changes only what it claims:
The three
test/test_timeout.pyfailures are present at base142d798aunchanged, so they are not caused by this diff; they look environment-related (they spawn interpreters to test the timeout decorator). The mock suite passes:python -m pytest mock_tests -q→65 passed.Consequence, measured rather than asserted from the spec — building a real Weaviate-shaped query and lexing it with
graphql-core(a spec-faithful parser, since I have no Weaviate server in this environment):I did not run the integration/compose suite, so the server-side rejection is demonstrated through the grammar and a spec parser rather than against a live Weaviate instance.
CLA:CONTRIBUTING.mdrequires a Contributor License Agreement for this repo; that is an account-owner action outside this diff.