Skip to content

fix(gql): collapse carriage returns when sanitizing GraphQL strings - #2168

Open
feiiiiii5 wants to merge 1 commit into
weaviate:mainfrom
feiiiiii5:fix/graphql-strip-carriage-return
Open

feiiiiii5 wants to merge 1 commit into
weaviate:mainfrom
feiiiiii5:fix/graphql-strip-carriage-return

Conversation

@feiiiiii5

Copy link
Copy Markdown

Problem

_sanitize_str() can emit a GraphQL string literal containing a raw carriage return, which makes the whole query fail to parse server-side.

strip_newlines() only replaces U+000A:

def strip_newlines(s: str) -> str:
    return s.replace("\n", " ")

A "foo\r\nbar" value therefore becomes "foo bar" with a stray \r left in front of the space, and a lone "foo\rbar" is untouched. In the GraphQL grammar LineTerminator is LF or CR, and an unescaped LineTerminator is not a legal character inside a string literal — which is exactly why this function replaces newlines at all. CR simply never got the same treatment as LF.

Reachable through the ordinary filter path: any Where operand whose valueText / valueTextList / valueString value carries a carriage return goes through _sanitize_str (weaviate/gql/filter.py:692,695), as does an aggregate query (weaviate/gql/aggregate.py:46). Text that passed through a Windows newline is the common source.

Change

Collapse both line terminators, and CRLF to a single space rather than to "space + stray CR":

def strip_newlines(s: str) -> str:
    # GraphQL treats CR as a line terminator inside string literals just like LF,
    # so a CRLF value must collapse to one space instead of leaving a bare CR.
    return s.replace("\r\n", "\n").replace("\r", "\n").replace("\n", " ")

No new dependency, no change to any other character, and existing LF behaviour is byte-identical.

How this was verified

Regression cases added to the repo's own parametrized test_sanitize_str in test/test_util.py (that test previously pinned only the backslash/quote cases, nothing about line terminators).

Base control — weaviate/util.py restored from 142d798a93177215e68d44601c9653a856b3c373 with git diff --stat -- weaviate/util.py printing nothing, so the run measures upstream code while the new cases are kept:

$ python -m pytest test/test_util.py -q -p no:opik -k sanitize_str
2 failed, 8 passed, 34 deselected in 0.80s
FAILED test/test_util.py::test_sanitize_str[foo\r\nbar-foo bar]
FAILED test/test_util.py::test_sanitize_str[foo\rbar-foo bar]

With the patch applied:

$ python -m pytest test/test_util.py -q -p no:opik -k sanitize_str
10 passed, 34 deselected in 0.73s

Whole unit suite, base and head, to show the diff changes only what it claims:

base:  5 failed, 488 passed, 1 skipped     # 3 in test/test_timeout.py (pre-existing here) + my 2 new cases
head:  3 failed, 490 passed, 1 skipped     # only the 3 pre-existing test_timeout.py failures remain

The three test/test_timeout.py failures are present at base 142d798a unchanged, so they are not caused by this diff; they look environment-related (they spawn interpreters to test the timeout decorator). The mock suite passes: python -m pytest mock_tests -q65 passed.

Consequence, measured rather than asserted from the spec — building a real Weaviate-shaped query and lexing it with graphql-core (a spec-faithful parser, since I have no Weaviate server in this environment):

plain (control) -> parses OK
LF (control)    -> parses OK
CRLF            -> PARSE ERROR: Syntax Error: Unterminated string.
bare CR         -> PARSE ERROR: Syntax Error: Unterminated string.

I did not run the integration/compose suite, so the server-side rejection is demonstrated through the grammar and a spec parser rather than against a live Weaviate instance.

CLA: CONTRIBUTING.md requires a Contributor License Agreement for this repo; that is an account-owner action outside this diff.

strip_newlines only replaced U+000A, so a value containing CRLF or a lone CR kept a raw carriage return inside the quoted GraphQL string literal. CR is a LineTerminator in the GraphQL grammar and is not a legal unescaped character inside a string, so the server rejects the request with 'Unterminated string' rather than running the query. Text that passed through a Windows newline is the common source.

@orca-security-eu orca-security-eu Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Orca Security Scan Summary

Status Check Issues by priority
Passed Passed Infrastructure as Code high 0   medium 0   low 0   info 0 View in Orca
Passed Passed SAST high 0   medium 0   low 0   info 0 View in Orca
Passed Passed Secrets high 0   medium 0   low 0   info 0 View in Orca
Passed Passed Vulnerabilities high 0   medium 0   low 0   info 0 View in Orca

@weaviate-git-bot

Copy link
Copy Markdown

To avoid any confusion in the future about your contribution to Weaviate, we work with a Contributor License Agreement. If you agree, you can simply add a comment to this PR that you agree with the CLA so that we can merge.

beep boop - the Weaviate bot 👋🤖

PS:
Are you already a member of the Weaviate Forum?

@feiiiiii5

Copy link
Copy Markdown
Author

I have read and I agree with the Contributor License Agreement.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants