Fix #2255: memos_search returns Chinese text as \uXXXX unicode escapes (Hermes adapter miss - #2256
Merged
Hun-ger merged 2 commits intoAug 26, 2026
Conversation
….dumps (MemTensor#2255) memos_search / memos_get / memos_timeline / memos_skill_list / memos_environment / memos_skill_get in the Hermes memory provider serialized their tool results back to the host LLM with the default ensure_ascii=True, which escaped every non-ASCII code point (notably Chinese memory content) to \uXXXX. The DB stored the correct UTF-8; only the wire JSON was mangled. This inflated tokens for Chinese users and made retrieval results unreadable when debugging. Add ensure_ascii=False to every json.dumps inside handle_tool_call (both the tool-result branches called out in the issue and the error/fallthrough branches, for a uniform pattern that matches the other json.dumps calls in this file at L742/1015/1021 that already pass the flag). Add HandleToolCallEnsureAsciiTests to apps/memos-local-plugin/tests/python/test_hermes_provider_pipeline.py with 8 regression cases (one per affected tool + both branches of memos_environment) that assert the returned JSON contains raw Chinese characters, contains no "\u" escape marker, and round-trips through json.loads. Verified the tests fail without the fix and pass with it; ruff check + ruff format clean; full test file 35/35 green. Fixes MemTensor#2255
Memtensor-AI
added
ai:generated
Collaborator
Author
🤖 Open Code ReviewTarget: PR #2256 ✅ OpenCodeReview: No comments generated. Looks good to me. Generated by cloud-assistant via Open Code Review. |
Collaborator
Author
🔧 Open Code Review requested Agent fixOpen Code Review found 2 issue(s). I have resumed the development Agent to fix them.
The Agent will push a new commit to this PR branch. OCR will recheck after the commit is pushed. |
Address PR MemTensor#2256 OCR review findings: 1. Add `test_memos_get_world_model_returns_utf8_chinese` to cover the `world_model` branch of `memos_get` (routed via `memory.get_world`). Without this test, an accidental removal of `ensure_ascii=False` from the world_model branch would go undetected — the previous tests only exercised the `trace` and `policy` kinds. 2. Remove the fragile `assertNotIn("\\u", raw)` guard from all HandleToolCallEnsureAsciiTests cases. That check was prone to false positives (any legitimate value containing a backslash followed by `u`, e.g. a Windows path, would fail the test) while providing weaker coverage than the `assertIn(_CH_..., raw)` + `json.loads` field equality assertions already present. Add a header comment explaining why the literal-in-raw check is the robust regression guard. Also add a parsed field-equality assertion to `test_memos_skill_list_returns_utf8_chinese` for parity with the other cases. All 36 tests in tests/python/test_hermes_provider_pipeline.py pass.
Collaborator
Author
✅ Automated Test Results: PASSEDAll tests passed (35/35 executed). memos_local_plugin/changed-repo-python: 35/35. Duration: 4s [advisory, non-gating] AI-generated tests on branch test/auto-gen-48b062ae885009c2-20260816111233: 194/196 passed, 2 failed — these do NOT affect the PR verdict; review the branch manually. Branch: |
Hun-ger
approved these changes
Aug 26, 2026
Hun-ger
merged commit Aug 26, 2026
0d9503a
into
MemTensor:fix-local-plugin-260824
2 of 18 checks passed
15 tasks
syzsunshine219
added a commit
that referenced
this pull request
Aug 28, 2026
## Description This PR consolidates the local-plugin reliability fixes developed and validated on `fix-local-plugin-260824`. ### Highlights - hardens Hermes bridge process detection, PID/status reconciliation, installer rollback, Windows-native provider behavior, and UTF-8 tool output; - makes secret environment fallback explicit and wires first-class `maxTokens` / `headers` configuration through the L3 and Skill Evolver clients; - moves idle-skill archival off the foreground path with bounded, atomic lifecycle work; - recovers useful crystallizer drafts when LLM output omits derivable fields; - bounds startup-recovery shutdown and adds the newest-trace index with collision-safe migration handling; - resolves DeepSeek Harness auxiliary reasoning capability from exact model metadata, with per-route TTL caching, concurrent lookup coalescing, adapter-update invalidation, and registration-bound race fallback. Contribution history is preserved from #2192, #2209, #2246, #2248, #2252, #2253, #2256, and #2284. The capability-driven implementation supersedes the negative probe cache proposed in #2285. No new runtime dependencies are introduced. Related Issue (Required): Fixes #2278 Reviewers: @hijzy @whipser030 ## Type of change - [x] Bug fix (non-breaking change which fixes an issue) - [x] Refactor (does not change functionality, e.g. code style improvements, linting) - [x] Documentation update ## How Has This Been Tested? - [x] Unit Test - `npm run test:unit -- --reporter=dot`: 179 files passed; 1,549 tests passed and 2 skipped. - `npm exec vitest run tests/unit/adapters/deepseek-harness-*.test.ts -- --reporter=dot`: 65/65 passed. - `python3 apps/memos-local-plugin/tests/python/test_hermes_provider_pipeline.py`: 36/36 passed. - [x] Test Script Or Test Steps - `npm run lint` - `npm run build:package` - `npm run check:hermes-version` - Ruff 0.11.8 check and format verification for both changed Python files. - Packed commit `d76c78aa` and installed the identical tarball into DSH `0.1.0-rc.6` `web` profiles on macOS and Windows. - Verified SHA-256 `61e805f862acbe0873581e22cac4592ab0a9ff4d23d607a5f9fac5df93e80253` on both hosts. - Verified unsupported-reasoning routes issue zero rejected `off` probes across repeated/concurrent calls, supported routes retain `off`, adapter invalidation refreshes the capability, and both installed profiles start with healthy Host LLM bridges. - [x] Pipeline Automated API Test - GitHub CI will run after PR creation. > Note: `make format` could not start because Poetry is not installed on the validation host. The equivalent Ruff 0.11.8 checks for the changed Python files passed. ## Checklist - [x] I have performed a self-review of my own code | 我已自行检查了自己的代码 - [x] I have commented my code in hard-to-understand areas | 我已在难以理解的地方对代码进行了注释 - [x] I have added tests that prove my fix is effective or that my feature works | 我已添加测试以证明我的修复有效或功能正常 - [x] I have created related documentation issue/PR in [MemOS-Docs](https://github.com/MemTensor/MemOS-Docs) (if applicable) | N/A: no external documentation change is required. - [x] I have linked the issue to this PR (if applicable) | 我已将 issue 链接到此 PR - [x] I have mentioned the person who will review this PR | 我已提及将审查此 PR 的人 ## Reviewer Checklist - [x] closes #2278 - [x] Made sure Checks passed locally; GitHub CI will run on creation - [x] Tests have been provided
steven-stack-s
pushed a commit
to steven-stack-s/dsh-memos-local
that referenced
this pull request
Sep 18, 2026
Address PR MemTensor#2256 OCR review findings: 1. Add `test_memos_get_world_model_returns_utf8_chinese` to cover the `world_model` branch of `memos_get` (routed via `memory.get_world`). Without this test, an accidental removal of `ensure_ascii=False` from the world_model branch would go undetected — the previous tests only exercised the `trace` and `policy` kinds. 2. Remove the fragile `assertNotIn("\\u", raw)` guard from all HandleToolCallEnsureAsciiTests cases. That check was prone to false positives (any legitimate value containing a backslash followed by `u`, e.g. a Windows path, would fail the test) while providing weaker coverage than the `assertIn(_CH_..., raw)` + `json.loads` field equality assertions already present. Add a header comment explaining why the literal-in-raw check is the robust regression guard. Also add a parsed field-equality assertion to `test_memos_skill_list_returns_utf8_chinese` for parity with the other cases. All 36 tests in tests/python/test_hermes_provider_pipeline.py pass.
steven-stack-s
pushed a commit
to steven-stack-s/dsh-memos-local
that referenced
this pull request
Sep 18, 2026
…de escapes (Hermes adapter miss (MemTensor#2256) ## Description Fixed MemTensor#2255: Chinese memory content returned by the Hermes adapter's memos_* tools was being serialized to the host LLM as `\uXXXX` escapes because every `json.dumps(...)` inside `MemTensorProvider.handle_tool_call` (apps/memos-local-plugin/adapters/hermes/memos_provider/__init__.py) relied on Python's default `ensure_ascii=True`. Data in the DB was always correct; only the on-wire JSON was mangled, inflating tokens and making retrieval unreadable for Chinese users. Added `ensure_ascii=False` to every `json.dumps` call inside `handle_tool_call` — both the tool-result branches called out in the issue (memos_search, memos_get trace/policy/world_model, memos_timeline, memos_skill_list, memos_environment list+query, memos_skill_get) and the error / fallthrough branches. The uniform pattern matches the three `json.dumps` calls elsewhere in the same file (L742/1015/1021) that already set the flag. Added `HandleToolCallEnsureAsciiTests` to `apps/memos-local-plugin/tests/python/test_hermes_provider_pipeline.py` — 8 regression cases that assert each affected tool returns raw Chinese characters, contains no `\u` escape marker, and round-trips through `json.loads`. Confirmed the tests fail without the fix and pass with it. Full test file `test_hermes_provider_pipeline.py` is 35/35 green (27 pre-existing + 8 new); `ruff check` and `ruff format --check` both clean on the touched files. Categorized as an opsp Bug quick-fix (no proposal/spec/design), one-line change per return statement. Committed on bugfix/autodev-2255-20260816025022257 and pushed to origin. Reviewers: @whipser030, @hijzy. Related Issue (Required): Fixes MemTensor#2255 ## Type of change Please delete options that are not relevant. - [ ] Bug fix (non-breaking change which fixes an issue) - [x] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] Refactor (does not change functionality, e.g. code style improvements, linting) - [ ] Documentation update ## How Has This Been Tested? Automated tests are pending. - [ ] Unit Test - [ ] Test Script Or Test Steps (please provide) - [ ] Pipeline Automated API Test (please provide) ## Checklist - [ ] I have performed a self-review of my own code - [ ] I have commented my code in hard-to-understand areas - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] I have created related documentation issue/PR in [MemOS-Docs](https://github.com/MemTensor/MemOS-Docs) (if applicable) - [x] I have linked the issue to this PR (if applicable) - [x] I have mentioned the person who will review this PR @whipser030, @hijzy please review this PR. ## Reviewer Checklist - [x] closes MemTensor#2255 - [ ] Made sure Checks passed - [ ] Tests have been provided
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Fixed #2255: Chinese memory content returned by the Hermes adapter's memos_* tools was being serialized to the host LLM as
\uXXXXescapes because everyjson.dumps(...)insideMemTensorProvider.handle_tool_call(apps/memos-local-plugin/adapters/hermes/memos_provider/init.py) relied on Python's defaultensure_ascii=True. Data in the DB was always correct; only the on-wire JSON was mangled, inflating tokens and making retrieval unreadable for Chinese users.Added
ensure_ascii=Falseto everyjson.dumpscall insidehandle_tool_call— both the tool-result branches called out in the issue (memos_search, memos_get trace/policy/world_model, memos_timeline, memos_skill_list, memos_environment list+query, memos_skill_get) and the error / fallthrough branches. The uniform pattern matches the threejson.dumpscalls elsewhere in the same file (L742/1015/1021) that already set the flag.Added
HandleToolCallEnsureAsciiTeststoapps/memos-local-plugin/tests/python/test_hermes_provider_pipeline.py— 8 regression cases that assert each affected tool returns raw Chinese characters, contains no\uescape marker, and round-trips throughjson.loads. Confirmed the tests fail without the fix and pass with it. Full test filetest_hermes_provider_pipeline.pyis 35/35 green (27 pre-existing + 8 new);ruff checkandruff format --checkboth clean on the touched files.Categorized as an opsp Bug quick-fix (no proposal/spec/design), one-line change per return statement. Committed on bugfix/autodev-2255-20260816025022257 and pushed to origin. Reviewers: @whipser030, @hijzy.
Related Issue (Required): Fixes #2255
Type of change
Please delete options that are not relevant.
How Has This Been Tested?
Automated tests are pending.
Checklist
@whipser030, @hijzy please review this PR.
Reviewer Checklist