Skip to content

fix(openai-compatible): consistently apply configured reasoning effort - #1604

Merged
edelauna merged 2 commits into
Zoo-Code-Org:mainfrom
JunyongParkDev:fix/openai-compatible-reasoning-effort
Sep 12, 2026
Merged

fix(openai-compatible): consistently apply configured reasoning effort#1604
edelauna merged 2 commits into
Zoo-Code-Org:mainfrom
JunyongParkDev:fix/openai-compatible-reasoning-effort

Conversation

@JunyongParkDev

Copy link
Copy Markdown
Contributor

Related GitHub Issue

Closes: #993 #1524

Description

OpenAI Compatible could display a selected reasoning effort such as max while sending a stale top-level value such as low or medium. The settings UI stores its selection in openAiCustomModelInfo.reasoningEffort, but request construction previously gave the shared top-level reasoningEffort priority.

  • Resolve effort from the custom model setting so requests use the value shown in the UI. Preserve explicit disable behavior and compatibility with profiles whose enable flag is unset.
  • Apply the resolved reasoning parameters consistently to streaming and non-streaming chat, the o1/o3/o4 request path, and single completions.
  • Restrict DeepSeek's resolved request parameters to known supported models. This prevents inherited single completions from applying V4 reasoning fields to an unknown model ID that uses fallback metadata.
  • Cover request construction, cached edits before Save, profile persistence, and saved settings returned to the webview with regression tests.

This improves provider reliability without changing the settings UI, storage format, or the shared reasoning resolver's precedence for other providers.

Test Procedure

Validated locally on macOS with Node.js 22.23.1 and pnpm 10.8.1.

The request tests inspect the arguments passed to the OpenAI SDK. They cover conflicting settings (max versus low, and high/xhigh versus medium), stale none/disable values, explicit disable, an unset enable flag, and missing custom effort across all five request paths. DeepSeek tests also verify omission for unknown models and disabled reasoning, and max for a supported model.

Settings tests verify that selecting max updates custom model metadata, unsaved edits survive a live state refresh, Save sends the cached value, and persistence preserves the settings for enabled, disabled, and unset states.

After installing dependencies, reproduce the full package checks from the repository root. Finish bundling before running the test suites:

pnpm bundle
VITEST_MAX_WORKERS=4 pnpm --dir src exec vitest run
VITEST_MAX_WORKERS=4 pnpm --dir webview-ui exec vitest run
pnpm --dir src check-types
pnpm --dir webview-ui check-types
pnpm lint

Results:

Check Result
OpenAI Compatible and four inheriting providers 260 passed, 1 existing skip
Full extension test suite 8,349 passed, 39 existing skips
Full webview test suite 1,859 passed
Type checks, repository lint, and changed-file formatting Passed

Coverage from the five provider suites is 95.40% lines / 88.19% branches for openai.ts and 100% lines / 84.37% branches for deepseek.ts. The added DeepSeek condition is exercised in both directions. The core OpenAI request tests also detected all five independently reverted fix variants.

Pre-Submission Checklist

  • Issue Linked: This PR is linked to an approved GitHub Issue (see "Related GitHub Issue" above). The issues are linked; maintainer approval and assignment have not been verified.
  • Scope: My changes are focused on the linked issue (one major feature/fix per PR).
  • Self-Review: I have performed a thorough self-review of my code.
  • Testing: New and/or updated tests have been added to cover my changes.
  • Visual Snapshot (UI changes only): Not applicable; there are no changes to rendered UI, layout, or styling.
  • Documentation Impact: I have considered if my changes require documentation updates (see "Documentation Updates" below).
  • Contribution Guidelines: I have read and agree to the Contributor Guidelines.

Visual Snapshots

Not applicable. The UI changes in this PR are limited to tests.

Videos (interaction / animation only)

Not applicable. No new interaction or animation is introduced.

Documentation Updates

  • No documentation updates are required.

Additional Notes

Get in Touch

Use the custom model effort selected in settings instead of stale top-level
values, and propagate it consistently across streaming, non-streaming,
O-series, and single-completion requests while respecting explicit disable.

Keep inherited DeepSeek completions from sending V4 reasoning fields for
unknown model IDs. Add regression coverage for request paths and support
boundaries.

Signed-off-by: JunyongParkDev <jun94.park@samsung.com>
Verify that the OpenAI-compatible selector updates custom model metadata,
unsaved edits survive live state refreshes, and Save sends the cached value.
Cover profile reload and storage-to-webview round trips with reasoning
enabled, disabled, and unset while a stale top-level effort remains.

Signed-off-by: JunyongParkDev <jun94.park@samsung.com>
@coderabbitai

coderabbitai Bot commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

Review Change StackReview Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Advanced

Run ID: 9f6f5bb1-65eb-4226-88fc-b3db29a6dd3b

📥 Commits

Reviewing files that changed from the base of the PR and between 01c7357 and 426a191.

📒 Files selected for processing (8)
  • src/api/providers/__tests__/deepseek.spec.ts
  • src/api/providers/__tests__/openai.spec.ts
  • src/api/providers/deepseek.ts
  • src/api/providers/openai.ts
  • src/core/config/__tests__/ProviderSettingsManager.spec.ts
  • src/core/webview/__tests__/ClineProvider.spec.ts
  • webview-ui/src/components/settings/__tests__/ApiOptions.spec.tsx
  • webview-ui/src/components/settings/__tests__/SettingsView.unsaved-changes.spec.tsx

Included review availability: Your plan provides up to 4 included reviews per hour; 3 remain after this review.

📜 Recent review details
⚠️ CI failures not shown inline (2)

GitHub Actions: Visual Regression / 2_webview-visual.txt: fix(openai-compatible): consistently apply configured reasoning effort

Conclusion: failure

View job details

##[group]Run pnpm --filter `@roo-code/vscode-webview` test:visual
 �[36;1mpnpm --filter `@roo-code/vscode-webview` test:visual�[0m
 shell: sh -e {0}
 env:
   PNPM_HOME: /github/home/setup-pnpm/node_modules/.bin
   STORE_PATH: /__w/.pnpm-store/v10
 ##[endgroup]
 > `@roo-code/vscode-webview`@ test:visual /__w/Zoo-Code/Zoo-Code/webview-ui
 > playwright test -c playwright-ct.config.ts
 Running 52 tests using 2 workers
   ✓   1 [chromium] › src/components/chat/__tests__/ChatTextArea.visual.tsx:7:2 › renders the production chat composer in the VS Code dark theme (6.3s)
   ✓   2 [chromium] › src/components/chat/__tests__/Announcement.links.visual.tsx:4:1 › announcement links open exactly once through the extension host (8.2s)
   ✓   3 [chromium] › src/components/chat/__tests__/ChatTextArea.visual.tsx:7:2 › renders the production chat composer in the VS Code light theme (3.4s)
   ✓   4 [chromium] › src/components/chat/__tests__/ChatTextArea.visual.tsx:7:2 › renders the production chat composer in the VS Code high-contrast theme (3.3s)
   ✓   5 [chromium] › src/components/chat/__tests__/ChatTextArea.visual.tsx:7:2 › renders the production chat composer in the VS Code high-contrast-light theme (3.5s)
   ✓   6 [chromium] › src/components/chat/__tests__/ThemeAwareControls.visual.tsx:36:2 › renders selectors and confirmation dialogs in the VS Code dark theme (3.2s)
   ✓   7 [chromium] › src/components/chat/__tests__/ThemeAwareControls.visual.tsx:36:2 › renders selectors and confirmation dialogs in the VS Code light theme (2.5s)
   ✓   8 [chromium] › src/components/chat/__tests__/ThemeSensitiveStatus.visual.tsx:7:2 › audits status controls in the VS Code dark theme (2.4s)
   ✓   9 [chromium] › src/components/chat/__tests__/ThemeSensitiveStatus.visual.tsx:7:2 › audits status controls in the VS Code light theme (2.5s)
   ✓  10 [chromium] › src/components/chat/__tests__/ThemeSensitiveStatus.visual.tsx:7:2 › audits status controls in the VS Code high-contrast theme (2.5s)
   ✓  12 [chro...

GitHub Actions: Visual Regression / webview-visual: fix(openai-compatible): consistently apply configured reasoning effort

Conclusion: failure

View job details

##[group]Run pnpm --filter `@roo-code/vscode-webview` test:visual
 �[36;1mpnpm --filter `@roo-code/vscode-webview` test:visual�[0m
 shell: sh -e {0}
 env:
   PNPM_HOME: /github/home/setup-pnpm/node_modules/.bin
   STORE_PATH: /__w/.pnpm-store/v10
 ##[endgroup]
 > `@roo-code/vscode-webview`@ test:visual /__w/Zoo-Code/Zoo-Code/webview-ui
 > playwright test -c playwright-ct.config.ts
 Running 52 tests using 2 workers
   ✓   1 [chromium] › src/components/chat/__tests__/ChatTextArea.visual.tsx:7:2 › renders the production chat composer in the VS Code dark theme (6.3s)
   ✓   2 [chromium] › src/components/chat/__tests__/Announcement.links.visual.tsx:4:1 › announcement links open exactly once through the extension host (8.2s)
   ✓   3 [chromium] › src/components/chat/__tests__/ChatTextArea.visual.tsx:7:2 › renders the production chat composer in the VS Code light theme (3.4s)
   ✓   4 [chromium] › src/components/chat/__tests__/ChatTextArea.visual.tsx:7:2 › renders the production chat composer in the VS Code high-contrast theme (3.3s)
   ✓   5 [chromium] › src/components/chat/__tests__/ChatTextArea.visual.tsx:7:2 › renders the production chat composer in the VS Code high-contrast-light theme (3.5s)
   ✓   6 [chromium] › src/components/chat/__tests__/ThemeAwareControls.visual.tsx:36:2 › renders selectors and confirmation dialogs in the VS Code dark theme (3.2s)
   ✓   7 [chromium] › src/components/chat/__tests__/ThemeAwareControls.visual.tsx:36:2 › renders selectors and confirmation dialogs in the VS Code light theme (2.5s)
   ✓   8 [chromium] › src/components/chat/__tests__/ThemeSensitiveStatus.visual.tsx:7:2 › audits status controls in the VS Code dark theme (2.4s)
   ✓   9 [chromium] › src/components/chat/__tests__/ThemeSensitiveStatus.visual.tsx:7:2 › audits status controls in the VS Code light theme (2.5s)
   ✓  10 [chromium] › src/components/chat/__tests__/ThemeSensitiveStatus.visual.tsx:7:2 › audits status controls in the VS Code high-contrast theme (2.5s)
   ✓  12 [chro...
🧰 Additional context used
📓 Path-based instructions (7)
Treat model, provider, MCP, path, command, and tool data as untrusted.

⚙️ CodeRabbit configuration file

Files:

  • src/api/providers/__tests__/deepseek.spec.ts
  • src/api/providers/deepseek.ts
  • src/api/providers/openai.ts
  • src/api/providers/__tests__/openai.spec.ts
For persisted settings, verify the complete schema/storage/runtime/webview round trip, shared default semantics, and focused true plus false/unset tests.

⚙️ CodeRabbit configuration file

Files:

  • src/core/config/__tests__/ProviderSettingsManager.spec.ts
  • src/core/webview/__tests__/ClineProvider.spec.ts
  • webview-ui/src/components/settings/__tests__/ApiOptions.spec.tsx
  • webview-ui/src/components/settings/__tests__/SettingsView.unsaved-changes.spec.tsx
Require regression coverage at the lowest valid harness with behavior-focused assertions, including relevant negative, error, false/unset, and boundary cases.

⚙️ CodeRabbit configuration file

Files:

  • src/api/providers/__tests__/deepseek.spec.ts
  • src/core/config/__tests__/ProviderSettingsManager.spec.ts
  • src/core/webview/__tests__/ClineProvider.spec.ts
  • src/api/providers/__tests__/openai.spec.ts
  • webview-ui/src/components/settings/__tests__/ApiOptions.spec.tsx
  • webview-ui/src/components/settings/__tests__/SettingsView.unsaved-changes.spec.tsx
Check strict typing and exhaustive behavior across normal, boundary, error, cancellation, retry, and compatibility paths.

⚙️ CodeRabbit configuration file

Files:

  • src/api/providers/__tests__/deepseek.spec.ts
  • src/api/providers/deepseek.ts
  • src/core/config/__tests__/ProviderSettingsManager.spec.ts
  • src/core/webview/__tests__/ClineProvider.spec.ts
  • src/api/providers/openai.ts
  • src/api/providers/__tests__/openai.spec.ts
  • webview-ui/src/components/settings/__tests__/ApiOptions.spec.tsx
  • webview-ui/src/components/settings/__tests__/SettingsView.unsaved-changes.spec.tsx
Check React state and effect dependencies, cleanup, accessibility, i18n, and light/dark theme behavior.

⚙️ CodeRabbit configuration file

Files:

  • webview-ui/src/components/settings/__tests__/ApiOptions.spec.tsx
  • webview-ui/src/components/settings/__tests__/SettingsView.unsaved-changes.spec.tsx
Verify extension/webview contracts, cancellation and error propagation, VS Code lifecycle correctness, and behavior under retries and partial failure.

⚙️ CodeRabbit configuration file

Files:

  • src/api/providers/__tests__/deepseek.spec.ts
  • src/api/providers/deepseek.ts
  • src/core/config/__tests__/ProviderSettingsManager.spec.ts
  • src/core/webview/__tests__/ClineProvider.spec.ts
  • src/api/providers/openai.ts
  • src/api/providers/__tests__/openai.spec.ts
Act as an adversarial second-opinion reviewer.

⚙️ CodeRabbit configuration file

Files:

  • src/api/providers/__tests__/deepseek.spec.ts
  • src/api/providers/deepseek.ts
  • src/core/config/__tests__/ProviderSettingsManager.spec.ts
  • src/core/webview/__tests__/ClineProvider.spec.ts
  • src/api/providers/openai.ts
  • src/api/providers/__tests__/openai.spec.ts
  • webview-ui/src/components/settings/__tests__/ApiOptions.spec.tsx
  • webview-ui/src/components/settings/__tests__/SettingsView.unsaved-changes.spec.tsx
🔇 Additional comments (8)
src/api/providers/deepseek.ts (1)

107-113: LGTM!

src/api/providers/openai.ts (1)

239-239: LGTM!

Also applies to: 301-303, 318-318, 357-357, 374-374, 409-409

src/api/providers/__tests__/openai.spec.ts (1)

11-11: LGTM!

Also applies to: 954-1035, 1038-1065

webview-ui/src/components/settings/__tests__/SettingsView.unsaved-changes.spec.tsx (1)

1-1: LGTM!

Also applies to: 271-271, 290-290, 685-734

src/api/providers/__tests__/deepseek.spec.ts (1)

151-175: LGTM!

src/core/config/__tests__/ProviderSettingsManager.spec.ts (1)

7-7: LGTM!

Also applies to: 488-509

src/core/webview/__tests__/ClineProvider.spec.ts (1)

12-12: LGTM!

Also applies to: 22-22, 1484-1501

webview-ui/src/components/settings/__tests__/ApiOptions.spec.tsx (1)

197-204: LGTM!

Also applies to: 529-534, 552-557


📝 Summary

Summary by CodeRabbit

  • New Features
    • Reasoning-effort settings now respect each model’s supported capabilities and custom model configuration.
    • OpenAI-compatible models can use custom reasoning-effort levels, including model-specific options such as “max.”
    • Reasoning settings are applied consistently across streaming, non-streaming, and completion requests.
  • Bug Fixes
    • Unknown DeepSeek models no longer receive unsupported reasoning settings.
    • Custom reasoning-effort changes remain intact while editing and after saving and reloading provider settings.

Walkthrough

The change resolves reasoning effort from custom model metadata, restricts DeepSeek reasoning metadata to supported models, propagates reasoning parameters across OpenAI request paths, and adds coverage for persistence and unsaved settings behavior.

Changes

Reasoning effort handling

Layer / File(s) Summary
Provider model resolution and requests
src/api/providers/openai.ts, src/api/providers/deepseek.ts, src/api/providers/__tests__/*
OpenAI requests use reasoning parameters resolved from custom model metadata. DeepSeek exposes reasoning metadata only for supported models. Tests cover streaming, non-streaming, completion, and custom-model cases.
Provider settings persistence
src/core/config/__tests__/ProviderSettingsManager.spec.ts, src/core/webview/__tests__/ClineProvider.spec.ts
Tests verify that reasoning settings and custom model metadata survive save, reload, and webview state updates.
Reasoning effort editing and unsaved state
webview-ui/src/components/settings/__tests__/ApiOptions.spec.tsx, webview-ui/src/components/settings/__tests__/SettingsView.unsaved-changes.spec.tsx
Tests verify model-specific effort options, custom "max" selection, and preservation of edits until Save.

Priority: ➖ Normal

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Bug fix · Severity of issue fixed: Medium

Sequence Diagram(s)

sequenceDiagram
  participant SettingsView
  participant ApiOptions
  participant ProviderSettings
  SettingsView->>ApiOptions: edit reasoning effort
  ApiOptions->>ProviderSettings: cache enablement and custom effort
  SettingsView->>ProviderSettings: save configuration
  ProviderSettings-->>SettingsView: return persisted settings
Loading

Merge Risk: ⚪ Minimal · up to 426a1

The reasoning-effort propagation and settings-state changes have no identified merge-blocking risk.

🚥 Pre-merge checks | ✅ 8
✅ Passed checks (8 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Issue #993 requires the selected custom reasoning effort to reach local OpenAI-compatible requests instead of a stale top-level value. OpenAiHandler.getModel() resolves reasoningEffort from `openA…
Out of Scope Changes check ✅ Passed The changes stay within Issue #993. They update OpenAI-compatible request construction, preserve resolver and storage behavior, and add regression coverage for request construction and settings persis…
Regression Evidence ✅ Passed Focused coverage is present at the provider and settings layers. openai.spec.ts asserts SDK request bodies for streaming and non-streaming chat, both O3 paths, and completePrompt, including custom…
Security Boundaries ✅ Passed PASS. The two changed production files only adjust reasoning request construction. OpenAI spreads the resolver's literal reasoning payload into requests, and the reasoning values are constrained by …
Persistence Integrity ✅ Passed No changed persistence path exists. The only non-test changes are in src/api/providers/openai.ts and src/api/providers/deepseek.ts, and they change request parameter construction. The persistence …
Lifecycle Resource Cleanup ✅ Passed PASS. The changed production paths only compute model metadata and spread reasoning parameters into existing OpenAI/DeepSeek request objects. The diff adds no listener, watcher, timer, task, provider …
Title check ✅ Passed The title clearly identifies the main change: consistent application of configured reasoning effort for OpenAI-compatible providers.
Description check ✅ Passed The description follows the required template and provides linked issues, implementation details, test procedures, test results, checklist status, snapshot and documentation notes. The Get in Touch se…
✨ Finishing Touches 💡 1
🛠️ Fix failing CI checks 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

Review status

Thanks for contributing. This comment tracks the review sequence and the next action.

Current step: The required review sequence passed. Remaining merge requirements apply.

Review-state labels are managed by this workflow; do not edit them manually.

@codecov

codecov Bot commented Sep 11, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@github-actions github-actions Bot added coderabbit-review-active Required CI passed; CodeRabbit review is active awaiting-coderabbit Waiting for CodeRabbit to approve the latest commit labels Sep 11, 2026
@github-actions github-actions Bot added awaiting-maintainer CodeRabbit approved; waiting for a human maintainer and removed coderabbit-review-active Required CI passed; CodeRabbit review is active awaiting-coderabbit Waiting for CodeRabbit to approve the latest commit labels Sep 11, 2026
@edelauna
edelauna added this pull request to the merge queue Sep 12, 2026
@github-actions github-actions Bot removed the awaiting-maintainer CodeRabbit approved; waiting for a human maintainer label Sep 12, 2026
Merged via the queue into Zoo-Code-Org:main with commit d5f7795 Sep 12, 2026
24 of 25 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Reasoning level is always 'medium' for local OpenAI compatible providers

2 participants