Skip to content

feat(#3996): resolve image output capability from models.dev - #4019

Merged
dgageot merged 2 commits into
mainfrom
gemini-output-capability
Sep 9, 2026
Merged

feat(#3996): resolve image output capability from models.dev#4019
dgageot merged 2 commits into
mainfrom
gemini-output-capability

Conversation

@aheritier

@aheritier aheritier commented Aug 22, 2026

Copy link
Copy Markdown
Collaborator

What and why

Add latest-only output_capabilities.image configuration/schema support. Explicit true or false wins. Omitted, empty, or unset metadata consults exact provider-qualified models.dev output metadata; unknown models, unavailable providers, and lookup failures remain disabled. No capability is inferred from a model-name prefix. Numbered configuration packages remain frozen.

Part of #3996. Review this PR against its immediate parent, #4017, rather than the aggregate stack against main.

Commit inventory

Head: 51559ff510bf01ea053e3923c329c80ba724dca5; parent SHA: 2c5fd913464ef376471cc098b8e9481367f16b6a.

Validation

Build, test compilation, owning-package tests and the named fixture passed at this PR head.

Exact deterministic fixture command:

go test -v -count=1 ./pkg/modelinfo ./pkg/config/latest -run 'TestResolveOutputImage|Test.*OutputCapabilities'

Matched top-level tests: pkg/config/latest: 7; pkg/modelinfo: 1.

Deterministic scope: the named local fixture exercises this PR boundary with disposable configuration/stores and fake or loopback providers as applicable. Every listed package ran nonzero matching top-level tests.

Deferred/live scope: No new live catalogue/provider run was executed; deterministic resolution covers explicit and omitted metadata. The final stack head passed build, lint, full tests, an uncached full suite, focused race tests and documentation checks in disposable environments. Remote CI is tracked by the checks below; no new paid-provider or active-database validation was run.

@aheritier aheritier added area/config For configuration parsing, YAML, environment variables area/docs Documentation changes area/providers For features/issues/fixes related to LLM providers (Bedrock, LiteLLM, Qwen, custom, etc.) area/testing Test infrastructure, CI/CD, test runners, evaluation kind/feat PR adds a new feature (maps to feat:). Use on PRs only. labels Aug 22, 2026
@aheritier
aheritier force-pushed the gemini-output-capability branch 2 times, most recently from ebd9306 to 33c6981 Compare August 24, 2026 15:12
@aheritier
aheritier force-pushed the gemini-output-capability branch from 33c6981 to 5bb315a Compare August 25, 2026 20:24
@aheritier
aheritier force-pushed the gemini-output-capability branch 2 times, most recently from 35efc65 to 5bb315a Compare September 1, 2026 16:19
@aheritier
aheritier force-pushed the gemini-output-capability branch from 5bb315a to d1bcf0a Compare September 1, 2026 16:29
aheritier

This comment was marked as resolved.

@aheritier
aheritier force-pushed the gemini-output-capability branch from a1f86ba to 0862c0a Compare September 8, 2026 06:17
@aheritier
aheritier force-pushed the gemini-output-capability branch from 0862c0a to bded66b Compare September 8, 2026 07:57
@aheritier
aheritier force-pushed the gemini-output-capability branch from bded66b to d5a25cb Compare September 8, 2026 17:31
@aheritier
aheritier force-pushed the gemini-output-capability branch from d5a25cb to 51559ff Compare September 8, 2026 20:33
@aheritier
aheritier marked this pull request as ready for review September 8, 2026 20:38
@aheritier
aheritier requested a review from a team as a code owner September 8, 2026 20:38
@dgageot
dgageot force-pushed the gemini-output-capability branch from 51559ff to 9435d35 Compare September 9, 2026 07:41
Base automatically changed from gemini-request-diagnostics to main September 9, 2026 11:56
Add ModelConfig.OutputCapabilities (output_capabilities.image in YAML), a latest-only tri-state override for a model's generative output capability. Explicit false has highest precedence, explicit true follows, and an omitted flag — including output_capabilities: {} — is left for models.dev resolution in the following commit. No capability is ever inferred from the model name.

Reject output_capabilities on first_available selectors because one block cannot unambiguously describe every candidate. Update the schema, model docs, and example, and cover true, false, omitted, and empty-block YAML states, shorthand behavior, cloning, schema strictness, and selector validation.
Resolve image-output capability with one explicit precedence chain: false disables it, true enables it, and an omitted image flag — whether output_capabilities is absent or {} — enables it only when the exact known models.dev record lists image in Modalities.Output. Unknown models, unavailable catalogue data, and lookup failures stay off.

Keep lookup case-insensitive and bounded by the existing models.dev timeout. Capability is never inferred from provider or model-name patterns.
@aheritier
aheritier force-pushed the gemini-output-capability branch from 9435d35 to fe1ddb9 Compare September 9, 2026 11:56
@dgageot
dgageot merged commit f2b318d into main Sep 9, 2026
19 checks passed
@dgageot
dgageot deleted the gemini-output-capability branch September 9, 2026 12:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/config For configuration parsing, YAML, environment variables area/docs Documentation changes area/providers For features/issues/fixes related to LLM providers (Bedrock, LiteLLM, Qwen, custom, etc.) area/testing Test infrastructure, CI/CD, test runners, evaluation kind/feat PR adds a new feature (maps to feat:). Use on PRs only.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants