Adopt Foundation Models OS 27 surfaces: image attachments, capabilities, context size

[james-333i]

This updates the Foundation Models adapter for OS 27 while keeping the package building unchanged on Xcode 26.

Image input: Transcript image segments were previously dropped when converting to a FoundationModels transcript. On OS 27 they now map to Transcript.Segment.attachment with an ImageAttachment, built directly for URL sources and via ImageIO decoding for data sources. Verified end to end on an iPhone running iOS 27 (image prompt answered by the on-device model).

Capabilities and context: SystemLanguageModel now exposes contextSize (back-deployed by Apple to OS 26) and supportsImageInput, which reports the OS 27 vision capability and returns false on earlier OS versions.

Generation options: the adapter now sets maximumResponseTokens on FoundationModels.GenerationOptions. The property has existed since OS 26 but was never assigned, so response token limits were silently ignored on the Foundation Models backend. Verified on device: a capped request now terminates at the limit.

The OS 27 references are behind #if compiler(>=6.4) with runtime availability checks, so Xcode 26 builds still compile and the deployment floor is unchanged

[mattt]

Hi @james-333i. Thank you for this. The maximumResponseTokens fix alone is worth having, and the image-attachment mapping is a nice preview of what's coming.

Here's what I'm thinking for the project overall (details in #210):

• Cut a 1.0 soon against the current (OS 26) Foundation Models API, with some cleanup of places where we've grown beyond it.
• Start a 2.0 for the OS 27 API, which I expect to be source-breaking in a few places (session construction, image inputs, generation options).

Since everything here is availability-gated and doesn't change the OS 26 surface, I'm inclined to take it now rather than hold it for 2.0. Two small things before I merge: can you split out the maximumResponseTokens fix as its own commit so it's easy to cherry-pick, and let me look at how contextSize / supportsImageInput line up with what we do for the LanguageModel protocol more broadly?

Does that sound reasonable?

[james-333i]

@mattt Sounds good. I've split the maximumResponseTokens fix into its own commit so it cherry-picks cleanly. On contextSize and supportsImageInput, I kept both as passthroughs on SystemLanguageModel since the LanguageModel protocol has no capability surface today.

[Copilot]

🟡 Changes recommended

URL handling, decode failures, and EXIF orientation can produce missing or incorrectly oriented image input.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Updates the Foundation Models adapter for OS 27 multimodal and model metadata APIs while retaining compatibility guards.

Changes:

• Adds image attachment conversion and capability/context reporting.
• Applies maximum response-token limits.
• Introduces OS/compiler availability guards.

File summaries

File	Description
SystemLanguageModel.swift	Adds OS 27 image support, capabilities, context size, and response limits.

Review details

Suppressed comments (2)

Sources/AnyLanguageModel/Models/SystemLanguageModel.swift:807

• A failed ImageIO decode is converted to nil, and the surrounding compactMap silently removes the image. An invalid or unsupported image can therefore produce a successful text-only request and a misleading model answer. Make this conversion throwing and propagate a clear invalid-image error instead of dropping the segment.

                    guard
                        let imageSource = CGImageSourceCreateWithData(data as CFData, nil),
                        let cgImage = CGImageSourceCreateImageAtIndex(imageSource, 0, nil)
                    else {
                        return nil

Sources/AnyLanguageModel/Models/SystemLanguageModel.swift:809

• CGImageSourceCreateImageAtIndex does not apply EXIF orientation, and the resulting CGImage no longer carries the source metadata for the attachment initializer to infer. Data-backed portrait photos can consequently be sent rotated or mirrored. Read kCGImagePropertyOrientation from imageSource and pass it to ImageAttachment.init(_:orientation:).

                        let imageSource = CGImageSourceCreateWithData(data as CFData, nil),
                        let cgImage = CGImageSourceCreateImageAtIndex(imageSource, 0, nil)
                    else {
                        return nil
                    }
                    self.init(cgImage)

• Files reviewed: 1/1 changed files
• Comments generated: 1
• Review effort level: Balanced

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[james-333i]

@mattt Rebased onto main. The image mapping now accepts only file URLs for ImageAttachment(imageURL:), matching Apple's documentation and the Llama adapter. Other URLs are dropped.

[mattt]

Hi @james-333i. This is continuing as #241, which carries your image-attachment commit as-is. Two changes there: the maximumResponseTokens commit is dropped because #230 already forwards every generation option, and contextSize is declared only on toolchains whose SDK has it (it's back-deployed to 26.0 but first appears in the 26.4 SDK, which is why the Xcode 26.0 CI jobs failed here). Closing this one in favor of that. Thanks for the device verification and the rebase!

[james-333i]

@mattt Thanks for carrying it over and for the toolchain gate on contextSize. That explains the CI failures.