-
Notifications
You must be signed in to change notification settings - Fork 387
spec: Dynamic View Content via embedded resources #699
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -225,6 +225,30 @@ interface UIResourceMeta { | |
| * - omitted: host decides border | ||
| */ | ||
| prefersBorder?: boolean, | ||
| /** | ||
| * MIME types of dynamic content payloads this View renders | ||
| * | ||
| * When present, the View acts as a renderer for typed payloads returned | ||
| * by its associated tools as embedded resources (see Data Passing: | ||
| * Dynamic View Content). Does not affect the resource's own `mimeType`, | ||
| * which remains `text/html;profile=mcp-app`. | ||
| * | ||
| * This is a validation contract, not a routing mechanism: it declares | ||
| * what the View can parse so hosts can review, prefetch, type-filter, | ||
| * and size-limit at connection time. Routing is implicit: payloads are | ||
| * delivered to the calling tool's declared View (see Data Passing: | ||
| * Dynamic View Content). | ||
| * | ||
| * Single-format renderers declare one entry. Multiple entries support: | ||
| * distinct payload types rendered by one View (e.g., a chart format | ||
| * alongside GeoJSON), format version migration (e.g., | ||
| * `"application/a2ui+json;v=2"` alongside `"application/a2ui+json"`), | ||
| * or separate full-document and incremental-update types. | ||
| * | ||
| * @example | ||
| * ["application/a2ui+json"] | ||
| */ | ||
| contentMimeTypes?: string[], | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is interesting. As far as I can tell, this lets a single resource (likely a static renderer) handle multiple EmbeddedResource types in a single tool call result, right? So I think it unlocks things like:
Do I have this right? Are there known use cases where a single view wants to mix types like this? If it's not common / nobody will use it, I'm wondering if we should just make this a single string for now, but I guess that makes forward compat hard if the use case ever arises... |
||
| } | ||
| ``` | ||
|
|
||
|
|
@@ -254,6 +278,7 @@ The resource content is returned via `resources/read`: | |
| }; | ||
| domain?: string; | ||
| prefersBorder?: boolean; | ||
| contentMimeTypes?: string[]; // Dynamic content payload types this View renders | ||
| }; | ||
| }; | ||
| }]; | ||
|
|
@@ -262,7 +287,7 @@ The resource content is returned via `resources/read`: | |
|
|
||
| #### Metadata Location | ||
|
|
||
| `UIResourceMeta` (CSP, permissions, domain, prefersBorder) may be provided on either or both: | ||
| `UIResourceMeta` (CSP, permissions, domain, prefersBorder, contentMimeTypes) may be provided on either or both: | ||
|
|
||
| - **`resources/list`:** On the resource entry's `_meta.ui` field. Useful as a static default that hosts can review at connection time. | ||
| - **`resources/read`:** On each content item's `_meta.ui` field. Useful for per-response overrides or dynamic metadata that is only known at read time. | ||
|
|
@@ -1378,6 +1403,8 @@ View behavior (optional): | |
|
|
||
| Host MUST send this notification when tool execution completes (if the View is displayed during tool execution). | ||
|
|
||
| When the host has negotiated dynamic content support (see Client\<\>Server Capability Negotiation), the delivered `CallToolResult` MUST include, unmodified, any embedded resource content blocks marked with `_meta.ui.content`, except blocks dropped under the validation and size rules in Data Passing: Dynamic View Content. Hosts MAY omit unmarked content blocks per their existing policies. | ||
|
|
||
| `ui/notifications/tool-cancelled` - Tool execution was cancelled | ||
|
|
||
| ```typescript | ||
|
|
@@ -1719,6 +1746,7 @@ The tool's execution result: | |
|
|
||
| - `content`: Text representation for model context and text-only hosts | ||
| - `structuredContent`: Structured data optimized for UI rendering (not added to model context) | ||
| - Marked embedded resources: Typed dynamic content payloads for the View (not added to model context; see Dynamic View Content below) | ||
| - `_meta`: Additional metadata (timestamps, version info, etc.) not intended for model context | ||
|
|
||
| #### 3. Interactive Updates | ||
|
|
@@ -1736,6 +1764,97 @@ This pattern enables interactive, self-updating views. | |
|
|
||
| Note: Tools with `visibility: ["app"]` are hidden from the agent but remain callable by apps via `tools/call`. This enables UI-only interactions (refresh buttons, form submissions) without exposing implementation details to the model. See the Visibility section under Resource Discovery for details. | ||
|
|
||
| #### 4. Dynamic View Content (via embedded resources) | ||
|
|
||
| Some UI systems are generative in nature: the server produces a declarative, typed UI description at tool-call time (data conforming to a UI format, not executable code), and a generic predeclared View renders it. [A2UI](https://a2ui.org) (`application/a2ui+json`) is the primary example. | ||
|
|
||
| This section addresses **server-authored** UI descriptions: payloads built by the server, typically from data the model never sees, and excluded from model context. Model-authored descriptions, where the agent generates the UI description itself, already work without any spec change, delivered as tool *input* to a renderer View. | ||
|
|
||
| `structuredContent` is a poor fit for server-authored payloads: it is untyped (no MIME type), single-valued, and bound to the tool's `outputSchema`. And because it carries no MIME type, a host that natively renders the payload format has no standard way to recognize the payload inside it, so servers would have to build different responses for different host classes. | ||
|
|
||
| For these cases, tool results MAY carry dynamic content payloads as standard MCP embedded resource content blocks, marked for View consumption: | ||
|
|
||
| ```typescript | ||
| /** | ||
| * Marker for dynamic content payloads. Currently empty; future fields | ||
| * (e.g., renderer targeting for multi-view tool results) may be added. | ||
| */ | ||
| interface McpUiContentBlockMeta {} | ||
|
|
||
| // Embedded resource content block within CallToolResult.content: | ||
| { | ||
| type: "resource", | ||
| resource: { | ||
| uri: string, // Ephemeral payload identifier (any scheme except ui://) | ||
| mimeType: string, // MUST match a declared contentMimeTypes entry | ||
| text?: string, // Payload as string | ||
| blob?: string // OR base64-encoded payload | ||
| }, | ||
| _meta: { | ||
| ui: { | ||
| content: McpUiContentBlockMeta | ||
| } | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| Routing is implicit: marked payloads are delivered to the View of the calling tool (its `_meta.ui.resourceUri`). `contentMimeTypes` is the guardrail: the reviewable contract of what that View can parse, declared on the View resource at connection time, against which hosts validate payloads. | ||
|
|
||
| Declaring a MIME type in `contentMimeTypes` does not claim all payloads of that type for one View: a server MAY register multiple Views declaring the same type, each linked to different tools. Routing follows the tool linkage; the declaration exists so hosts can review, prefetch, type-filter, and size-limit deterministically. | ||
|
|
||
| **Requirements:** | ||
|
|
||
| - The target View MUST declare the payload's `mimeType` in its `contentMimeTypes` (see UI Resource Format) | ||
| - Payload URIs are ephemeral identifiers per RFC 3986; servers are NOT required to serve them via `resources/read`. The `ui://` scheme MUST NOT be used for payload URIs; it remains reserved for renderable UI resources | ||
| - A tool result MAY contain multiple marked payloads; Views SHOULD process them in array order | ||
| - Marked payloads are presentation data. Consistent with `structuredContent`, servers SHOULD still return a meaningful text `content` block for model context and text-only hosts | ||
|
|
||
| **Host behavior** (when dynamic content support is negotiated, for tools linked to a View declaring `contentMimeTypes`): | ||
|
|
||
| - Host MUST deliver marked embedded resource blocks, unmodified, in the `CallToolResult` sent via `ui/notifications/tool-result`, except blocks dropped under the rules below | ||
| - Host MUST deliver marked embedded resource blocks, unmodified, in `tools/call` responses returned to Views during the interactive phase, subject to the same rules | ||
| - Host SHOULD NOT add marked payloads to model context (they are presentation data, analogous to `structuredContent`). Host MAY note their presence to the model | ||
| - Host MAY drop marked blocks whose `mimeType` is not declared in the target View's `contentMimeTypes` or not covered by the host's advertised `contentMimeTypes`, and SHOULD log such drops | ||
| - Host MAY enforce payload size limits; when dropping a payload, Host SHOULD deliver the remainder of the result rather than failing | ||
|
|
||
| When dynamic content support is not negotiated, hosts MAY strip marked blocks from delivered results; servers SHOULD NOT rely on marked payloads being delivered (see the `contentMimeTypes` extension setting). | ||
|
|
||
| No new messages are introduced: the existing `ui/notifications/tool-result` notification and proxied `tools/call` responses are the delivery channel. The View extracts marked payloads from the delivered result and renders them; interactive updates return new payloads through the same loop. | ||
|
|
||
| View-instance lifecycle is unchanged from the rest of this specification. Each agent-initiated tool call that renders UI gets its own View instance, which receives that call's marked payloads via `ui/notifications/tool-result`; successive conversation turns therefore produce successive independent surfaces, not one shared iframe. Within an instance, app-initiated `tools/call` responses return payloads to the calling instance (incremental updates to its surface). | ||
|
|
||
| **Example (A2UI renderer):** | ||
|
|
||
| ```json | ||
| // Renderer resource (predeclared, prefetchable, reviewable) | ||
| { | ||
| "uri": "ui://a2ui-server/renderer", | ||
| "name": "a2ui_renderer", | ||
| "mimeType": "text/html;profile=mcp-app", | ||
| "_meta": { | ||
| "ui": { "contentMimeTypes": ["application/a2ui+json"] } | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. this requires all a2ui mimetypes to go to the same mcp resource, what if we want different resources to handle different toolcalls, eg where two different resources are used ot render a2ui
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Preferably the renderer should be generic and not dependant on a specific tool call result. It should just be the bridge that converts any A2UI resources (returned from every tool call) into MCP Apps compatible content. There's no actualy duplication as |
||
| } | ||
| } | ||
|
|
||
| // Tool result | ||
| { | ||
| "content": [ | ||
| { "type": "text", "text": "Found 3 flights TLV→SFO. Best: UA954, $1,240." }, | ||
| { | ||
| "type": "resource", | ||
| "resource": { | ||
| "uri": "a2ui://a2ui-server/surfaces/flight-search-8f3a", | ||
| "mimeType": "application/a2ui+json", | ||
| "text": "{\"beginRendering\":{...}}" | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. how do we handle multiple createSurface calls? A2UI can create multiple surfaces, eg each conversation turn we load a new UI inline. Are all tool cals going to the same iframe for the mcp apps resource |
||
| }, | ||
| "_meta": { "ui": { "content": {} } } | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. why is this preferred to putting "_meta": { "ui": { "content": { ui_host: "ui://a2ui-server/renderer" } } } and not pre-registering the mime type with a ui resource |
||
| } | ||
| ] | ||
| } | ||
| ``` | ||
|
|
||
| The renderer translates payload-level events into `tools/call` requests (typically to tools with `visibility: ["app"]`); responses carry new marked payloads (e.g., incremental A2UI updates), which the renderer applies. A host that natively renders a payload format MAY instead advertise that format in the top-level `mimeTypes` extension setting, render marked payloads directly, and skip instantiating the HTML renderer, allowing servers to serve a single tool response to MCP Apps hosts, native hosts, and text-only hosts. | ||
|
|
||
| ### App-Provided Tools | ||
|
|
||
| Apps can register their own tools that hosts and agents can call, making apps **introspectable and accessible** to the model. This complements the existing capability where apps call server tools (via host proxy). | ||
|
|
@@ -2192,7 +2311,8 @@ Clients advertise MCP Apps support in the initialize request using the extension | |
| "capabilities": { | ||
| "extensions": { | ||
| "io.modelcontextprotocol/ui": { | ||
| "mimeTypes": ["text/html;profile=mcp-app"] | ||
| "mimeTypes": ["text/html;profile=mcp-app"], | ||
| "contentMimeTypes": ["application/a2ui+json"] | ||
| } | ||
| } | ||
| }, | ||
|
|
@@ -2206,7 +2326,8 @@ Clients advertise MCP Apps support in the initialize request using the extension | |
|
|
||
| **Extension Settings:** | ||
|
|
||
| - `mimeTypes`: Array of supported content types (REQUIRED, e.g., `["text/html;profile=mcp-app"]`) | ||
| - `mimeTypes`: Array of supported content types (REQUIRED, e.g., `["text/html;profile=mcp-app"]`). Entries are the UI resource content types the host can render; a host MAY additionally include payload formats it renders natively without an HTML renderer (see Data Passing: Dynamic View Content) | ||
| - `contentMimeTypes`: Array of dynamic content payload types the host will forward to Views per Data Passing: Dynamic View Content (OPTIONAL). Advertising a non-empty value is what negotiates dynamic content support. Hosts MAY advertise `["*"]` to indicate they forward any payload type declared by a View's `contentMimeTypes`; hosts never need to interpret payloads, only route them into the sandboxed View. Servers SHOULD check this setting before registering renderer-pattern tools and SHOULD degrade to text-only or `structuredContent`-driven variants when absent. A host that natively renders a payload format (without an HTML renderer) advertises that format in `mimeTypes` instead. | ||
|
|
||
| Future versions may add additional settings: | ||
|
|
||
|
|
@@ -2474,6 +2595,26 @@ Apps are forward-deployed emanations of server tools, running in the client cont | |
|
|
||
| See [Security Implications: App-Provided Tools Security](#5-app-provided-tools-security) for detailed considerations. | ||
|
|
||
| #### 7. Dynamic View Content via Embedded Resources | ||
|
|
||
| **Decision:** Allow tool results to carry typed dynamic content payloads as marked embedded resources, delivered to the tool's predeclared View through existing channels. | ||
|
|
||
| **Rationale:** | ||
|
|
||
| - Enables generative UI formats (e.g., A2UI) where the server emits a declarative UI description at call time and a generic predeclared renderer interprets it | ||
| - Embedded resources are typed (MIME), multi-valued, URI-addressed, and part of core MCP, unlike `structuredContent`, which remains the channel for template-bound data | ||
| - Non-MCP-Apps hosts that natively render a payload format can consume the same tool response by reading the marked embedded resource directly, enabling one server response across host classes | ||
| - Requires no new messages: delivery rides on `ui/notifications/tool-result` and proxied `tools/call` responses | ||
|
|
||
| This does not revisit decision #1 (Predeclared Resources vs. Inline Embedding): the renderer remains a predeclared, prefetchable, reviewable `ui://` resource. Embedded resources here carry only data payloads consumed by that renderer: the same template/data split as `structuredContent`, extended with typed, self-describing payloads. | ||
|
|
||
| **Alternatives considered:** | ||
|
|
||
| - **`structuredContent`:** Untyped, single-valued, `outputSchema`-bound, and invisible to native payload-format hosts | ||
| - **Dedicated `ui/notifications/content` message:** Payloads are produced *by* tool calls, and `ui/notifications/tool-result` already delivers results to the View; a parallel message carrying the same payloads would force every host and SDK to define ordering between the two for no expressive gain. It would also be invisible outside this extension, forfeiting the one-response-across-host-classes property that keeping payloads in standard `CallToolResult.content` provides. The one thing it would genuinely add, server-push content outside a tool call, is orthogonal and can be added later as a separate message | ||
| - **MIME-type inference without a marker:** Servers legitimately return embedded resources for other purposes (files, records); the explicit `_meta.ui.content` marker makes routing intent unambiguous and provides a forward-compatible slot for future fields (e.g., renderer targeting for multi-view results) | ||
| - **Per-payload renderer pointers (call-time routing):** Hosts would have nothing to validate against (any tool result could aim arbitrary payloads at any resource) and would lose connection-time reviewability, prefetching, and deterministic type-filtering. Routing therefore stays on the predeclared tool-to-View link (`resourceUri`), with `contentMimeTypes` as its guardrail, mirroring `resourceUri` + `mimeTypes` for the View itself; explicit per-payload targeting can be added later for multi-view results | ||
|
|
||
| ### Backward Compatibility | ||
|
|
||
| The proposal builds on the existing core protocol. There are no incompatibilities. | ||
|
|
@@ -2643,6 +2784,20 @@ App tools MUST be tied to the app's lifecycle: | |
| - Hosts MUST NOT persist app tool registrations across sessions | ||
| - Calling a tool from a closed app MUST return an error | ||
|
|
||
| #### 6. Dynamic Content Payloads | ||
|
|
||
| Dynamic View Content payloads are data, not code: they are interpreted by a renderer that is itself sandboxed, CSP-constrained, and reviewable under this specification's existing model. Declarative formats narrow the attack surface relative to arbitrary HTML precisely because the executable component (the renderer) is static and predeclared. | ||
|
|
||
| **View behavior:** | ||
|
|
||
| - Renderers MUST treat payloads as untrusted input (no `eval` or direct `innerHTML` of payload-derived strings) | ||
| - Payload-referenced network and media origins remain subject to the renderer's declared CSP; a payload cannot expand the View's network reach | ||
|
|
||
| **Host behavior:** | ||
|
|
||
| - Payloads flow through auditable JSON-RPC with declared MIME types; hosts MAY log, size-limit, and type-filter them | ||
| - Marked payloads are excluded from model context by default, limiting prompt injection surface | ||
|
|
||
| ### Other risks | ||
|
|
||
| - **Social engineering:** UI can still display misleading content. Hosts should clearly indicate sandboxed UI boundaries. | ||
|
|
@@ -2652,3 +2807,5 @@ App tools MUST be tied to the app's lifecycle: | |
|
|
||
| - The resource prefix `ui://` will be reserved for MCP Apps | ||
| - The label `io.modelcontextprotocol/ui` is reserved | ||
| - The `_meta.ui.content` key on tool result content blocks is reserved for Dynamic View Content | ||
| - The `contentMimeTypes` field is reserved in `UIResourceMeta` and in the `io.modelcontextprotocol/ui` extension settings | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can we add a comment on native UI host to this spec change?
It's in the conversation of this PR.
Maybe just have a section saying
mimeTypes can support other types besides text/html;profile=mcp-app. If that's supported the dynamic view content is rendered using the host's native renderer. How that's implemented is done by the mimeType's SDKs. Thsi is useful for natively rendering content such as in mobile apps that use native components over iframes.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The idea of this PR is actually to explicitly not change the mimeType the hosts need to render or understand - but to standardize the "bridge" between any other mimeType and the
text/html;profile=mcp-appthat MCP Apps hosts already know how to render.Any additional mimeType top-level support is out of this PR's scope and should be discussed carefully to understand if it should even be under the MCP Apps spec