fix(auth): ignore non-metadata JSON when probing for protected resource metadata

[easyinplay]

Why

AuthorizationManager probes its base URL first when looking for RFC 9728 protected resource metadata, and probe_resource_metadata_url treats any 200 there as "this URL is the metadata document". ResourceServerMetadata has only optional fields, so an unrelated JSON object deserializes into an all-None value, and validate_resource_metadata_resource then fails hard with Protected resource metadata missing required resource field. The error propagates out of resolve_metadata(), so the .well-known fallbacks on the following lines never run.

Servers that answer GET / with a JSON health payload hit this. Against https://mcp.tavily.com today:

GET https://mcp.tavily.com/
200 application/json  {"status":"healthy","message":"MCP server is running"}

GET https://mcp.tavily.com/.well-known/oauth-protected-resource
404

GET https://mcp.tavily.com/.well-known/oauth-protected-resource/mcp
200 {"resource":"https://mcp.tavily.com/mcp",
     "authorization_servers":["https://mcp.tavily.com/"],
     "scopes_supported":["openid","offline_access"],
     "bearer_methods_supported":["header"]}

GET https://mcp.tavily.com/.well-known/oauth-authorization-server
200 {"issuer":"https://mcp.tavily.com/","authorization_endpoint":"https://mcp.tavily.com/authorize",
     "token_endpoint":"https://mcp.tavily.com/token","registration_endpoint":"https://mcp.tavily.com/register"}

The server publishes valid metadata at both well-known locations, using the RFC 9728 path-insertion form. resolve_metadata() still returns Metadata error: Protected resource metadata missing required resource field, because it stops at the health payload and never reaches either. The workaround is to configure the endpoint path rather than the host, which is not always what the caller has.

This is the JSON-object half of #810. That PR made a non-JSON body at the base URL a soft failure so discovery could continue. A JSON body that is not a metadata document still deserializes, so it takes the hard path instead.

Sampling the origin root of 36 reachable public remote MCP servers, 18 answer 200. All 18 are currently read as "this URL is the protected resource metadata document"; 17 of them escape only because their body is HTML or markdown rather than a JSON object.

Standards

RFC 9728 Section 3 requires the metadata document to live at a URL formed by inserting a well-known URI string into the resource identifier, and Section 5.1 lets a 401 point at it through WWW-Authenticate. The 2026-07-28 authorization server discovery requirements list the same two mechanisms and no others. The resource URL itself is not a metadata location under either, so a 200 from it carries no metadata claim.

RFC 9728 Section 3.2 allows additional members in the document, so tightening deserialization is not an option here: real documents carry fields this struct does not model, including the bearer_methods_supported in the Tavily document above.

What this changes

fetch_resource_metadata_from_url returns Ok(None) when the parsed document has none of resource, authorization_server, or authorization_servers, with a debug! line, matching how the same function already handles a non-200 status and a body that is not JSON. Discovery then continues to the well-known paths and to authorization server metadata.

A document carrying any of those fields still goes through validate_resource_metadata_resource unchanged, so a server that advertises a metadata URL through WWW-Authenticate and serves a malformed document there still gets a hard error. protected_resource_discovery_rejects_missing_resource and protected_resource_discovery_rejects_mismatched_resource both reach the document through an explicit challenge pointer, and both stay green.

Alternative

The narrower reading is that the base URL should never be treated as a metadata location at all, only as a source of a WWW-Authenticate challenge. That is what the TypeScript SDK does: discoverOAuthProtectedResourceMetadata only ever fetches /.well-known/oauth-protected-resource{path} or a URL taken from the challenge. Scoping the StatusCode::OK arm of probe_resource_metadata_url to the well-known probe would have the same effect here, and no existing test covers the base-URL-200 path, so that shape stays green too. I went with the document-shape check because it keeps working for servers that do serve metadata at the endpoint itself. Happy to send the other shape instead if you prefer it.

Test plan

resolve_metadata_ignores_non_metadata_json_at_the_base_url mirrors resolve_metadata_reports_authorization_server_metadata, with the base URL answering a health payload twice instead of 404: once for the probe, once for the fetch, which is what a real server does. Before the change it fails with

called `Result::unwrap()` on an `Err` value: MetadataError("Protected resource metadata missing required resource field")

and after it passes.

cargo +nightly fmt --all
cargo clippy --all-targets --all-features -- -D warnings
cargo test -p rmcp --lib --features auth

cargo test -p rmcp --lib --features auth reports 373 passed; 1 failed. The one failure is default_http_client_preserves_connection_failure_cause, which asserts on an OS connection-refused string and fails the same way on an unmodified checkout of this branch point on a non-English Windows host.

[easyinplay]

Both are right, and they are the same seam from two sides: the guard sat in the shared fetch, while the two callers need opposite answers to "this document is not metadata".

Your two comments also sort into two axes, which is what I used to look for the rest of them:

• what the document is — every field of ResourceServerMetadata is optional, so the type cannot tell "not this document" from "this document is broken"
• where that decision is made — deciding once, outside the loop, is what let a single answer end discovery

Four commits. The first two are your comments; the last two are the same two axes applied to the rest of the path, so this does not come back a fifth time.

The loop (83743b0). discover_resource_metadata_url returned the first candidate that answered 200 and left; the body was read afterwards, outside the loop, so a candidate answering 200 with something else was only recognised once the remaining candidates were gone. The probe already held that body and discarded it, which also cost the winning candidate a second GET. The loop body now reads the body it already has, so a candidate that is not the document costs one request and the loop moves on.

Where the url came from (b165775). A .well-known candidate is a guess, so its answer only rules out that candidate. An advertised url has no better alternative to move on to, so it reports what the server got wrong instead of degrading quietly.

Origin has to survive the next hop (f2cbb68). The field-presence guard was only half the decision. validate_resource_metadata_resource ran afterwards, outside the loop, with four fatal returns that did not look at origin — so a .well-known candidate answering {"error":"not_found","resource":"/.well-known/oauth-protected-resource"} (a catch-all handler, which is what makes this common) failed on resource field is not a valid URL and took the whole run down: remaining candidates untried, authorization server discovery untried, legacy fallback unreachable. That is the bug this PR started from, reached through a different field. JSON parse, field presence and validation now happen in one place and all three answer by origin.

Nothing is requested twice (aaf529e + e7cebf1). Two leftovers on that axis:

• An advertised url that answers 404, or HTML, was soft — but the block returned instead of falling through, so a challenge pointing at a .well-known url that a reverse proxy does not route skipped the candidate loop entirely, including the path where the document actually was. It now falls through. The split is whether the server answered a document there: 200 with a body that parses is answered-and-wrong (hard), 404 or non-JSON is not-here (soft, keep looking).
• With the loop no longer stopping at the first 401, a pointer from one candidate can be another candidate, so that url could be fetched twice. A HashSet of what has been requested covers both that and the advertised pointer.

e7cebf1 is the same idea one level down: a server publishing both authorization_server and authorization_servers with the same value — the usual way to straddle the draft and RFC 9728 — had that server walked twice, all four well-known shapes each time.

Six new tests, each verified to fail with its own half reverted. Two examples:

• reverting the validation split fails with MetadataError("Protected resource metadata resource field is not a valid URL") — the catch-all case above
• reverting the fall-through leaves the challenge test resolving past the document to an authorization server it was never meant to reach

Two existing tests fed the document twice to satisfy the duplicated GET; both lost the duplicate.

Three things I noticed on the same thread and left out of this PR, since they are wider than what it started as. Happy to open issues, or fold any of them in if you would rather:

1. fetch_authorization_metadata reads a url from generate_discovery_urls and a url named in a protected resource document's authorization_servers the same way — both soft. If none of the named servers can be reached, resolution continues on to the legacy endpoints, so the document's choice of authorization server does not reach the caller. source does carry LegacyEndpointFallback, so the information is available to callers that read it. Every named server being rejected by the SSRF guard ends up on the same path. Whether those two are worth telling apart from an ordinary fallback looks more like a design call than a fix, so I left it alone.
2. When an advertised url is soft-failed on the challenge path, resolve_metadata re-runs the whole discovery, so a url that is both the challenge pointer and a well-known candidate is fetched twice across the two passes.
3. An issuer mismatch aborts the authorization server loop rather than trying the next candidate. That reads deliberate to me (fail closed on a poisoned server), and it matches how transport errors are handled — but no test covers a document naming more than one server, so nothing pins the intent.

One behaviour note on this PR as a whole: 09c5e51 means a server publishing the document directly at the resource url is no longer discovered that way. That is the point of your second comment, but it is a break for anyone relying on it and no test records it as intentional.

cargo clippy --all-targets --all-features -- -D warnings is clean and cargo test -p rmcp --all-features is 761 passed, with the same platform-worded connection-error assertion failing as before.