SEP-2350: accumulate scopes on step-up auth

[dogacancolak]

Implements MCP spec PR #2350 (closes #2349): on a 403 insufficient_scope step-up challenge, the client now unions the challenge scopes with its previously-requested scope set instead of replacing them. Stateless servers can now emit per-operation WWW-Authenticate challenges (the spec-recommended posture) without forcing the client to drop prior grants on every step-up.

Motivation and Context

The previous spec told servers to include "previously granted scopes" in insufficient_scope challenges, conflicting with RFC 6750 §3.1. The amended spec moves accumulation to the client.

The SDK was replacing client_metadata.scope with the challenge scopes, so a client doing read → write → admin operations would lose prior scopes on each step-up and trigger re-auth loops.

What changed

• 403 step-up branch in src/mcp/client/auth/oauth2.py now calls union_scopes(client_metadata.scope, challenge_scopes) instead of overwriting. The 401 branch is unchanged (replace semantics) — full re-login remains the natural down-scoping opportunity.
• Union source is client_metadata.scope (what we requested last time), not current_tokens.scope (what the AS granted). If the AS narrowed the grant at consent time, we still re-request the broader set on the next step-up rather than silently dropping scopes the user previously declined.

Behavior

• read requested → 403 scope="write" → re-auth with read write.
• read write requested, granted read only → 403 scope="delete" → re-auth with read write delete (write survives the narrow grant).
• 401 (expired token) → scope is replaced by initial set, accumulated set is dropped. This is the down-scoping path.
• Happy path with a valid token → scope untouched.

Tests

Unit tests for union_scopes.

End-to-end tests driving async_auth_flow cover the entire step-up branch, not just the union behavior:

• Step-up retry uses the new access token. After step-up succeeds, the retried request carries the stepped-up token, not the rejected one.
• Multi-step accumulation. read → step-up adds write → step-up adds admin ends with read write admin, not just admin.
• Union source is requested, not granted. If the user requested read write admin but the AS only granted read write at consent, a later step-up for delete re-authorizes with read write admin delete — the declined admin is re-requested rather than silently dropped.
• No scope in challenge falls back to PRM. A step-up challenge with no scope attribute uses the priority ladder (PRM, then AS metadata).
• Non-step-up 403 is a no-op. A 403 without insufficient-scope does not trigger re-auth.
• 401 is the down-scoping opportunity. A fresh 401 replaces an accumulated read write admin with the initial auth handshake's narrower read. Pins the invariant so a future "let's union on 401 too" change fails loudly.
• Happy path does not mutate scope. A successful request with a valid token leaves the requested scope set untouched.
• Step-up errors are logged and re-raised. Failures during step-up are surfaced rather than silently swallowed.

Limitations / follow-ups

• Refresh-token path does not re-evaluate scope. As long as the refresh token is valid, the over-permissive accumulated set stays — the 401 down-scoping path is never hit. This is a limitation of the refresh_token flow itself.

[SamMorrowDrums]

Hi @dogacancolak, I read through #2676 carefully while aligning my own server-side step-up PR (TypeScript SDK #1624) with SEP-2350. Posting cross-SDK context in case it helps.

On your two questions:

What is the recommended behaviour when that token expires and there is no refresh token?

I surveyed other SDKs. It's not clear and I've asked Den.

• Drop accumulated on 401 (full re-auth, natural down-scoping): Python main, your PR #2676 (intentional), Swift, Rust, C#, Ruby, TypeScript main.
• Union on 401 too: Go SDK, TypeScript PR #1657 (proposed).

I think your choice to drop on 401 is well argued. It matches the principle of least privilege, prevents an over-broad accumulated set from outliving a session, and respects the user's opportunity to re-consent, but could lead to lots of re-auth. The "401 is the down-scoping opportunity" sounds correct.

That said, SEP-2350 does not actually mandate this. It only mandates client-side union on 403. Two SDKs already go the other way. I am going to raise this with @dend as a spec clarification because silent divergence here will cause real interop pain. I will tag you in that thread.

Which scopes are kept during step-up? Requested or granted?

Your choice to union from client_metadata.scope (what we requested last time) rather than current_tokens.scope also makes a lot of sense. If the AS narrowed the grant at consent (user declined admin), the client should probably still re-ask for the broader requested set on the next step-up rather than silently dropping the request. Either way step up can get a user out of that one if the declined scope is required for an operation later.

The Swift SDK does it from existing token scopes and Rust does it from current_scopes. Neither addresses your scenario.

On the server side (TS SDK #1624): my updated PR now emits only the per-operation required scope by default, so a Python client applying your union_scopes(client_metadata.scope, challenge_scopes) will see exactly the SEP-2350 happy path. There is an opt-in includeGrantedScopes flag for servers that want to defend against non-accumulating clients, but that should not affect your union logic, since client_metadata.scope still tracks the requested set locally.

Thanks for pulling at these threads.