Skip to content

Add ValidationOptions for lenient output schema validation #1260

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

Add ValidationOptions for lenient output schema validation #1260

wants to merge 3 commits into from
+265 −8

Conversation

lorenss-m
Copy link

Motivation and Context

Currently, MCP clients raise RuntimeError when tools don't return structured content matching their outputSchema. This breaks integrations with servers that have schema/implementation mismatches.

This PR adds optional lenient validation that converts these errors to warnings, allowing clients to continue operating with imperfect servers.

How Has This Been Tested?

  • Added test suite in tests/client/test_validation_options.py
  • All existing tests pass
  • Tested with real MCP servers exhibiting validation issues

Breaking Changes

None. Default behavior unchanged. Lenient validation is opt-in:

from mcp.client.session import ValidationOptions

# Opt-in to lenient validation
options = ValidationOptions(strict_output_validation=False)
session = ClientSession(read, write, validation_options=options)

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation update

Checklist

  • I have read the MCP Documentation
  • My code follows the repository's style guidelines
  • New and existing tests pass locally
  • I have added appropriate error handling
  • I have added or updated documentation as needed

Additional context

  • Added ValidationOptions class following existing patterns (e.g., TransportSecuritySettings)
  • Modified ClientSession.call_tool() to check options before raising validation errors
  • Fixed workspace config issue with structured_output_lowlevel.py by creating proper package structure

@lorenss-m lorenss-m requested a review from a team as a code owner August 10, 2025 21:22
@lorenss-m lorenss-m requested a review from dsp-ant August 10, 2025 21:22
@lorenss-m lorenss-m force-pushed the pr/validation-options branch 5 times, most recently from 26c6806 to 7722e38 Compare August 10, 2025 22:05
ochafik
ochafik reviewed Aug 19, 2025
View reviewed changes
Copy link

@ochafik ochafik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @lorenss-m, thanks for sending this!

Seems reasonable to add this option given the spec only says clients "should" validate structured results against the output schema.

I'm curious tho, which examples of non-conforming outputs did you find / how bad were they?

cc/ @bhosmer-ant since authored initial support in #993

src/mcp/client/session.py
)
else:
try:
validate(result.structuredContent, output_schema)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What error would None produce here, do we need the two branches?

src/mcp/client/session.py Outdated
class ValidationOptions(BaseModel):
"""Options for controlling validation behavior in MCP client sessions."""

strict_output_validation: bool = Field(
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wondering if we wanna make this sound more explicitly about tool call output, as there's also validation happening around elicitation (https://modelcontextprotocol.io/specification/2025-06-18/client/elicitation#security-considerations)

Maybe something like validate_structured_outputs (no sure we need strict in the name, if we don't raise an exception the output isn't really validated, warnings being often ignored)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added the validate_structured_outputs suggestion!

src/mcp/client/session.py Outdated
validate(result.structuredContent, output_schema)
except ValidationError as e:
if self._validation_options.strict_output_validation:
raise RuntimeError(f"Invalid structured content returned by tool {name}: {e}")
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: might wanna raise ... from e

Kludex
Kludex reviewed Aug 20, 2025
View reviewed changes
src/mcp/client/session.py Outdated
@@ -18,6 +18,17 @@
logger = logging.getLogger("client")


class ValidationOptions(BaseModel):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's not introduce parameters that are BaseModel's please. The best experience is usually to have flat parameters.

@lorenss-m lorenss-m closed this Aug 26, 2025
@lorenss-m lorenss-m force-pushed the pr/validation-options branch from 82db20c to 9a8592e Compare August 26, 2025 08:50
@lorenss-m lorenss-m reopened this Aug 26, 2025
@lorenss-m lorenss-m requested a review from a team as a code owner August 26, 2025 08:57
@lorenss-m lorenss-m requested a review from pcarleton August 26, 2025 08:57
@lorenss-m
Copy link
Author

Added the validate_structured_outputs suggestion!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ochafik ochafik ochafik left review comments

@Kludex Kludex Kludex left review comments

@dsp-ant dsp-ant Awaiting requested review from dsp-ant dsp-ant is a code owner automatically assigned from modelcontextprotocol/python-sdk

@pcarleton pcarleton Awaiting requested review from pcarleton pcarleton is a code owner automatically assigned from modelcontextprotocol/python-sdk-auth

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@lorenss-m @ochafik @Kludex