Optional Fields in Agent Output Cause JSON Schema Error with AWS Bedrock

[samDobsonDev]

Summary:
When using the OpenAI Agents SDK with a custom agent output schema that includes Optional fields (defined using Pydantic), the request to AWS Bedrock fails with a 400 Bad Request error. The error indicates that the generated JSON schema is invalid according to JSON Schema Draft 2020-12. If I remove the Optional fields, the request succeeds, but then the model is forced to always populate those fields, which is not the desired behaviour.

Environment:
Agents SDK Version: 0.0.12
Model: Claude 3.5 Sonnet via AWS Bedrock with LiteLLM

Steps to Reproduce

1. Define an output schema using Pydantic, where several fields are marked as Optional, such as this example:

class Link(BaseModel):
    """Represents a clickable link."""
    label: str = Field(..., description="Text that appears in the clickable bubble. MAX 20 CHARACTERS") # Required
    url: str = Field(..., description="URL the link points to") # Required

class Body(BaseModel):
    """Represents the main body of text with an optional link."""
    text: str = Field(..., description="Main body of text. This is required.")  # Required
    link: Optional[Link] = Field(None, description="A clickable link that appears AFTER the main body of text.")  # Optional

class QuickReply(BaseModel):
    """Represents a quick reply button."""
    title: str = Field(..., description="Text that appears to the user in the bubble and also the text that is sent to the LLM when clicked. 1-2 WORDS AND MAX 20 CHARACTERS")  # Required

class Button(BaseModel):
    """Represents a button in carousel elements."""
    label: str = Field(..., description="Text that appears on the button. MAX 20 CHARACTERS") # Required
    url: str = Field(..., description="URL the button points to") # Required

class CarouselElement(BaseModel):
    """Represents an element in a carousel."""
    title: str = Field(..., description="Title of the carousel element") # Required
    subtitle: Optional[str] = Field(None, description="Subtitle of the carousel element") # Optional
    imageUrl: Optional[str] = Field(None, description="URL of the image for the carousel element") # Optional
    buttons: List[Button] = Field(default_factory=list, description="List of buttons for this carousel element") # Optional (default empty list)

class Carousel(BaseModel):
    """Represents a carousel containing multiple elements."""
    elements: List[CarouselElement] = Field(..., description="List of carousel elements") # Required

class RichResponse(BaseModel):
    """Agent response container with rich messaging components
    body: Main text content with optional link (REQUIRED)
    quick_replies: Post-message action buttons (OPTIONAL)
    carousel: Carousel containing multiple elements (OPTIONAL)
    """
    body: Body = Field(..., description="Main body of text with optional link. This is required.")  # Required
    quick_replies: Optional[List[QuickReply]] = Field(default_factory=list, description="List of quick reply buttons. These appear AFTER the main body of text. These are optional.")  # Optional (default empty list)
    carousel: Optional[Carousel] = Field(default_factory=list, description="A carousel containing multiple elements, These are optional.")  # Optional (default empty list)

3. Set this schema as the Agent's output.
4. Run the Agent, triggering a call to any model on AWS Bedrock using LitellmModel
5. Observe that the following error occurs:BedrockError: {"message":"The model returned the following errors: tools.0.custom.input_schema: JSON schema is invalid. It must match JSON Schema draft 2020-12."}

Expected Behavior

1. The Agent should accept Optional fields in the output schema without causing a JSON schema validation error.
2. The model should only populate optional fields when relevant, leaving them absent or null otherwise.

[rm-openai]

Looking into this, thanks for the report!

[rm-openai]

@samDobsonDev I think there's something specific about Bedrock that isn't working. When I use Claude directly, it works ok (script below). I don't have a way to repro on bedrock - do you have a contact there that can help? Another thing that might help is to change output_type to this:

 agent = Agent(output_type=AgentOutputSchema(RichResponse, strict_json_schema=False))

Can you try that and see if it helps?

---

Code that worked for me hitting claude directly:

from __future__ import annotations

import asyncio
from typing import Optional

from pydantic import BaseModel, Field

from agents import Agent, Runner, function_tool

"""This example uses the built-in support for LiteLLM. To use this, ensure you have the
ANTHROPIC_API_KEY environment variable set.
"""


class Link(BaseModel):
    """Represents a clickable link."""

    label: str = Field(
        ..., description="Text that appears in the clickable bubble. MAX 20 CHARACTERS"
    )  # Required
    url: str = Field(..., description="URL the link points to")  # Required


class Body(BaseModel):
    """Represents the main body of text with an optional link."""

    text: str = Field(..., description="Main body of text. This is required.")  # Required
    link: Optional[Link] = Field(
        None, description="A clickable link that appears AFTER the main body of text."
    )  # Optional


class QuickReply(BaseModel):
    """Represents a quick reply button."""

    title: str = Field(
        ...,
        description="Text that appears to the user in the bubble and also the text that is sent to the LLM when clicked. 1-2 WORDS AND MAX 20 CHARACTERS",
    )  # Required


class Button(BaseModel):
    """Represents a button in carousel elements."""

    label: str = Field(
        ..., description="Text that appears on the button. MAX 20 CHARACTERS"
    )  # Required
    url: str = Field(..., description="URL the button points to")  # Required


class CarouselElement(BaseModel):
    """Represents an element in a carousel."""

    title: str = Field(..., description="Title of the carousel element")  # Required
    subtitle: Optional[str] = Field(
        None, description="Subtitle of the carousel element"
    )  # Optional
    imageUrl: Optional[str] = Field(
        None, description="URL of the image for the carousel element"
    )  # Optional
    buttons: list[Button] = Field(
        default_factory=list, description="list of buttons for this carousel element"
    )  # Optional (default empty list)


class Carousel(BaseModel):
    """Represents a carousel containing multiple elements."""

    elements: list[CarouselElement] = Field(
        ..., description="list of carousel elements"
    )  # Required


class RichResponse(BaseModel):
    """Agent response container with rich messaging components
    body: Main text content with optional link (REQUIRED)
    quick_replies: Post-message action buttons (OPTIONAL)
    carousel: Carousel containing multiple elements (OPTIONAL)
    """

    body: Body = Field(
        ..., description="Main body of text with optional link. This is required."
    )  # Required
    quick_replies: Optional[list[QuickReply]] = Field(
        default_factory=list,
        description="list of quick reply buttons. These appear AFTER the main body of text. These are optional.",
    )  # Optional (default empty list)
    carousel: Optional[Carousel] = Field(
        default_factory=list,
        description="A carousel containing multiple elements, These are optional.",
    )  # Optional (default empty list)


@function_tool
def get_data():
    return f"Body should be like Google Maps. Quick replies for directions, hours, and phone number. 2 carousel elements with 2 buttons each."


async def main():
    agent = Agent(
        name="Assistant",
        instructions="First call the get_data tool, then use the results to respond to the user.",
        # We prefix with litellm/ to tell the Runner to use the LitellmModel
        model="litellm/anthropic/claude-3-5-sonnet-20240620",
        tools=[get_data],
        output_type=RichResponse,
    )

    result = await Runner.run(agent, "What's the best rich response?")
    print(result.final_output)


if __name__ == "__main__":
    import os

    if os.getenv("ANTHROPIC_API_KEY") is None:
        raise ValueError(
            "ANTHROPIC_API_KEY is not set. Please set it the environment variable and try again."
        )

    asyncio.run(main())

[samDobsonDev]

Hi @rm-openai. Just tried your suggestion of:

agent = Agent(output_type=AgentOutputSchema(RichResponse, strict_json_schema=False))

...but ran into the same error:

litellm.exceptions.BadRequestError: litellm.BadRequestError: BedrockException - {"message":"The model returned the following errors: tools.0.custom.input_schema: JSON schema is invalid. It must match JSON Schema draft 2020-12 (https://json-schema.org/draft/2020-12). Learn more about tool use at https://docs.anthropic.com/en/docs/tool-use."}

We do have a contact we can reach out to, I'll see if I can maybe get in touch with them and see if they've experienced anything like this before

[samDobsonDev]

So I managed to "fix" this by removing the Optional wrapper and instead using default values with the Pydantic Field metadata provider. So instead of:

link: Optional[Link] = Field(None, ...)

I'm now using:

link: Link = Field(None, ...)

I also tried using something like...

link: Link | None = Field(default=None, ...)

...but this also caused the same exception to be thrown once the request reaches Bedrock.

[rm-openai]

@samDobsonDev that's really strange. Makes me think there's something wrong about the Bedrock JSON schema validator. Especially given that it works with Anthropic directly. Anyway I'm glad you managed to get a fix working!