Skip to content

Tool calling with LiteLLM and thinking models fail #765

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
gal-checksum opened this issue May 27, 2025 · 2 comments
Open

Tool calling with LiteLLM and thinking models fail #765

gal-checksum opened this issue May 27, 2025 · 2 comments
Labels
bug Something isn't working

Comments

@gal-checksum
Copy link

Describe the bug

When running the agents SDK with tool calling and a thinking model through LITELLM (e.g. sonnet 4) getting this error

litellm.exceptions.BadRequestError: litellm.BadRequestError: AnthropicException - {"type":"error","error":{"type":"invalid_request_error","message":"messages.1.content.0.type: Expected `thinking` or `redacted_thinking`, but found `text`. When `thinking` is enabled, a final `assistant` message must start with a thinking block (preceeding the lastmost set of `tool_use` and `tool_result` blocks). We recommend you include thinking blocks from previous turns. To avoid this requirement, disable `thinking`. Please consult our documentation at https://docs.anthropic.com/en/docs/build-with-claude/extended-thinking"}}

Debug information

  • Agents SDK version: 0.0.16
  • Python version 3.13

Repro steps

  1. Run the agents sdk with sonnet 4
  2. Produce a scenario that requires 2 tool calls or more
  3. Get the failure above

Expected behavior

Everything works :)

@gal-checksum gal-checksum added the bug Something isn't working label May 27, 2025
@rm-openai
Copy link
Collaborator

Can you please provide a full working script? Happy to take a look!

@gal-checksum
Copy link
Author

@rm-openai see below

from agents import (
    Agent,
    function_tool,
    RunContextWrapper,
    Runner,
    ModelSettings,
)
from dataclasses import dataclass
import asyncio
from agents.extensions.models.litellm_model import LitellmModel
import os
from openai.types import Reasoning


from dotenv import load_dotenv

# Load environment variables from .env file
load_dotenv()


@dataclass
class Count:
    count: int


@function_tool
def count(ctx: RunContextWrapper[Count]) -> str:
    """
    Increments the count by 1 and returns the count

    Returns:
        A string with the count
    """
    ctx.context.count += 1
    return f"Counted to {ctx.context.count}"


count_ctx = Count(count=0)

agent = Agent[Count](
    name="Counter Agent",
    instructions="Count until the number the user tells you to stop using count tool",
    tools=[count],
    model=LitellmModel(
        model="anthropic/claude-sonnet-4-20250514",
        api_key=os.getenv("ANTHROPIC_API_KEY"),
    ),
    model_settings=ModelSettings(
        reasoning=Reasoning(effort="high", summary="detailed")
    ),
)


async def main():
    results = await Runner.run(
        agent, input="Count to 10", context=count_ctx, max_turns=30
    )
    print(results)


if __name__ == "__main__":
    asyncio.run(main())

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants