-
Notifications
You must be signed in to change notification settings - Fork 1.5k
Ordering of events in Runner.run_streamed is incorrect #583
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
Hi team, let me know if more details are needed ? Currently, there is no obvious workaround for this issue |
@rm-openai bumping this up |
Looking at the code, it seems like this is a side effect of using the same code for streaming and non streaming results. openai-agents-python/src/agents/run.py Lines 709 to 724 in 5fe096d
To resolve the issue, we can pass the reference to streamed_result or the event_queue to these methods and add to the event queue as soon as an item is available, or make a separate implementation for streaming case. Will be happy to raise a PR, if any of these look good or you have something else in mind |
Uh oh!
There was an error while loading. Please reload this page.
Describe the bug
When using
Runner.run_streamed
, the events forrun_item_stream_event
are not received until the agent message and tool execution completes. Ifon_tool_start
andon_tool_end
hooks are used, the events are streamed even after these methods return.Debug information
v0.0.12
3.10.14
Repro steps
Adapted from https://github.com/openai/openai-agents-python/blob/main/examples/basic/stream_items.py
Current Output
=== Run starting === -- Agent updated -- -- Raw Output: Item done: message -- -- Raw Output: Item done: function_call -- -- Hook: On Tool Start -- -- Tool Execution -- -- Hook: On Tool End -- -- Run Item: Message Output -- -- Run Item: Tool Called -- -- Run Item: Tool Output -- -- Raw Output: Item done: message -- -- Run Item: Message Output -- === Run complete ===
Expected Output
Expected behavior
The
run_item_stream_event
events should stream as soon as the raw_items are completed and not wait for tool execution. This is important in applications where some logic is needed to be performed based on the events, where this ordering can caused issuesAnother case where this will cause issues is where the LLM output is provided to the user through run_items and tool calls are involved. In such cases the user will receive any output only after the tool execution is completed
The text was updated successfully, but these errors were encountered: