-
Notifications
You must be signed in to change notification settings - Fork 286
Description
Please read this first
- Have you read the docs? Agents SDK docs YES
- Have you searched for related issues? Others may have faced similar issues. YES
Describe the bug
Critical Bug: Session configuration breaks tool registration in RealtimeSession
When initializing a RealtimeSession with ANY config object (even without tools property), the session's tool registration becomes broken. Tools defined on the RealtimeAgent stop working, and the agent cannot call any tools.
This is a fundamental design flaw where session configuration and tool functionality are mutually exclusive, making it impossible to customize audio settings (voice, turn detection, noise reduction, etc.) while maintaining tool functionality.
The bug is in the RealtimeSession.#getSessionConfig() method. When session config is provided:
// In realtimeSession.js line ~147
const base = {
...(this.#lastSessionConfig ?? {}),
...(this.options.config ?? {}), // ← Session config gets spread here
...(additionalConfig ?? {}),
};
const fullConfig = {
...base, // ← Session config properties are now in base
instructions,
voice: this.#currentAgent.voice,
model: this.options.model,
tools: this.#currentTools, // ← Tools get added AFTER session config
tracing: tracingConfig,
};
The problem: Even though tools are added after the session config spread, the session config somehow interferes with the tool registration process. This suggests the library has internal logic that processes session config in a way that breaks tool discovery.
Impact:
- Critical functionality loss: Users cannot use tools when customizing audio
- Workaround required: Must choose between tools OR audio customization
- Poor developer experience: Configuration that should work together doesn't
- Documentation gap: No warning that session config breaks tools
The library should either:
- Fix the tool registration conflict when session config is present
- Provide alternative audio configuration methods (e.g., on the agent or transport level)
- Add clear documentation about this limitation
- Separate audio config from session config to prevent conflicts
Debug information
Agents SDK version: @openai/agents-realtime@0.0.16
Runtime environment: Node.js (browser environment with WebRTC transport)
Repro steps
import { RealtimeAgent, RealtimeSession } from '@openai/agents-realtime';
import { tool } from '@openai/agents-realtime';
import { z } from 'zod';
// 1. Create a simple tool
const testTool = tool({
name: 'test_tool',
description: 'A test tool',
parameters: z.object({
input: z.string()
}),
execute: async ({ input }) => {
console.log('Tool called with:', input);
return { success: true, result: input };
}
});
// 2. Create agent with tools
const agent = new RealtimeAgent({
name: 'Test Agent',
tools: [testTool]
});
// 3. Create session WITHOUT config - tools work ✅
const workingSession = new RealtimeSession(agent);
// Tools will be available and functional
// 4. Create session WITH config - tools break ❌
const brokenSession = new RealtimeSession(agent, {
config: {
voice: 'alloy', // Even just voice setting breaks tools
// No tools property, but tools still don't work
}
});
// Tools become unavailable despite being defined on agent
Expected behavior
- Tools should work regardless of session config: Session configuration should only affect audio/behavior settings, not interfere with tool registration
- Audio customization should be possible: Users should be able to customize voice, turn detection, noise reduction, etc. without losing tool functionality
- Separation of concerns: Tool registration (agent-level) and session configuration (audio/behavior) should be independent