Skip to content

Why does the Computer protocol not have the goto method? #547

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wjayesh opened this issue Apr 18, 2025 · 2 comments
Open

Why does the Computer protocol not have the goto method? #547

wjayesh opened this issue Apr 18, 2025 · 2 comments
Labels
enhancement New feature or request

Comments

@wjayesh
Copy link

wjayesh commented Apr 18, 2025

Describe the feature

Right now, the computers you create can't define (or use) the goto method to go to a specific URL. This means that if you are at a page that is not a search engine, it is hard to go to a completely different page, unless the link to the said page is on the current page.

The BasePlaywrightComputer class still defines the goto method but since this is not part of the Computer protocol, the agent doesn't use it. Am I missing something or this protocol missing an important function?
https://github.com/openai/openai-cua-sample-app/blob/eb2d58ba77ffd3206d3346d6357093647d29d99c/computers/base_playwright.py#L136

@wjayesh wjayesh added the enhancement New feature or request label Apr 18, 2025
@rm-openai
Copy link
Collaborator

It's not part of the base CUA protocol: https://platform.openai.com/docs/api-reference/responses/object#responses/object-output-computer-tool-call-action

That said, we should indeed make the Computer extensible. Rather than adding goto, we should allow arbitrary additional actions to be exposed by the Computer.

PR welcome, or else I can get to it at some point!

@itsaphel
Copy link

I'd also appreciate this! The openai-cua-sample-app has goto as a custom action, so it feels relatively standard (if not standardised!) to have?

https://github.com/openai/openai-cua-sample-app/blob/a842bece4e255ec4d9d248e01a1a1d33c9c6ce5b/examples/playwright_with_custom_functions.py

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants