Skip to content

Benchmark Function #31356

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
mfeurer opened this issue May 12, 2025 · 1 comment
Closed

Benchmark Function #31356

mfeurer opened this issue May 12, 2025 · 1 comment
Labels
Needs Triage Issue requires triage New Feature

Comments

@mfeurer
Copy link
Contributor

mfeurer commented May 12, 2025

Describe the workflow you want to enable

I would like to define multiple pipelines and compare them against each other on multiple datasets.

Describe your proposed solution

A single helper function that executes this benchmark fully in parallel. This would allow

Describe alternatives you've considered, if relevant

There is an MLR3 function that inspired this issue.

Additional context

Reasoning: I'm currently co-teaching a course where students can do the exercises in R using MLR3 or Python using scikit-learn. Doing the exercises in R appears to be less repetitive overall, as for example, there is a simple function for benchmarking. Also, it would require less time to actually wait for the results to finish as one could make more use of parallelism.

@mfeurer mfeurer added New Feature Needs Triage Issue requires triage labels May 12, 2025
@adrinjalali
Copy link
Member

You can already simply do that by creating your Pipeline objects, and passing them to a cross_val_score in a look over your datasets and pipelines. Since the code snippet required to do this is just a few lines, I don't think it makes sense for us to add it to the public API.

@adrinjalali adrinjalali closed this as not planned Won't fix, can't repro, duplicate, stale May 12, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Needs Triage Issue requires triage New Feature
Projects
None yet
Development

No branches or pull requests

2 participants