Skip to content

Conversation

ghost
Copy link

@ghost ghost commented Nov 12, 2021

#21598 @adrinjalali

Changed the number of features and permutations
permut_before
permut_after

@TomDLT
Copy link
Member

TomDLT commented Nov 12, 2021

Thanks for the PR.
I would just change the number of uncorrelated features to e.g. 20, because there is little need to use so many in this example.
However, I would keep the number of permutations to 1000 to have a reasonable p-value estimate.

@ghost
Copy link
Author

ghost commented Nov 13, 2021

@TomDLT Allright, agreed

@ghost
Copy link
Author

ghost commented Nov 13, 2021

Even better time improvement from 1:44 min (before) to 0:11 min (after)

@adrinjalali adrinjalali changed the title Changed the number of features and permutations in real and dummy sub-examples for increased example execution speed Changed the number of features and permutations in real and dummy sub-examples for increased example execution speed plot_permutation_tests_for_classification.py Nov 13, 2021
@adrinjalali adrinjalali mentioned this pull request Nov 13, 2021
41 tasks
@adrinjalali
Copy link
Member

Could you also check if using another model other than SVC could speed up the process?

@adrinjalali
Copy link
Member

@thomasjpfan re number of unrelated features, you happy with this?

@ghost
Copy link
Author

ghost commented Nov 13, 2021

@adrinjalali Yes, I also checked One-vs-One Logistic Regression, One-vs-All Logistic Regression and Multinomial Logistic Regression...all three were significantly slower than SVC.

@adrinjalali
Copy link
Member

I'd also check HistGradientBoosting*

@ghost
Copy link
Author

ghost commented Nov 15, 2021

@adrinjalali Just checked it, also way worse (12 sec for LogReg vs 10 min HGB)...so far Logistic Regression seems unreached by other models

@TomDLT TomDLT changed the title Changed the number of features and permutations in real and dummy sub-examples for increased example execution speed plot_permutation_tests_for_classification.py Improve speed plot_permutation_tests_for_classification.py Nov 16, 2021
@TomDLT TomDLT merged commit 263181a into scikit-learn:main Nov 16, 2021
@ghost ghost deleted the speed_increased_example_permutation branch November 16, 2021 17:32
glemaitre pushed a commit to glemaitre/scikit-learn that referenced this pull request Nov 22, 2021
glemaitre pushed a commit to glemaitre/scikit-learn that referenced this pull request Nov 29, 2021
samronsin pushed a commit to samronsin/scikit-learn that referenced this pull request Nov 30, 2021
glemaitre pushed a commit to glemaitre/scikit-learn that referenced this pull request Dec 24, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants