-
-
Notifications
You must be signed in to change notification settings - Fork 25.8k
Implementing Temperature Scaling #29517
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
in _weight_boosting.py.
`ensemble/_weight_boosting.py` file, moving it below the `Examples` section for improved organization. Included an AdaBoost example reference within the DecisionTree class in the `tree/-class.py` file.
…sion Trees user guide. - Modified the doc-string wording in the `AdaBoostClassifier` class referencing to the aforementioned example.
…related to temperature scaling.
- Modified the `negative_log_likelihood` function to allow labels to be one-hot.
- Modified the `negative_log_likelihood` function to allow labels to be one-hot. - Added the `_temperature_scaling_test.py` file.
…e outputs from `decision_function` function. Also added the `_additive_smoothing` function to avoid numerical instability when applying logarithm.
…argument. 2. Modified the `_temperature_scaling` function. The initial temperature is now 1.0, and the optimised temperature is in interval [1e-2, inf). 3. Revise doc-strings.
…py` for the first PR draft.
❌ Linting issuesThis PR is introducing linting issues. Here's a summary of the issues. Note that you can avoid having linting issues by enabling You can see the details of the linting issues under the
|
Thanks for the PR @virchan. I see you're new to this repo. This PR as is right now, is quite far from something we'd merge. I suggest you continue with a few easier issues in the meantime, get more familiar with the codebase, and then try tackling more challenging issues. So I'm closing this PR for now. |
Thank you for your time! |
Reference Issues/PRs
Towards #28574
What does this implement/fix? Explain your changes.
Temperature scaling is a probability calibration method for multi-class classification. It is given by the formula:
where$\ell_i$ s' are logits from a multi-class classifier, and $T$ is the temperature parameter trained to calibrate the softmax function.
This PR implements the temperature scaling method under the
CalibratedClassifierCV
class by adding a new optionmethod='temperature'
when instantiating theCalibratedClassifierCV
object:When the
.fit()
method is called, a temperature parameter is trained. The.predict_proba()
method then computes multi-class probabilities using the above formula. The temperature parameter is initially set to1.0
([Guo et al. 2017, Section 5]), and the optimised temperature takes a value in the interval[1e-2, inf)
for numerical stability.This PR includes two files:
sklearn/calibration_temperature.py
, intended to replacesklearn/calibration.py
in the final merge.sklearn/_temperature_scaling_test.py
, providing demonstrations of this new feature, which will not be included in the final merge.References