Skip to content

Metric.ndcg score #24482

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 8 commits into from
Closed

Metric.ndcg score #24482

wants to merge 8 commits into from

Conversation

mae5357
Copy link

@mae5357 mae5357 commented Sep 20, 2022

Reference Issues/PRs

Fixes #21335 and #20119

What does this implement/fix? Explain your changes.

Computing Normalized Discounted Cumulative Gain (NDCG) does not make sense for single predictions. Throw an error if y_true is a list of length 1 for NDCG and DCG.

Any other comments?

Test that this throws the appropriate error by running:

from sklearn.metrics import ndcg_score

y_true = [[1]]
y_pred = [[1]]

print(ndcg_score(y_true, y_pred))

@Micky774
Copy link
Contributor

Hey there @mae5357, thank you for the contribution! Could you please:

  1. Add some tests to confirm that the new error is properly raised?
  2. Add a changelog entry describing the addition of the new error?

Regarding the code, I wonder if it would make sense just to include the check in _check_dcg_traget_type that way we don't need to introduce a new private function that could otherwise be easily inline.

@haiatn
Copy link
Contributor

haiatn commented Oct 4, 2022

I see a failed check on test coverege

@Kshitij68
Copy link
Contributor

Hi @mae5357 Do you plan to continue working on this? If not, I'd like to continue.

@glemaitre glemaitre mentioned this pull request Nov 2, 2022
Closed
@glemaitre glemaitre mentioned this pull request Feb 24, 2023
Merged
@jeremiedbb jeremiedbb added the Superseded PR has been replace by a newer PR label Mar 2, 2023
@jeremiedbb
Copy link
Member

Closing in favor of #25672

@jeremiedbb jeremiedbb closed this Mar 2, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
module:metrics Superseded PR has been replace by a newer PR
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

NDCG score doesn't work with binary relevance and a list of 1 element
5 participants
@mae5357 @Micky774 @haiatn @Kshitij68 @jeremiedbb