Skip to content

Inconsistent median/quantile behaviour now _weighted_percentile ignores NaNs #31367

@lucyleeow

Description

@lucyleeow

As of #29034, _weighted_percentile handles NaNs by ignoring them when calculating percentile.
np.median and np.percentile on the other hand, will return NaN if a NaN is present in the input (np.nanmedian and np.nanpercentile will ignore nans).

There are many cases in the codebase where, if sample_weight is None, a np function is used (NaN returned), if sample_weight is given, _weighted_percentile used and NaNs ignored.

Summary of affected cases:

Maybe we could assess on a case by case basis whether it makes sense to return NaN if present in the input? @ogrisel suggested that we may want to raise a warning in some cases as well.

cc @StefanieSenger

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions