Skip to content

FIX Support readonly sparse datasets for manhattan_distances #25432

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
Jan 26, 2023
Merged

FIX Support readonly sparse datasets for manhattan_distances #25432

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
15f39d1
TST Add non-regression test for #7981
jjerphan Jan 19, 2023
bf2b5b4
FIX Support readonly sparse datasets for manhattan
jjerphan Jan 19, 2023
61fa92c
Merge branch 'main' into fix/support-for-readonly-sparse-datasets-for…
jjerphan Jan 24, 2023
f8c3885
DOC Add entry in whats_new/v1.2.rst for 1.2.1
jjerphan Jan 24, 2023
286ee5b
FIX Fix comment
jjerphan Jan 24, 2023
4f05003
Update sklearn/metrics/tests/test_pairwise.py
jjerphan Jan 24, 2023
80f4af3
DOC Move entry to whats_new/v1.3.rst
jjerphan Jan 24, 2023
9181608
Update sklearn/metrics/tests/test_pairwise.py
jjerphan Jan 24, 2023
ea38633
Merge branch 'main' into fix/support-for-readonly-sparse-datasets-for…
jjerphan Jan 26, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions doc/whats_new/v1.3.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -143,6 +143,12 @@ Changelog
- |Enhancement| Added the parameter `fill_value` to :class:`impute.IterativeImputer`.
:pr:`25232` by :user:`Thijs van Weezel <ValueInvestorThijs>`.

:mod:`sklearn.metrics`
......................

- |Fix| :func:`metric.manhattan_distances` now supports readonly sparse datasets.
:pr:`25432` by :user:`Julien Jerphanion <jjerphan>`.

:mod:`sklearn.naive_bayes`
..........................

Expand Down
12 changes: 9 additions & 3 deletions sklearn/metrics/_pairwise_fast.pyx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,9 +35,15 @@ def _chi2_kernel_fast(floating[:, :] X,
result[i, j] = -res


def _sparse_manhattan(floating[::1] X_data, int[:] X_indices, int[:] X_indptr,
floating[::1] Y_data, int[:] Y_indices, int[:] Y_indptr,
double[:, ::1] D):
def _sparse_manhattan(
const floating[::1] X_data,
const int[:] X_indices,
const int[:] X_indptr,
const floating[::1] Y_data,
const int[:] Y_indices,
const int[:] Y_indptr,
double[:, ::1] D,
):
"""Pairwise L1 distances for CSR matrices.

Usage:
Expand Down
12 changes: 12 additions & 0 deletions sklearn/metrics/tests/test_pairwise.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@
from scipy.spatial.distance import minkowski as wminkowski

from sklearn.utils.fixes import sp_version, parse_version
from sklearn.utils.parallel import delayed, Parallel

import pytest

Expand Down Expand Up @@ -1541,3 +1542,14 @@ def test_numeric_pairwise_distances_datatypes(metric, global_dtype, y_is_x):
dist = pairwise_distances(X, Y, metric=metric, **params)

assert_allclose(dist, expected_dist)


def test_sparse_manhattan_readonly_dataset():
# Non-regression test for: https://github.com/scikit-learn/scikit-learn/issues/7981
matrices1 = [csr_matrix(np.ones((5, 5)))]
matrices2 = [csr_matrix(np.ones((5, 5)))]
# Joblib memory maps datasets which makes them read-only.
# The following call was reporting as failing in #7981, but this must pass.
Parallel(n_jobs=2, max_nbytes=0)(
delayed(manhattan_distances)(m1, m2) for m1, m2 in zip(matrices1, matrices2)
)