Skip to content

ValueError buffer source array is read-only when computing manahattan_distances of a CSR matrix inside Parallel #7981

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
lesteve opened this issue Dec 5, 2016 · 9 comments · Fixed by #25432
Closed

ValueError buffer source array is read-only when computing manahattan_distances of a CSR matrix inside Parallel #7981

lesteve opened this issue Dec 5, 2016 · 9 comments · Fixed by #25432
Labels
Bug help wanted module:metrics

Comments

@lesteve
Copy link
Member

lesteve commented Dec 5, 2016 •
edited by adrinjalali
Loading

From #5481 (comment).

Description

ValueError buffer source array is read-only when computing manahattan_distances of a CSR matrix inside Parallel

Steps/Code to Reproduce

import numpy as np

from scipy.sparse import csr_matrix

from sklearn.externals.joblib import Parallel, delayed
from sklearn.metrics.pairwise import manhattan_distances

matrices1 = [csr_matrix(np.ones((5, 5)))]
matrices2 = [csr_matrix(np.ones((5, 5)))]

Parallel(n_jobs=-1, max_nbytes=0)(
    delayed(manhattan_distances)(m1, m2)
    for m1, m2 in zip(matrices1, matrices2))

Full traceback:

---------------------------------------------------------------------------
RemoteTraceback                           Traceback (most recent call last)
RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/home/le243287/dev/scikit-learn/sklearn/externals/joblib/_parallel_backends.py", line 344, in __call__
    return self.func(*args, **kwargs)
  File "/home/le243287/dev/scikit-learn/sklearn/externals/joblib/parallel.py", line 131, in __call__
    return [func(*args, **kwargs) for func, args, kwargs in self.items]
  File "/home/le243287/dev/scikit-learn/sklearn/externals/joblib/parallel.py", line 131, in <listcomp>
    return [func(*args, **kwargs) for func, args, kwargs in self.items]
  File "/home/le243287/dev/scikit-learn/sklearn/metrics/pairwise.py", line 533, in manhattan_distances
    X.shape[1], D)
  File "sklearn/metrics/pairwise_fast.pyx", line 53, in sklearn.metrics.pairwise_fast._sparse_manhattan (sklearn/metrics/pairwise_fast.c:4688)
    def _sparse_manhattan(floating1d X_data, int[:] X_indices, int[:] X_indptr,
  File "stringsource", line 644, in View.MemoryView.memoryview_cwrapper (sklearn/metrics/pairwise_fast.c:13277)
  File "stringsource", line 345, in View.MemoryView.memoryview.__cinit__ (sklearn/metrics/pairwise_fast.c:9512)
ValueError: buffer source array is read-only

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/volatile/le243287/miniconda3/lib/python3.5/multiprocessing/pool.py", line 119, in worker
    result = (True, func(*args, **kwds))
  File "/home/le243287/dev/scikit-learn/sklearn/externals/joblib/_parallel_backends.py", line 353, in __call__
    raise TransportableException(text, e_type)
sklearn.externals.joblib.my_exceptions.TransportableException: TransportableException
___________________________________________________________________________
ValueError                                         Mon Dec  5 13:19:44 2016
PID: 25869           Python 3.5.2: /volatile/le243287/miniconda3/bin/python
...........................................................................
/home/le243287/dev/scikit-learn/sklearn/externals/joblib/parallel.py in __call__(self=<sklearn.externals.joblib.parallel.BatchedCalls object>)
    126     def __init__(self, iterator_slice):
    127         self.items = list(iterator_slice)
    128         self._size = len(self.items)
    129 
    130     def __call__(self):
--> 131         return [func(*args, **kwargs) for func, args, kwargs in self.items]
        self.items = [(<function manhattan_distances>, (<5x5 sparse matrix of type '<class 'numpy.float6... stored elements in Compressed Sparse Row format>, <5x5 sparse matrix of type '<class 'numpy.float6... stored elements in Compressed Sparse Row format>), {})]
    132 
    133     def __len__(self):
    134         return self._size
    135 

...........................................................................
/home/le243287/dev/scikit-learn/sklearn/externals/joblib/parallel.py in <listcomp>(.0=<list_iterator object>)
    126     def __init__(self, iterator_slice):
    127         self.items = list(iterator_slice)
    128         self._size = len(self.items)
    129 
    130     def __call__(self):
--> 131         return [func(*args, **kwargs) for func, args, kwargs in self.items]
        func = <function manhattan_distances>
        args = (<5x5 sparse matrix of type '<class 'numpy.float6... stored elements in Compressed Sparse Row format>, <5x5 sparse matrix of type '<class 'numpy.float6... stored elements in Compressed Sparse Row format>)
        kwargs = {}
    132 
    133     def __len__(self):
    134         return self._size
    135 

...........................................................................
/home/le243287/dev/scikit-learn/sklearn/metrics/pairwise.py in manhattan_distances(X=<5x5 sparse matrix of type '<class 'numpy.float6... stored elements in Compressed Sparse Row format>, Y=<5x5 sparse matrix of type '<class 'numpy.float6... stored elements in Compressed Sparse Row format>, sum_over_features=True, size_threshold=500000000.0)
    528         X = csr_matrix(X, copy=False)
    529         Y = csr_matrix(Y, copy=False)
    530         D = np.zeros((X.shape[0], Y.shape[0]))
    531         _sparse_manhattan(X.data, X.indices, X.indptr,
    532                           Y.data, Y.indices, Y.indptr,
--> 533                           X.shape[1], D)
        X.shape = (5, 5)
        D = array([[ 0.,  0.,  0.,  0.,  0.],
       [ 0.,  ...0.,  0.,  0.],
       [ 0.,  0.,  0.,  0.,  0.]])
    534         return D
    535 
    536     if sum_over_features:
    537         return distance.cdist(X, Y, 'cityblock')

...........................................................................
/home/le243287/dev/scikit-learn/sklearn/metrics/pairwise_fast.cpython-35m-x86_64-linux-gnu.so in sklearn.metrics.pairwise_fast._sparse_manhattan (sklearn/metrics/pairwise_fast.c:4688)()
     48                     if nom != 0:
     49                         res  += denom * denom / nom
     50                 result[i, j] = -res
     51 
     52 
---> 53 def _sparse_manhattan(floating1d X_data, int[:] X_indices, int[:] X_indptr,
     54                       floating1d Y_data, int[:] Y_indices, int[:] Y_indptr,
     55                       np.npy_intp n_features, double[:, ::1] D):
     56     """Pairwise L1 distances for CSR matrices.
     57 

...........................................................................
/home/le243287/dev/scikit-learn/sklearn/metrics/pairwise_fast.cpython-35m-x86_64-linux-gnu.so in View.MemoryView.memoryview_cwrapper (sklearn/metrics/pairwise_fast.c:13277)()
    639 
    640 
    641 
    642 
    643 
--> 644 
    645 
    646 
    647 
    648 

...........................................................................
/home/le243287/dev/scikit-learn/sklearn/metrics/pairwise_fast.cpython-35m-x86_64-linux-gnu.so in View.MemoryView.memoryview.__cinit__ (sklearn/metrics/pairwise_fast.c:9512)()
    340 
    341 
    342 
    343 
    344 
--> 345 
    346 
    347 
    348 
    349 

ValueError: buffer source array is read-only
___________________________________________________________________________
"""

The above exception was the direct cause of the following exception:

TransportableException                    Traceback (most recent call last)
/home/le243287/dev/scikit-learn/sklearn/externals/joblib/parallel.py in retrieve(self)
    681                 if 'timeout' in getfullargspec(job.get).args:
--> 682                     self._output.extend(job.get(timeout=self.timeout))
    683                 else:

/volatile/le243287/miniconda3/lib/python3.5/multiprocessing/pool.py in get(self, timeout)
    607         else:
--> 608             raise self._value
    609 

TransportableException: TransportableException
___________________________________________________________________________
ValueError                                         Mon Dec  5 13:19:44 2016
PID: 25869           Python 3.5.2: /volatile/le243287/miniconda3/bin/python
...........................................................................
/home/le243287/dev/scikit-learn/sklearn/externals/joblib/parallel.py in __call__(self=<sklearn.externals.joblib.parallel.BatchedCalls object>)
    126     def __init__(self, iterator_slice):
    127         self.items = list(iterator_slice)
    128         self._size = len(self.items)
    129 
    130     def __call__(self):
--> 131         return [func(*args, **kwargs) for func, args, kwargs in self.items]
        self.items = [(<function manhattan_distances>, (<5x5 sparse matrix of type '<class 'numpy.float6... stored elements in Compressed Sparse Row format>, <5x5 sparse matrix of type '<class 'numpy.float6... stored elements in Compressed Sparse Row format>), {})]
    132 
    133     def __len__(self):
    134         return self._size
    135 

...........................................................................
/home/le243287/dev/scikit-learn/sklearn/externals/joblib/parallel.py in <listcomp>(.0=<list_iterator object>)
    126     def __init__(self, iterator_slice):
    127         self.items = list(iterator_slice)
    128         self._size = len(self.items)
    129 
    130     def __call__(self):
--> 131         return [func(*args, **kwargs) for func, args, kwargs in self.items]
        func = <function manhattan_distances>
        args = (<5x5 sparse matrix of type '<class 'numpy.float6... stored elements in Compressed Sparse Row format>, <5x5 sparse matrix of type '<class 'numpy.float6... stored elements in Compressed Sparse Row format>)
        kwargs = {}
    132 
    133     def __len__(self):
    134         return self._size
    135 

...........................................................................
/home/le243287/dev/scikit-learn/sklearn/metrics/pairwise.py in manhattan_distances(X=<5x5 sparse matrix of type '<class 'numpy.float6... stored elements in Compressed Sparse Row format>, Y=<5x5 sparse matrix of type '<class 'numpy.float6... stored elements in Compressed Sparse Row format>, sum_over_features=True, size_threshold=500000000.0)
    528         X = csr_matrix(X, copy=False)
    529         Y = csr_matrix(Y, copy=False)
    530         D = np.zeros((X.shape[0], Y.shape[0]))
    531         _sparse_manhattan(X.data, X.indices, X.indptr,
    532                           Y.data, Y.indices, Y.indptr,
--> 533                           X.shape[1], D)
        X.shape = (5, 5)
        D = array([[ 0.,  0.,  0.,  0.,  0.],
       [ 0.,  ...0.,  0.,  0.],
       [ 0.,  0.,  0.,  0.,  0.]])
    534         return D
    535 
    536     if sum_over_features:
    537         return distance.cdist(X, Y, 'cityblock')

...........................................................................
/home/le243287/dev/scikit-learn/sklearn/metrics/pairwise_fast.cpython-35m-x86_64-linux-gnu.so in sklearn.metrics.pairwise_fast._sparse_manhattan (sklearn/metrics/pairwise_fast.c:4688)()
     48                     if nom != 0:
     49                         res  += denom * denom / nom
     50                 result[i, j] = -res
     51 
     52 
---> 53 def _sparse_manhattan(floating1d X_data, int[:] X_indices, int[:] X_indptr,
     54                       floating1d Y_data, int[:] Y_indices, int[:] Y_indptr,
     55                       np.npy_intp n_features, double[:, ::1] D):
     56     """Pairwise L1 distances for CSR matrices.
     57 

...........................................................................
/home/le243287/dev/scikit-learn/sklearn/metrics/pairwise_fast.cpython-35m-x86_64-linux-gnu.so in View.MemoryView.memoryview_cwrapper (sklearn/metrics/pairwise_fast.c:13277)()
    639 
    640 
    641 
    642 
    643 
--> 644 
    645 
    646 
    647 
    648 

...........................................................................
/home/le243287/dev/scikit-learn/sklearn/metrics/pairwise_fast.cpython-35m-x86_64-linux-gnu.so in View.MemoryView.memoryview.__cinit__ (sklearn/metrics/pairwise_fast.c:9512)()
    340 
    341 
    342 
    343 
    344 
--> 345 
    346 
    347 
    348 
    349 

ValueError: buffer source array is read-only
___________________________________________________________________________

During handling of the above exception, another exception occurred:

JoblibValueError                          Traceback (most recent call last)
/tmp/test_manhattan.py in <module>()
     11 Parallel(n_jobs=2, max_nbytes=0)(
     12     delayed(manhattan_distances)(m1, m2)
---> 13     for m1, m2 in zip(matrices1, matrices2))

/home/le243287/dev/scikit-learn/sklearn/externals/joblib/parallel.py in __call__(self, iterable)
    766                 # consumption.
    767                 self._iterating = False
--> 768             self.retrieve()
    769             # Make sure that we get a last message telling us we are done
    770             elapsed_time = time.time() - self._start_time

/home/le243287/dev/scikit-learn/sklearn/externals/joblib/parallel.py in retrieve(self)
    717                     ensure_ready = self._managed_backend
    718                     backend.abort_everything(ensure_ready=ensure_ready)
--> 719                 raise exception
    720 
    721     def __call__(self, iterable):

JoblibValueError: JoblibValueError
___________________________________________________________________________
Multiprocessing exception:
...........................................................................
/volatile/le243287/miniconda3/bin/ipython in <module>()
      1 #!/volatile/le243287/miniconda3/bin/python
      2 if __name__ == '__main__':
      3     import sys
      4     import IPython
      5 
----> 6     sys.exit(IPython.start_ipython())
      7 
      8 
      9 
     10 

...........................................................................
/volatile/le243287/miniconda3/lib/python3.5/site-packages/IPython/__init__.py in start_ipython(argv=None, **kwargs={})
    114     kwargs : various, optional
    115         Any other kwargs will be passed to the Application constructor,
    116         such as `config`.
    117     """
    118     from IPython.terminal.ipapp import launch_new_instance
--> 119     return launch_new_instance(argv=argv, **kwargs)
        launch_new_instance = <bound method Application.launch_instance of <class 'IPython.terminal.ipapp.TerminalIPythonApp'>>
        argv = None
        kwargs = {}
    120 
    121 def start_kernel(argv=None, **kwargs):
    122     """Launch a normal IPython kernel instance (as opposed to embedded)
    123     

...........................................................................
/volatile/le243287/miniconda3/lib/python3.5/site-packages/traitlets/config/application.py in launch_instance(cls=<class 'IPython.terminal.ipapp.TerminalIPythonApp'>, argv=None, **kwargs={})
    652         """Launch a global instance of this Application
    653 
    654         If a global instance already exists, this reinitializes and starts it
    655         """
    656         app = cls.instance(**kwargs)
--> 657         app.initialize(argv)
        app.initialize = <bound method TerminalIPythonApp.initialize of <IPython.terminal.ipapp.TerminalIPythonApp object>>
        argv = None
    658         app.start()
    659 
    660 #-----------------------------------------------------------------------------
    661 # utility functions, for convenience

...........................................................................
/home/le243287/dev/scikit-learn/<decorator-gen-109> in initialize(self=<IPython.terminal.ipapp.TerminalIPythonApp object>, argv=None)
      1 
----> 2 
      3 
      4 
      5 
      6 
      7 
      8 
      9 
     10 

...........................................................................
/volatile/le243287/miniconda3/lib/python3.5/site-packages/traitlets/config/application.py in catch_config_error(method=<function TerminalIPythonApp.initialize>, app=<IPython.terminal.ipapp.TerminalIPythonApp object>, *args=(None,), **kwargs={})
     82     message, and exit the app.
     83 
     84     For use on init methods, to prevent invoking excepthook on invalid input.
     85     """
     86     try:
---> 87         return method(app, *args, **kwargs)
        method = <function TerminalIPythonApp.initialize>
        app = <IPython.terminal.ipapp.TerminalIPythonApp object>
        args = (None,)
        kwargs = {}
     88     except (TraitError, ArgumentError) as e:
     89         app.print_help()
     90         app.log.fatal("Bad config encountered during initialization:")
     91         app.log.fatal(str(e))

...........................................................................
/volatile/le243287/miniconda3/lib/python3.5/site-packages/IPython/terminal/ipapp.py in initialize(self=<IPython.terminal.ipapp.TerminalIPythonApp object>, argv=None)
    310         # and draw the banner
    311         self.init_banner()
    312         # Now a variety of things that happen after the banner is printed.
    313         self.init_gui_pylab()
    314         self.init_extensions()
--> 315         self.init_code()
        self.init_code = <bound method InteractiveShellApp.init_code of <IPython.terminal.ipapp.TerminalIPythonApp object>>
    316 
    317     def init_shell(self):
    318         """initialize the InteractiveShell instance"""
    319         # Create an InteractiveShell instance.

...........................................................................
/volatile/le243287/miniconda3/lib/python3.5/site-packages/IPython/core/shellapp.py in init_code(self=<IPython.terminal.ipapp.TerminalIPythonApp object>)
    268         if self.hide_initial_ns:
    269             self.shell.user_ns_hidden.update(self.shell.user_ns)
    270 
    271         # command-line execution (ipython -i script.py, ipython -m module)
    272         # should *not* be excluded from %whos
--> 273         self._run_cmd_line_code()
        self._run_cmd_line_code = <bound method InteractiveShellApp._run_cmd_line_...Python.terminal.ipapp.TerminalIPythonApp object>>
    274         self._run_module()
    275 
    276         # flush output, so itwon't be attached to the first cell
    277         sys.stdout.flush()

...........................................................................
/volatile/le243287/miniconda3/lib/python3.5/site-packages/IPython/core/shellapp.py in _run_cmd_line_code(self=<IPython.terminal.ipapp.TerminalIPythonApp object>)
    389         elif self.file_to_run:
    390             fname = self.file_to_run
    391             if os.path.isdir(fname):
    392                 fname = os.path.join(fname, "__main__.py")
    393             try:
--> 394                 self._exec_file(fname, shell_futures=True)
        self._exec_file = <bound method InteractiveShellApp._exec_file of <IPython.terminal.ipapp.TerminalIPythonApp object>>
        fname = '/tmp/test_manhattan.py'
    395             except:
    396                 self.shell.showtraceback(tb_offset=4)
    397                 if not self.interact:
    398                     self.exit(1)

...........................................................................
/volatile/le243287/miniconda3/lib/python3.5/site-packages/IPython/core/shellapp.py in _exec_file(self=<IPython.terminal.ipapp.TerminalIPythonApp object>, fname='/tmp/test_manhattan.py', shell_futures=True)
    323                     else:
    324                         # default to python, even without extension
    325                         self.shell.safe_execfile(full_filename,
    326                                                  self.shell.user_ns,
    327                                                  shell_futures=shell_futures,
--> 328                                                  raise_exceptions=True)
    329         finally:
    330             sys.argv = save_argv
    331 
    332     def _run_startup_files(self):

...........................................................................
/volatile/le243287/miniconda3/lib/python3.5/site-packages/IPython/core/interactiveshell.py in safe_execfile(self=<IPython.terminal.interactiveshell.TerminalInteractiveShell object>, fname='/tmp/test_manhattan.py', *where=({'In': [''], 'Out': {}, 'Parallel': <class 'sklearn.externals.joblib.parallel.Parallel'>, '_': '', '__': '', '___': '', '__builtin__': <module 'builtins' (built-in)>, '__builtins__': <module 'builtins' (built-in)>, '__doc__': 'Automatically created module for IPython interactive environment', '__file__': '/tmp/test_manhattan.py', ...},), **kw={'exit_ignore': False, 'raise_exceptions': True, 'shell_futures': True})
   2476         with prepended_to_syspath(dname), self.builtin_trap:
   2477             try:
   2478                 glob, loc = (where + (None, ))[:2]
   2479                 py3compat.execfile(
   2480                     fname, glob, loc,
-> 2481                     self.compile if kw['shell_futures'] else None)
        self.compile = <IPython.core.compilerop.CachingCompiler object>
        kw = {'exit_ignore': False, 'raise_exceptions': True, 'shell_futures': True}
   2482             except SystemExit as status:
   2483                 # If the call was made with 0 or None exit status (sys.exit(0)
   2484                 # or sys.exit() ), don't bother showing a traceback, as both of
   2485                 # these are considered normal by the OS:

...........................................................................
/volatile/le243287/miniconda3/lib/python3.5/site-packages/IPython/utils/py3compat.py in execfile(fname='/tmp/test_manhattan.py', glob={'In': [''], 'Out': {}, 'Parallel': <class 'sklearn.externals.joblib.parallel.Parallel'>, '_': '', '__': '', '___': '', '__builtin__': <module 'builtins' (built-in)>, '__builtins__': <module 'builtins' (built-in)>, '__doc__': 'Automatically created module for IPython interactive environment', '__file__': '/tmp/test_manhattan.py', ...}, loc={'In': [''], 'Out': {}, 'Parallel': <class 'sklearn.externals.joblib.parallel.Parallel'>, '_': '', '__': '', '___': '', '__builtin__': <module 'builtins' (built-in)>, '__builtins__': <module 'builtins' (built-in)>, '__doc__': 'Automatically created module for IPython interactive environment', '__file__': '/tmp/test_manhattan.py', ...}, compiler=<IPython.core.compilerop.CachingCompiler object>)
    181 
    182     def execfile(fname, glob, loc=None, compiler=None):
    183         loc = loc if (loc is not None) else glob
    184         with open(fname, 'rb') as f:
    185             compiler = compiler or compile
--> 186             exec(compiler(f.read(), fname, 'exec'), glob, loc)
        compiler = <IPython.core.compilerop.CachingCompiler object>
        f.read = <built-in method read of _io.BufferedReader object>
        fname = '/tmp/test_manhattan.py'
        glob = {'In': [''], 'Out': {}, 'Parallel': <class 'sklearn.externals.joblib.parallel.Parallel'>, '_': '', '__': '', '___': '', '__builtin__': <module 'builtins' (built-in)>, '__builtins__': <module 'builtins' (built-in)>, '__doc__': 'Automatically created module for IPython interactive environment', '__file__': '/tmp/test_manhattan.py', ...}
        loc = {'In': [''], 'Out': {}, 'Parallel': <class 'sklearn.externals.joblib.parallel.Parallel'>, '_': '', '__': '', '___': '', '__builtin__': <module 'builtins' (built-in)>, '__builtins__': <module 'builtins' (built-in)>, '__doc__': 'Automatically created module for IPython interactive environment', '__file__': '/tmp/test_manhattan.py', ...}
    187     
    188     # Refactor print statements in doctests.
    189     _print_statement_re = re.compile(r"\bprint (?P<expr>.*)$", re.MULTILINE)
    190     def _print_statement_sub(match):

...........................................................................
/tmp/test_manhattan.py in <module>()
      8 matrices1 = [csr_matrix(np.ones((5, 5)))]
      9 matrices2 = [csr_matrix(np.ones((5, 5)))]
     10 
     11 Parallel(n_jobs=2, max_nbytes=0)(
     12     delayed(manhattan_distances)(m1, m2)
---> 13     for m1, m2 in zip(matrices1, matrices2))
     14 
     15 
     16 
     17 

...........................................................................
/home/le243287/dev/scikit-learn/sklearn/externals/joblib/parallel.py in __call__(self=Parallel(n_jobs=2), iterable=<generator object <genexpr>>)
    763             if pre_dispatch == "all" or n_jobs == 1:
    764                 # The iterable was consumed all at once by the above for loop.
    765                 # No need to wait for async callbacks to trigger to
    766                 # consumption.
    767                 self._iterating = False
--> 768             self.retrieve()
        self.retrieve = <bound method Parallel.retrieve of Parallel(n_jobs=2)>
    769             # Make sure that we get a last message telling us we are done
    770             elapsed_time = time.time() - self._start_time
    771             self._print('Done %3i out of %3i | elapsed: %s finished',
    772                         (len(self._output), len(self._output),

---------------------------------------------------------------------------
Sub-process traceback:
---------------------------------------------------------------------------
ValueError                                         Mon Dec  5 13:19:44 2016
PID: 25869           Python 3.5.2: /volatile/le243287/miniconda3/bin/python
...........................................................................
/home/le243287/dev/scikit-learn/sklearn/externals/joblib/parallel.py in __call__(self=<sklearn.externals.joblib.parallel.BatchedCalls object>)
    126     def __init__(self, iterator_slice):
    127         self.items = list(iterator_slice)
    128         self._size = len(self.items)
    129 
    130     def __call__(self):
--> 131         return [func(*args, **kwargs) for func, args, kwargs in self.items]
        self.items = [(<function manhattan_distances>, (<5x5 sparse matrix of type '<class 'numpy.float6... stored elements in Compressed Sparse Row format>, <5x5 sparse matrix of type '<class 'numpy.float6... stored elements in Compressed Sparse Row format>), {})]
    132 
    133     def __len__(self):
    134         return self._size
    135 

...........................................................................
/home/le243287/dev/scikit-learn/sklearn/externals/joblib/parallel.py in <listcomp>(.0=<list_iterator object>)
    126     def __init__(self, iterator_slice):
    127         self.items = list(iterator_slice)
    128         self._size = len(self.items)
    129 
    130     def __call__(self):
--> 131         return [func(*args, **kwargs) for func, args, kwargs in self.items]
        func = <function manhattan_distances>
        args = (<5x5 sparse matrix of type '<class 'numpy.float6... stored elements in Compressed Sparse Row format>, <5x5 sparse matrix of type '<class 'numpy.float6... stored elements in Compressed Sparse Row format>)
        kwargs = {}
    132 
    133     def __len__(self):
    134         return self._size
    135 

...........................................................................
/home/le243287/dev/scikit-learn/sklearn/metrics/pairwise.py in manhattan_distances(X=<5x5 sparse matrix of type '<class 'numpy.float6... stored elements in Compressed Sparse Row format>, Y=<5x5 sparse matrix of type '<class 'numpy.float6... stored elements in Compressed Sparse Row format>, sum_over_features=True, size_threshold=500000000.0)
    528         X = csr_matrix(X, copy=False)
    529         Y = csr_matrix(Y, copy=False)
    530         D = np.zeros((X.shape[0], Y.shape[0]))
    531         _sparse_manhattan(X.data, X.indices, X.indptr,
    532                           Y.data, Y.indices, Y.indptr,
--> 533                           X.shape[1], D)
        X.shape = (5, 5)
        D = array([[ 0.,  0.,  0.,  0.,  0.],
       [ 0.,  ...0.,  0.,  0.],
       [ 0.,  0.,  0.,  0.,  0.]])
    534         return D
    535 
    536     if sum_over_features:
    537         return distance.cdist(X, Y, 'cityblock')

...........................................................................
/home/le243287/dev/scikit-learn/sklearn/metrics/pairwise_fast.cpython-35m-x86_64-linux-gnu.so in sklearn.metrics.pairwise_fast._sparse_manhattan (sklearn/metrics/pairwise_fast.c:4688)()
     48                     if nom != 0:
     49                         res  += denom * denom / nom
     50                 result[i, j] = -res
     51 
     52 
---> 53 def _sparse_manhattan(floating1d X_data, int[:] X_indices, int[:] X_indptr,
     54                       floating1d Y_data, int[:] Y_indices, int[:] Y_indptr,
     55                       np.npy_intp n_features, double[:, ::1] D):
     56     """Pairwise L1 distances for CSR matrices.
     57 

...........................................................................
/home/le243287/dev/scikit-learn/sklearn/metrics/pairwise_fast.cpython-35m-x86_64-linux-gnu.so in View.MemoryView.memoryview_cwrapper (sklearn/metrics/pairwise_fast.c:13277)()
    639 
    640 
    641 
    642 
    643 
--> 644 
    645 
    646 
    647 
    648 

...........................................................................
/home/le243287/dev/scikit-learn/sklearn/metrics/pairwise_fast.cpython-35m-x86_64-linux-gnu.so in View.MemoryView.memoryview.__cinit__ (sklearn/metrics/pairwise_fast.c:9512)()
    340 
    341 
    342 
    343 
    344 
--> 345 
    346 
    347 
    348 
    349 

ValueError: buffer source array is read-only
___________________________________________________________________________
@lesteve lesteve mentioned this issue Dec 5, 2016
Closed
@lesteve
Copy link
Member Author

lesteve commented Dec 5, 2016

Maybe similar to #4772.

@rahenri
Copy link

rahenri commented May 15, 2017

I just ran into this using latest version of numpy from pip.

@lesteve
Copy link
Member Author

lesteve commented Oct 27, 2017

There is a limitation of cython that typed memoryviews does not work with read-only data. Maybe we can fix this in scikit-learn, PR more than welcome in this case. Edit: quickly looking at the stacktrace, probably not.

@jnothman
Copy link
Member

jnothman commented Oct 28, 2017 via email

@lesteve
Copy link
Member Author

lesteve commented Nov 4, 2017

If you are controlling the Parallel object as in my snippet above, you can use max_nbytes=None to prevent the auto-memmapping. If you are not in control of the Parallel object, then tough luck, maybe we could fix it in scikit-learn by not using cython type memoryviews in _sparse_manhattan.

wehlutyk added a commit to ixxi-dante/jwalk that referenced this issue Nov 20, 2017
@wehlutyk
fix/corpus: allow setting max_nbytes in joblib.Parallel
fd8d2e8 
Lets us work around an issue similar to
scikit-learn/scikit-learn#7981
@rth rth mentioned this issue Feb 12, 2018
Closed
@Shihab-Shahriar
Copy link

Shihab-Shahriar commented Oct 15, 2019 •
edited
Loading

Encountered this today with the apply function of Random Forest Classifier. Happens only for comparatively big datasets like mnist, not digits or iris.

@thomasjpfan
Copy link
Member

I just tested this issue with the master version of Cython using const fused memoryviews and the issue was resolved.

@isaacgerg
Copy link

isaacgerg commented Nov 19, 2019 •
edited
Loading

Using sklearn 0.21.3, i run into this error when using DictionaryLearning when n_job>1 and a very large dataset. e.g.

dl = sklearn.decomposition.DictionaryLearning(int(1e2), n_jobs=1, alpha=1e-4, fit_algorithm='cd', verbose=2)
dl.fit(patches)

This was referenced Oct 3, 2021
Closed
Merged
@petergaultney
Copy link

This has bitten us within a GridSearchCV.

The workaround suggested by @lesteve did a great job; we ended up having to subclass joblib.LokyBackend, hack its configure method to set max_nbytes=None, and then use the joblib facilities for setting the backend directly. Kind of a disaster, but it works.

This was referenced Jan 18, 2023
Merged
Merged
jjerphan added a commit to jjerphan/scikit-learn that referenced this issue Jan 19, 2023
@jjerphan @lesteve
TST Add non-regression test for scikit-learn#7981
15f39d1 
This reproducer is adapted from the one of this message:
scikit-learn#7981 (comment)

Co-authored-by: Loïc Estève <loic.esteve@ymail.com>
@jjerphan jjerphan mentioned this issue Jan 19, 2023
Merged
lorentzenchr added a commit that referenced this issue Jan 26, 2023
@jjerphan @lesteve
@lorentzenchr @ogrisel
FIX Support readonly sparse datasets for manhattan_distances (#25432)
b69abf5 
* TST Add non-regression test for #7981

This reproducer is adapted from the one of this message:
#7981 (comment)

Co-authored-by: Loïc Estève <loic.esteve@ymail.com>

* FIX Support readonly sparse datasets for manhattan

* DOC Add entry in whats_new/v1.2.rst for 1.2.1

* FIX Fix comment

* Update sklearn/metrics/tests/test_pairwise.py

Co-authored-by: Christian Lorentzen <lorentzen.ch@gmail.com>

* DOC Move entry to whats_new/v1.3.rst

* Update sklearn/metrics/tests/test_pairwise.py

Co-authored-by: Olivier Grisel <olivier.grisel@ensta.org>

Co-authored-by: Loïc Estève <loic.esteve@ymail.com>
Co-authored-by: Christian Lorentzen <lorentzen.ch@gmail.com>
Co-authored-by: Olivier Grisel <olivier.grisel@ensta.org>
AdarshPrusty7 added a commit to AdarshPrusty7/GSGP that referenced this issue Mar 6, 2023
@AdarshPrusty7 @albuzenet
@glemaitre @jjerphan @remilvus @ogrisel @jeremiedbb @shadyelgewily-slimstock @jpangas @fsiola @betatim @lorentzenchr @lesteve @Anthony22-dev @anthonypic22 @yuanx749 @thomasjpfan @OmarManzoor @rprkh @GaelVaroquaux @ArturoAmorQ @Micky774 @davidblnc @changyaochen @nicolafan @Vincent-Maladiere @partev @ouss1508 @ashah002 @Ahmedbgh @pm155 @it176131 @sanjail3 @cozek @nwzish @mokwilliam @JanFidor @adam2392 @loganthomas @01-vyom @as-90 @Marvvxi @rickiepark @star1327p @ChVeen @sortofamudkip @sonnivs @alielkassas @yusufraji @tabeak @pooja-subramaniam @JuliaSchoepp @jack-mcivor @ZeeshanLone @MaxHalford @adrinjalali @genvalen @Shivachauhan17 @DayneSorvisto @rgommers @Kshitij68
Updates Boolean Primary (#2)
8d45050 
* ENH Raise NotFittedError in get_feature_names_out for MissingIndicator, KBinsDiscretizer, SplineTransformer, DictVectorizer (scikit-learn#25402)

Co-authored-by: Alex <alex.buzenet.fr@gmail.com>
Co-authored-by: Guillaume Lemaitre <g.lemaitre58@gmail.com>

* DOC Update date and contributors list for v1.2.1 (scikit-learn#25459)

* DOC Make MeanShift documentation clearer (scikit-learn#25305)

Co-authored-by: Guillaume Lemaitre <g.lemaitre58@gmail.com>

* Finishes boolean and arithmetic creation

* Skeleton for traditional GP

* DOC Reorder whats_new/v1.2.rst (scikit-learn#25461)

Follow-up of scikit-learn#25459

Co-authored-by: Olivier Grisel <olivier.grisel@ensta.org>
Co-authored-by: Jérémie du Boisberranger <jeremiedbb@users.noreply.github.com>

Co-authored-by: Olivier Grisel <olivier.grisel@ensta.org>
Co-authored-by: Jérémie du Boisberranger <jeremiedbb@users.noreply.github.com>

* FIX fix faulty test in `cross_validate` that used the wrong estimator (scikit-learn#25456)

* ENH Raise NotFittedError in get_feature_names_out for estimators that use ClassNamePrefixFeatureOutMixin and SelectorMixin (scikit-learn#25308)

Co-authored-by: Guillaume Lemaitre <g.lemaitre58@gmail.com>

* EFF Improve IsolationForest predict time (scikit-learn#25186)

Co-authored-by: Felipe Breve Siola <felipe.breve-siola@klarna.com>
Co-authored-by: Guillaume Lemaitre <g.lemaitre58@gmail.com>
Co-authored-by: Olivier Grisel <olivier.grisel@ensta.org>
Co-authored-by: Tim Head <betatim@gmail.com>

* MAINT refactor spectral_clustering to call SpectralClustering (scikit-learn#25392)

* TST reduce warnings in test_logistic.py (scikit-learn#25469)

* CI Build doc on CircleCI (scikit-learn#25466)

* DOC Update news footer for 1.2.1 (scikit-learn#25472)

* MAINT Validate parameter for `sklearn.cluster.cluster_optics_xi` (scikit-learn#25385)

Co-authored-by: adossantosalfam <anthony.dos_santos_alfama@insa-rouen.fr>
Co-authored-by: Guillaume Lemaitre <g.lemaitre58@gmail.com>

* MAINT Parameters validation for additive_chi2_kernel (scikit-learn#25424)

Co-authored-by: Guillaume Lemaitre <g.lemaitre58@gmail.com>

* Initial Program Creation

* CI Include linting in CircleCI (scikit-learn#25475)

* MAINT Update version number to 1.2.1 in SECURITY.md (scikit-learn#25471)

* TST Sets random_state for test_logistic.py (scikit-learn#25446)

* MAINT Remove -Wcpp warnings when compiling sklearn.decomposition._online_lda_fast (scikit-learn#25020)

Co-authored-by: Julien Jerphanion <git@jjerphan.xyz>

* FIX Support readonly sparse datasets for `manhattan_distances`  (scikit-learn#25432)

* TST Add non-regression test for scikit-learn#7981

This reproducer is adapted from the one of this message:
scikit-learn#7981 (comment)

Co-authored-by: Loïc Estève <loic.esteve@ymail.com>

* FIX Support readonly sparse datasets for manhattan

* DOC Add entry in whats_new/v1.2.rst for 1.2.1

* FIX Fix comment

* Update sklearn/metrics/tests/test_pairwise.py

Co-authored-by: Christian Lorentzen <lorentzen.ch@gmail.com>

* DOC Move entry to whats_new/v1.3.rst

* Update sklearn/metrics/tests/test_pairwise.py

Co-authored-by: Olivier Grisel <olivier.grisel@ensta.org>

Co-authored-by: Loïc Estève <loic.esteve@ymail.com>
Co-authored-by: Christian Lorentzen <lorentzen.ch@gmail.com>
Co-authored-by: Olivier Grisel <olivier.grisel@ensta.org>

* MAINT dynamically expose kulsinski and remove support in BallTree (scikit-learn#25417)

Co-authored-by: Loïc Estève <loic.esteve@ymail.com>
Co-authored-by: Julien Jerphanion <git@jjerphan.xyz>
closes scikit-learn#25212

* DOC Adds CirrusCI badge to readme (scikit-learn#25483)

* CI add linter display name (scikit-learn#25485)

* DOC update description of X in `FunctionTransformer.transform()`  (scikit-learn#24844)

* MAINT remove -Wcpp warnings when compiling sklearn.preprocessing._csr_polynomial_expansion (scikit-learn#25041)

* DOC more didactic example of bisecting kmeans (scikit-learn#25494)

Co-authored-by: Olivier Grisel <olivier.grisel@ensta.org>
Co-authored-by: Arturo Amor <86408019+ArturoAmorQ@users.noreply.github.com>
Co-authored-by: Jérémie du Boisberranger <34657725+jeremiedbb@users.noreply.github.com>

* ENH csr_row_norms optimization (scikit-learn#24426)

Co-authored-by: Julien Jerphanion <git@jjerphan.xyz>
Co-authored-by: Guillaume Lemaitre <g.lemaitre58@gmail.com>
Co-authored-by: Jérémie du Boisberranger <jeremiedbb@users.noreply.github.com>

* TST Allow callables as valid parameter regarding cloning estimator (scikit-learn#25498)

Co-authored-by: Guillaume Lemaitre <g.lemaitre58@gmail.com>
Co-authored-by: Loïc Estève <loic.esteve@ymail.com>
Co-authored-by: From: Tim Head <betatim@gmail.com>

* DOC Fixes sphinx search on website (scikit-learn#25504)

* FIX make IsotonicRegression always predict NumPy arrays (scikit-learn#25500)



Co-authored-by: Thomas J. Fan <thomasjpfan@gmail.com>

* FEA Add Gamma deviance as loss function to HGBT (scikit-learn#22409)

* FEA add gamma loss to HGBT

* DOC add whatsnew

* CLN address review comments

* TST make test_gamma pass by not testing out-of-sample

* TST compare gamma and poisson to LightGBM

* TST fix test_gamma by comparing to MSE HGBT instead of Poisson HGBT

* TST fix for test_same_predictions_regression for poisson

* CLN address review comments

* CLN nits

* CLN better comments

* TST use pytest.param with skip mark

* TST Correct conditional test parametrization mark

Co-authored-by: Christian Lorentzen <lorentzen.ch@gmail.com>

* CI Trigger CI

Builds currently fail because requests to Azure Ubuntu repository
timeout.

* DOC add comment for lax comparison with LightGBM

* CLN tuple needs trailing comma

---------

Co-authored-by: Julien Jerphanion <git@jjerphan.xyz>

* MAINT Remove -Wsign-compare warnings when compiling sklearn.tree._tree (scikit-learn#25507)

* MAINT add more intuition on OAS computation based on literature (scikit-learn#23867)

Co-authored-by: Guillaume Lemaitre <g.lemaitre58@gmail.com>

* CI Allow cirrus arm tests to run with cd build commit tag (scikit-learn#25514)

* CI Upload ARM wheels from CirrusCI to nightly and staging index (scikit-learn#25513)



Co-authored-by: Olivier Grisel <olivier.grisel@ensta.org>

* MAINT Remove -Wcpp warnings from sklearn.utils._seq_dataset (scikit-learn#25406)

* FIX Fixes linux ARM CI on CirrusCI (scikit-learn#25536)

* DOC Fix grammatical mistake in `mixture` module (scikit-learn#25541)

* DOC add missing trailing colon (scikit-learn#25542)

* MAINT Parameters validation for sklearn.datasets.make_classification (scikit-learn#25474)

Co-authored-by: Guillaume Lemaitre <g.lemaitre58@gmail.com>

* MNT Expose allow_nan tag in bagging (scikit-learn#25506)

* MAINT Clean-up comments and rename variables in `_middle_term_sparse_sparse_{32, 64}` (scikit-learn#25449)

Co-authored-by: Julien Jerphanion <git@jjerphan.xyz>

* DOC: remove incorrect statement (scikit-learn#25544)

* MAINT Parameters validation for reconstruct_from_patches_2d (scikit-learn#25384)

Co-authored-by: Guillaume Lemaitre <g.lemaitre58@gmail.com>

* MAINT Parameter validation for sklearn.metrics.d2_pinball_score (scikit-learn#25414)

Co-authored-by: Guillaume Lemaitre <g.lemaitre58@gmail.com>
Co-authored-by: Jérémie du Boisberranger <34657725+jeremiedbb@users.noreply.github.com>

* MAINT Parameters validation for spectral_clustering (scikit-learn#25378)

Co-authored-by: Guillaume Lemaitre <g.lemaitre58@gmail.com>

* MAINT Parameters validation for sklearn.datasets.fetch_kddcup99 (scikit-learn#25463)

Co-authored-by: Guillaume Lemaitre <g.lemaitre58@gmail.com>

* DOC Update MLPRegressor docs (scikit-learn#25556)

Co-authored-by: Ian Thompson <ian.thompson@hrblock.com>

* DOC Update docs for KMeans (scikit-learn#25546)

Co-authored-by: Jérémie du Boisberranger <34657725+jeremiedbb@users.noreply.github.com>

* FIX BisectingKMeans crashes randomly (scikit-learn#25563)

Fixes scikit-learn#25505

* ENH BaseLabelPropagation to accept sparse matrices (scikit-learn#19664)

Co-authored-by: Kaushik Amar Das <kaushik.amar.das@accenture.com>
Co-authored-by: Guillaume Lemaitre <g.lemaitre58@gmail.com>

* MAINT Remove travis ci config and related doc (scikit-learn#25562)

* DOC Add pynndescent to Approximate nearest neighbors in TSNE example (scikit-learn#25480)


Co-authored-by: Olivier Grisel <olivier.grisel@ensta.org>

* DOC Add docstring example to make_regression (scikit-learn#25551)

Co-authored-by: Thomas J. Fan <thomasjpfan@gmail.com>
Co-authored-by: Jérémie du Boisberranger <34657725+jeremiedbb@users.noreply.github.com>

* MAINT ensure that pos_label support all possible types (scikit-learn#25317)

* MAINT Parameters validation for sklearn.metrics.f1_score (scikit-learn#25557)

Co-authored-by: Jérémie du Boisberranger <34657725+jeremiedbb@users.noreply.github.com>

* ENH Adds `class_names` to `tree.export_text` (scikit-learn#25387)

Co-authored-by: Thomas J. Fan <thomasjpfan@gmail.com>
Co-authored-by: Guillaume Lemaitre <g.lemaitre58@gmail.com>

* MAINT Replace cnp.ndarray with memory views in sklearn.tree._tree (where possible) (scikit-learn#25540)

* DOC Change print format in TSNE example (scikit-learn#25569)

Co-authored-by: Olivier Grisel <olivier.grisel@ensta.org>

* FIX ColumnTransformer supports empty selection for pandas output (scikit-learn#25570)

Co-authored-by: Julien Jerphanion <git@jjerphan.xyz>

* DOC fix docstring of _plain_sgd (scikit-learn#25573)

* FIX Enable setting of sub-parameters for deprecated base_estimator param (scikit-learn#25477)

* DOC Improve minor and bug-fix release processes documentation (scikit-learn#25457)

Co-authored-by: Olivier Grisel <olivier.grisel@ensta.org>
Co-authored-by: Thomas J. Fan <thomasjpfan@gmail.com>
Co-authored-by: Jérémie du Boisberranger <jeremiedbb@yahoo.fr>

* MAINT Remove ReadonlyArrayWrapper from _loss module (scikit-learn#25555)

* MAINT Remove ReadonlyArrayWrapper from _loss module

* CLN Remove comments about Cython 3.0

* MAINT Remove ReadonlyArrayWrapper from _kmeans (scikit-learn#25554)

* MAINT Remove ReadonlyArrayWrapper from _kmeans

* more const and remove blas compile warnings

* CLN Adds comment about casting to non const pointers

* Update sklearn/utils/_cython_blas.pyx

* MAINT Remove ReadonlyArrayWrapper from DistanceMetric (scikit-learn#25553)

* DOC improve stop_words description w.r.t. max_df range in CountVectorizer (scikit-learn#25489)

* MAINT Removes ReadOnlyWrapper (scikit-learn#25586)

* MAINT Parameters validation for sklearn.metrics.log_loss (scikit-learn#25577)

* MAINT Adds comments and better naming into tree code (scikit-learn#25576)

* MAINT Adds comments and better naming into tree code

* CLN Use feature_values instead of Xf

* Apply suggestions from code review

Co-authored-by: Adam Li <adam2392@gmail.com>

* DOC Improve comment from review

* Apply suggestions from code review

Co-authored-by: Julien Jerphanion <git@jjerphan.xyz>
Co-authored-by: Olivier Grisel <olivier.grisel@ensta.org>

---------

Co-authored-by: Adam Li <adam2392@gmail.com>
Co-authored-by: Julien Jerphanion <git@jjerphan.xyz>
Co-authored-by: Olivier Grisel <olivier.grisel@ensta.org>

* FIX error when deserialzing a Tree instance from a read only buffer (scikit-learn#25585)

* DOC: fix typo in California Housing dataset description (scikit-learn#25613)

* ENH: Update KDTree, and example documentation (scikit-learn#25482)

* ENH: Update KDTree, and example documentation

* ENH: Add valid metric function and reference doc

* CHG: Documentation update

Co-authored-by: Adam Li <adam2392@gmail.com>

* CHG: make valid metric property and fix doc string

* FIX: documentation, and add code example

* ENH: Change valid metric to class method, and doc

* ENH: Change valid metric class variable, and doc

* FIX: documentation error

* FIX: documentation error

* CHG: Use class method for valid metrics

* FIX: CI problems

---------

Co-authored-by: Adam Li <adam2392@gmail.com>
Co-authored-by: Julien Jerphanion <git@jjerphan.xyz>

* TST Common test for checking estimator deserialization from a read only buffer (scikit-learn#25624)

* DOC fix comment in plot_logistic_l1_l2_sparsity.py (scikit-learn#25633)

Co-authored-by: Thomas J. Fan <thomasjpfan@gmail.com>

* DOC Places governance in navigation bar (scikit-learn#25618)

* MAINT Check pyproject toml is consistent with min_dependencies (scikit-learn#25610)

* MAINT Check pyproject toml is consistent with min_dependencies

* CLN Make it clear that only SciPy and Cython are checked

* CLN Revert auto formatter

* MAINT Use newest NumPy C API in tree._criterion (scikit-learn#25615)

* MAINT Use newest NumPy C API in tree._criterion

* FIX Use pointer for children

* FIX Fixes check_array nonfinite checks with ArrayAPI specification (scikit-learn#25619)

* FIX Fixes check_array nonfinite checks with ArrayAPI specification

* DOC Adds PR number

* FIX Test on both cupy and numpy

* DOC Correctly docstring in StackingRegressor.fit_transform (scikit-learn#25599)

* MAINT Remove Cython compilation warnings ahead of Cython3.0 release (scikit-learn#25621)

* ENH Preserve DataFrame dtypes in transform for feature selectors (scikit-learn#25102)

* FIX report properly n_iter_ when warm_start=True (scikit-learn#25443)

Co-authored-by: Guillaume Lemaitre <g.lemaitre58@gmail.com>

* DOC fix typo in KMeans's param. (scikit-learn#25649)

* FIX use const memory views in hist_gradient_boosting predictor (scikit-learn#25650)

* DOC modified the graph for better readability (scikit-learn#25644)

* MAINT Removes upper limit on setuptools (scikit-learn#25651)

* DOC improve the `warm_start` glossary entry (scikit-learn#25523)

* DOC Update governance document for SLEP020 (scikit-learn#25663)



Co-authored-by: Tim Head <betatim@gmail.com>
Co-authored-by: Christian Lorentzen <lorentzen.ch@gmail.com>

* FIX renormalization of y_pred inside log_loss (scikit-learn#25299)

* Remove renormalization of y_pred inside log_loss

* Deprecate eps parameter in log_loss

* ENH Allows target to be pandas nullable dtypes (scikit-learn#25638)

* DOC unify usage of 'w.r.t.' (scikit-learn#25683)

* MAINT Parameters validation for metrics.max_error (scikit-learn#25679)

* MAINT Parameters validation for datasets.make_friedman1 (scikit-learn#25674)

Co-authored-by: jeremie du boisberranger <jeremiedbb@yahoo.fr>

* MAINT Parameters validation for mean_pinball_loss (scikit-learn#25685)

Co-authored-by: jeremie du boisberranger <jeremiedbb@yahoo.fr>

* DOC Specify behavior of None for CountVectorizer (scikit-learn#25678)

* DOC Specify behaviour of None for TfIdfVectorizer max_features parameter (scikit-learn#25676)

Co-authored-by: Guillaume Lemaitre <g.lemaitre58@gmail.com>

* MAINT Set random state for plot_anomaly_comparison (scikit-learn#25675)

* MAINT Parameters validation for cluster.mean_shift (scikit-learn#25684)

Co-authored-by: jeremie du boisberranger <jeremiedbb@yahoo.fr>

* MAINT Parameters validation for sklearn.metrics.jaccard_score (scikit-learn#25680)

Co-authored-by: jeremie du boisberranger <jeremiedbb@yahoo.fr>

* DOC Add the custom compiler section back (scikit-learn#25667)

Co-authored-by: Thomas J. Fan <thomasjpfan@gmail.com>

* MAINT Parameters validation for precision_recall_fscore_support (scikit-learn#25681)

Co-authored-by: jeremie du boisberranger <jeremiedbb@yahoo.fr>

* FIX Allow negative tol in SequentialFeatureSelector (scikit-learn#25664)

* MAINT Replace deprecated cython conditional compilation (scikit-learn#25654)



Co-authored-by: Guillaume Lemaitre <g.lemaitre58@gmail.com>

* DOC fix formatting typo in related_projects (scikit-learn#25706)

* MAINT Parameters validation for metrics.mean_absolute_percentage_error (scikit-learn#25695)

* MAINT Parameters validation for metrics.precision_recall_curve (scikit-learn#25698)

Co-authored-by: jeremie du boisberranger <jeremiedbb@yahoo.fr>

* MAINT Parameter Validation for metrics.precision_score (scikit-learn#25708)

Co-authored-by: jeremie du boisberranger <jeremiedbb@yahoo.fr>

* CI Stablize build with random_state (scikit-learn#25701)

Co-authored-by: Jérémie du Boisberranger <34657725+jeremiedbb@users.noreply.github.com>

* MAINT Remove -Wcpp warnings when compiling arrayfuncs (scikit-learn#25415)

Co-authored-by: Thomas J. Fan <thomasjpfan@gmail.com>

* DOC Add scikit-learn-intelex to related projects (scikit-learn#23766)

Co-authored-by: Adrin Jalali <adrin.jalali@gmail.com>
Co-authored-by: Thomas J. Fan <thomasjpfan@gmail.com>

* ENH Support float32 in SGDClassifier and SGDRegressor (scikit-learn#25587)

* FIX Raise appropriate attribute error in ensemble (scikit-learn#25668)

* FIX Allow OrdinalEncoder's encoded_missing_value set to the cardinality (scikit-learn#25704)

* ENH Let csr_row_norms support multi-thread (scikit-learn#25598)

Co-authored-by: Jérémie du Boisberranger <34657725+jeremiedbb@users.noreply.github.com>
Co-authored-by: Vincent M <maladiere.vincent@yahoo.fr>

* MAINT Parameter Validation for feature_selection.chi2 (scikit-learn#25719)

Co-authored-by: jeremiedbb <jeremiedbb@yahoo.fr>

* MAINT Parameter Validation for feature_selection.f_classif (scikit-learn#25720)

Co-authored-by: Jérémie du Boisberranger <34657725+jeremiedbb@users.noreply.github.com>

* MAINT Parameters validation for sklearn.metrics.matthews_corrcoef (scikit-learn#25712)

Co-authored-by: jeremiedbb <jeremiedbb@yahoo.fr>

* MAINT parameter validation for sklearn.datasets.dump_svmlight_file (scikit-learn#25726)

Co-authored-by: jeremiedbb <jeremiedbb@yahoo.fr>

* MAINT Clean dead code in build helpers (scikit-learn#25661)

* MAINT Use newest NumPy C API in metrics._dist_metrics (scikit-learn#25702)

* CI Adds permissions to workflows that use GITHUB_TOKEN (scikit-learn#25600)

Co-authored-by: Olivier Grisel <olivier.grisel@ensta.org>

* FIX Improves error message in partial_fit when early_stopping=True (scikit-learn#25694)

Co-authored-by: Jérémie du Boisberranger <34657725+jeremiedbb@users.noreply.github.com>

* DOC Makes navbar static (scikit-learn#25688)

* MAINT Remove redundant sparse square euclidian distances function (scikit-learn#25731)

* MAINT Use float64 for accumulators in WeightVector* (scikit-learn#25721)

* API make PatchExtractor being a real scikit-learn transformer (scikit-learn#24230)

Co-authored-by: Thomas J. Fan <thomasjpfan@gmail.com>
Co-authored-by: Jérémie du Boisberranger <34657725+jeremiedbb@users.noreply.github.com>

* MAINT Update pyparsing.py to use bool instead of double negation (scikit-learn#25724)

* API Deprecates values in partial_dependence in favor of pdp_values (scikit-learn#21809)

Co-authored-by: Guillaume Lemaitre <g.lemaitre58@gmail.com>
Co-authored-by: Jérémie du Boisberranger <34657725+jeremiedbb@users.noreply.github.com>

* API Use grid_values instead of pdp_values in partial_dependence (scikit-learn#25732)

* MAINT remove np.product and inf/nan aliases in favor of canonical names (scikit-learn#25741)

* MAINT Parameters validation for metrics.label_ranking_loss (scikit-learn#25742)

Co-authored-by: Jérémie du Boisberranger <34657725+jeremiedbb@users.noreply.github.com>

* MAINT Parameters validation for metrics.coverage_error (scikit-learn#25748)

* MAINT Parameters validation for metrics.dcg_score (scikit-learn#25749)

* MAINT replace cnp.ndarray with memory views in _fast_dict (scikit-learn#25754)

* MAINT Parameter Validation for feature_selection.f_regression (scikit-learn#25736)

Co-authored-by: Jérémie du Boisberranger <34657725+jeremiedbb@users.noreply.github.com>

* MAINT Parameters validation for feature_selection.r_regression (scikit-learn#25734)

Co-authored-by: Jérémie du Boisberranger <34657725+jeremiedbb@users.noreply.github.com>

* MAINT Parameter Validation for metrics.get_scorer (scikit-learn#25738)

Co-authored-by: Jérémie du Boisberranger <34657725+jeremiedbb@users.noreply.github.com>

* DOC Move allowing pandas nullable dtypes to 1.2.2 (scikit-learn#25692)

* MAINT replace cnp.ndarray with memory views in sparsefuncs_fast (scikit-learn#25764)

* MAINT parameter validation for sklearn.datasets.fetch_covtype (scikit-learn#25759)

Co-authored-by: Jérémie du Boisberranger <34657725+jeremiedbb@users.noreply.github.com>

* MAINT Define centralized generic, but with explicit precision, types (scikit-learn#25739)

* CI Disable network when SciPy requires it (scikit-learn#25743)

* CI Open issue when arm wheel fails on CirrusCI (scikit-learn#25620)

* ENH Speed-up expected mutual information (scikit-learn#25713)

Co-authored-by: Kshitij Mathur <k.mathur68@gmail.com>
Co-authored-by: Guillaume Lemaitre <g.lemaitre58@gmail.com>
Co-authored-by: Omar Salman <omar.salman@arbisoft.com>

* FIX add retry mechanism to handle quotechar in read_csv (scikit-learn#25511)

* Merge Population Creation (#1)

---------

Co-authored-by: Alex Buzenet <94121450+albuzenet@users.noreply.github.com>
Co-authored-by: Alex <alex.buzenet.fr@gmail.com>
Co-authored-by: Guillaume Lemaitre <g.lemaitre58@gmail.com>
Co-authored-by: Julien Jerphanion <git@jjerphan.xyz>
Co-authored-by: Adam Kania <48769688+remilvus@users.noreply.github.com>
Co-authored-by: Olivier Grisel <olivier.grisel@ensta.org>
Co-authored-by: Jérémie du Boisberranger <jeremiedbb@users.noreply.github.com>
Co-authored-by: Shady el Gewily <90049412+shadyelgewily-slimstock@users.noreply.github.com>
Co-authored-by: John Pangas <swiftyxswaggy@outlook.com>
Co-authored-by: Felipe Siola <fsiola@gmail.com>
Co-authored-by: Felipe Breve Siola <felipe.breve-siola@klarna.com>
Co-authored-by: Tim Head <betatim@gmail.com>
Co-authored-by: Christian Lorentzen <lorentzen.ch@gmail.com>
Co-authored-by: Loïc Estève <loic.esteve@ymail.com>
Co-authored-by: Anthony22-dev <122220081+Anthony22-dev@users.noreply.github.com>
Co-authored-by: adossantosalfam <anthony.dos_santos_alfama@insa-rouen.fr>
Co-authored-by: Xiao Yuan <yuanx749@gmail.com>
Co-authored-by: Thomas J. Fan <thomasjpfan@gmail.com>
Co-authored-by: Omar Salman <omar.salman@arbisoft.com>
Co-authored-by: Rahil Parikh <75483881+rprkh@users.noreply.github.com>
Co-authored-by: Gael Varoquaux <gael.varoquaux@normalesup.org>
Co-authored-by: Arturo Amor <86408019+ArturoAmorQ@users.noreply.github.com>
Co-authored-by: Jérémie du Boisberranger <34657725+jeremiedbb@users.noreply.github.com>
Co-authored-by: Meekail Zain <34613774+Micky774@users.noreply.github.com>
Co-authored-by: davidblnc <40642621+davidblnc@users.noreply.github.com>
Co-authored-by: Changyao Chen <changyao.chen@gmail.com>
Co-authored-by: Nicola Fanelli <48762613+nicolafan@users.noreply.github.com>
Co-authored-by: Vincent M <maladiere.vincent@yahoo.fr>
Co-authored-by: partev <petrosyan@gmail.com>
Co-authored-by: ouss1508 <121971998+ouss1508@users.noreply.github.com>
Co-authored-by: ashah002 <97778401+ashah002@users.noreply.github.com>
Co-authored-by: Ahmedbgh <83551938+Ahmedbgh@users.noreply.github.com>
Co-authored-by: Pooja M <90301980+pm155@users.noreply.github.com>
Co-authored-by: Ian Thompson <ianiat11@gmail.com>
Co-authored-by: Ian Thompson <ian.thompson@hrblock.com>
Co-authored-by: SANJAI_3 <86285670+sanjail3@users.noreply.github.com>
Co-authored-by: Kaushik Amar Das <cozek@users.noreply.github.com>
Co-authored-by: Kaushik Amar Das <kaushik.amar.das@accenture.com>
Co-authored-by: Nawazish Alam <nawazishmail@gmail.com>
Co-authored-by: William M <64324808+Akbeeh@users.noreply.github.com>
Co-authored-by: Jérémie du Boisberranger <jeremiedbb@yahoo.fr>
Co-authored-by: JanFidor <66260538+JanFidor@users.noreply.github.com>
Co-authored-by: Adam Li <adam2392@gmail.com>
Co-authored-by: Logan Thomas <logan.thomas005@gmail.com>
Co-authored-by: Vyom Pathak <angerstick3@gmail.com>
Co-authored-by: as-90 <88336957+as-90@users.noreply.github.com>
Co-authored-by: Marvin Krawutschke <101656586+Marvvxi@users.noreply.github.com>
Co-authored-by: Haesun Park <haesunrpark@gmail.com>
Co-authored-by: Christine P. Chai <star1327p@gmail.com>
Co-authored-by: Christian Veenhuis <124370897+ChVeen@users.noreply.github.com>
Co-authored-by: Sortofamudkip <wishyutp0328@gmail.com>
Co-authored-by: sonnivs <48860780+sonnivs@users.noreply.github.com>
Co-authored-by: Ali H. El-Kassas <aliabdelmonem234@gmail.com>
Co-authored-by: Yusuf Raji <raji.yusuf234@gmail.com>
Co-authored-by: Tabea Kossen <tabeakossen@gmail.com>
Co-authored-by: Pooja Subramaniam <poojas2086@gmail.com>
Co-authored-by: JuliaSchoepp <63353759+JuliaSchoepp@users.noreply.github.com>
Co-authored-by: Jack McIvor <jacktmcivor@gmail.com>
Co-authored-by: zeeshan lone <56621467+still-learning-ev@users.noreply.github.com>
Co-authored-by: Max Halford <maxhalford25@gmail.com>
Co-authored-by: Adrin Jalali <adrin.jalali@gmail.com>
Co-authored-by: genvalen <genvalen@protonmail.com>
Co-authored-by: Shiva chauhan <103742975+Shivachauhan17@users.noreply.github.com>
Co-authored-by: Dayne <daynesorvisto@yahoo.ca>
Co-authored-by: Ralf Gommers <ralf.gommers@gmail.com>
Co-authored-by: Kshitij Mathur <k.mathur68@gmail.com>
@jilljenn jilljenn mentioned this issue Dec 27, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Bug help wanted module:metrics
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

FIX Support readonly sparse datasets for manhattan_distances jjerphan/scikit-learn
9 participants
@jnothman @rahenri @amueller @petergaultney @cmarmo @lesteve @thomasjpfan @Shihab-Shahriar @isaacgerg