diff --git a/doc/modules/svm.rst b/doc/modules/svm.rst index 386865d3d0a8a..62d566fe150ba 100644 --- a/doc/modules/svm.rst +++ b/doc/modules/svm.rst @@ -212,13 +212,12 @@ Then ``dual_coef_`` looks like this: Scores and probabilities ------------------------ -The :class:`SVC` method ``decision_function`` gives per-class scores -for each sample (or a single score per sample in the binary case). -When the constructor option ``probability`` is set to ``True``, -class membership probability estimates -(from the methods ``predict_proba`` and ``predict_log_proba``) are enabled. -In the binary case, the probabilities are calibrated using Platt scaling: -logistic regression on the SVM's scores, +The ``decision_function`` method of :class:`SVC` and :class:`NuSVC` gives +per-class scores for each sample (or a single score per sample in the binary +case). When the constructor option ``probability`` is set to ``True``, +class membership probability estimates (from the methods ``predict_proba`` and +``predict_log_proba``) are enabled. In the binary case, the probabilities are +calibrated using Platt scaling: logistic regression on the SVM's scores, fit by an additional cross-validation on the training data. In the multiclass case, this is extended as per Wu et al. (2004). @@ -245,7 +244,7 @@ and use ``decision_function`` instead of ``predict_proba``. * Platt `"Probabilistic outputs for SVMs and comparisons to regularized likelihood methods" - `. + `_. Unbalanced problems -------------------- @@ -399,7 +398,7 @@ Tips on Practical Use function can be configured to be almost the same as the :class:`LinearSVC` model. - * **Kernel cache size**: For :class:`SVC`, :class:`SVR`, :class:`nuSVC` and + * **Kernel cache size**: For :class:`SVC`, :class:`SVR`, :class:`NuSVC` and :class:`NuSVR`, the size of the kernel cache has a strong impact on run times for larger problems. If you have enough RAM available, it is recommended to set ``cache_size`` to a higher value than the default of @@ -423,10 +422,24 @@ Tips on Practical Use positive and few negative), set ``class_weight='balanced'`` and/or try different penalty parameters ``C``. - * The underlying :class:`LinearSVC` implementation uses a random - number generator to select features when fitting the model. It is - thus not uncommon, to have slightly different results for the same - input data. If that happens, try with a smaller tol parameter. + * **Randomness of the underlying implementations**: The underlying + implementations of :class:`SVC` and :class:`NuSVC` use a random number + generator only to shuffle the data for probability estimation (when + ``probability`` is set to ``True``). This randomness can be controlled + with the ``random_state`` parameter. If ``probability`` is set to ``False`` + these estimators are not random and ``random_state`` has no effect on the + results. The underlying :class:`OneClassSVM` implementation is similar to + the ones of :class:`SVC` and :class:`NuSVC`. As no probability estimation + is provided for :class:`OneClassSVM`, it is not random. + + The underlying :class:`LinearSVC` implementation uses a random number + generator to select features when fitting the model with a dual coordinate + descent (i.e when ``dual`` is set to ``True``). It is thus not uncommon, + to have slightly different results for the same input data. If that + happens, try with a smaller tol parameter. This randomness can also be + controlled with the ``random_state`` parameter. When ``dual`` is + set to ``False`` the underlying implementation of :class:`LinearSVC` is + not random and ``random_state`` has no effect on the results. * Using L1 penalization as provided by ``LinearSVC(loss='l2', penalty='l1', dual=False)`` yields a sparse solution, i.e. only a subset of feature diff --git a/doc/whats_new.rst b/doc/whats_new.rst index 5de27d3251787..965a7cd09a280 100644 --- a/doc/whats_new.rst +++ b/doc/whats_new.rst @@ -60,6 +60,12 @@ Model evaluation and meta-estimators - A scorer based on :func:`metrics.brier_score_loss` is also available. :issue:`9521` by :user:`Hanmin Qin `. +Linear, kernelized and related models + +- Deprecate ``random_state`` parameter in :class:`svm.OneClassSVM` as the + underlying implementation is not random. + :issue:`9497` by :user:`Albert Thomas `. + Bug fixes ......... @@ -82,6 +88,15 @@ Decomposition, manifold learning and clustering where all samples had equal similarity. :issue:`9612`. By :user:`Jonatan Samoocha `. +API changes summary +------------------- + +Linear, kernelized and related models + +- Deprecate ``random_state`` parameter in :class:`svm.OneClassSVM` as the + underlying implementation is not random. + :issue:`9497` by :user:`Albert Thomas `. + Version 0.19 ============ diff --git a/sklearn/svm/classes.py b/sklearn/svm/classes.py index 7c6642a504ad1..551451a47f5a6 100644 --- a/sklearn/svm/classes.py +++ b/sklearn/svm/classes.py @@ -88,10 +88,13 @@ class LinearSVC(BaseEstimator, LinearClassifierMixin, random_state : int, RandomState instance or None, optional (default=None) The seed of the pseudo random number generator to use when shuffling - the data. If int, random_state is the seed used by the random number - generator; If RandomState instance, random_state is the random number - generator; If None, the random number generator is the RandomState - instance used by `np.random`. + the data for the dual coordinate descent (if ``dual=True``). When + ``dual=False`` the underlying implementation of :class:`LinearSVC` + is not random and ``random_state`` has no effect on the results. If + int, random_state is the seed used by the random number generator; If + RandomState instance, random_state is the random number generator; If + None, the random number generator is the RandomState instance used by + `np.random`. max_iter : int, (default=1000) The maximum number of iterations to be run. @@ -509,11 +512,11 @@ class SVC(BaseSVC): Deprecated *decision_function_shape='ovo' and None*. random_state : int, RandomState instance or None, optional (default=None) - The seed of the pseudo random number generator to use when shuffling - the data. If int, random_state is the seed used by the random number - generator; If RandomState instance, random_state is the random number - generator; If None, the random number generator is the RandomState - instance used by `np.random`. + The seed of the pseudo random number generator used when shuffling + the data for probability estimates. If int, random_state is the + seed used by the random number generator; If RandomState instance, + random_state is the random number generator; If None, the random + number generator is the RandomState instance used by `np.random`. Attributes ---------- @@ -665,11 +668,11 @@ class NuSVC(BaseSVC): Deprecated *decision_function_shape='ovo' and None*. random_state : int, RandomState instance or None, optional (default=None) - The seed of the pseudo random number generator to use when shuffling - the data. If int, random_state is the seed used by the random number - generator; If RandomState instance, random_state is the random number - generator; If None, the random number generator is the RandomState - instance used by `np.random`. + The seed of the pseudo random number generator used when shuffling + the data for probability estimates. If int, random_state is the seed + used by the random number generator; If RandomState instance, + random_state is the random number generator; If None, the random + number generator is the RandomState instance used by `np.random`. Attributes ---------- @@ -1019,11 +1022,11 @@ class OneClassSVM(BaseLibSVM): Hard limit on iterations within solver, or -1 for no limit. random_state : int, RandomState instance or None, optional (default=None) - The seed of the pseudo random number generator to use when shuffling - the data. If int, random_state is the seed used by the random number - generator; If RandomState instance, random_state is the random number - generator; If None, the random number generator is the RandomState - instance used by `np.random`. + Ignored. + + .. deprecated:: 0.20 + ``random_state`` has been deprecated in 0.20 and will be removed in + 0.22. Attributes ---------- @@ -1080,6 +1083,11 @@ def fit(self, X, y=None, sample_weight=None, **params): If X is not a C-ordered contiguous array it is copied. """ + + if self.random_state is not None: + warnings.warn("The random_state parameter is deprecated and will" + " be removed in version 0.22.", DeprecationWarning) + super(OneClassSVM, self).fit(X, np.ones(_num_samples(X)), sample_weight=sample_weight, **params) return self