Skip to content

DOC, API: add random.__init__.pxd and document random.* functions #14948

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Nov 27, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
84 changes: 43 additions & 41 deletions doc/source/reference/random/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -22,34 +22,44 @@ Since Numpy version 1.17.0 the Generator can be initialized with a
number of different BitGenerators. It exposes many different probability
distributions. See `NEP 19 <https://www.numpy.org/neps/
nep-0019-rng-policy.html>`_ for context on the updated random Numpy number
routines. The legacy `.RandomState` random number routines are still
routines. The legacy `RandomState` random number routines are still
available, but limited to a single BitGenerator.

For convenience and backward compatibility, a single `~.RandomState`
For convenience and backward compatibility, a single `RandomState`
instance's methods are imported into the numpy.random namespace, see
:ref:`legacy` for the complete list.

.. _random-quick-start:

Quick Start
-----------

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rework the documentation in lines 36-52 to better guide users how to modify existing code to call default_rng

By default, `~Generator` uses bits provided by `PCG64` which
has better statistical properties than the legacy mt19937 random
number generator in `~.RandomState`.
Call `default_rng` to get a new instance of a `Generator`, then call its
methods to obtain samples from different distributions. By default,
`Generator` uses bits provided by `PCG64` which has better statistical
properties than the legacy `MT19937` used in `RandomState`.

.. code-block:: python

# Uses the old numpy.random.RandomState
# Do this
from numpy.random import default_rng
rng = default_rng()
vals = rng.standard_normal(10)
more_vals = rng.standard_normal(10)

# instead of this
from numpy import random
random.standard_normal()
vals = random.standard_normal(10)
more_vals = random.standard_normal(10)

`~Generator` can be used as a replacement for `~.RandomState`. Both class
instances now hold a internal `BitGenerator` instance to provide the bit
`Generator` can be used as a replacement for `RandomState`. Both class
instances hold a internal `BitGenerator` instance to provide the bit
stream, it is accessible as ``gen.bit_generator``. Some long-overdue API
cleanup means that legacy and compatibility methods have been removed from
`~.Generator`
`Generator`

=================== ============== ============
`~.RandomState` `~.Generator` Notes
`RandomState` `Generator` Notes
------------------- -------------- ------------
``random_sample``, ``random`` Compatible with `random.random`
``rand``
Expand All @@ -58,21 +68,12 @@ cleanup means that legacy and compatibility methods have been removed from
``random_integers``
------------------- -------------- ------------
``tomaxint`` removed Use ``integers(0, np.iinfo(np.int_).max,``
``endpoint=False)``
``endpoint=False)``
------------------- -------------- ------------
``seed`` removed Use `~.SeedSequence.spawn`
``seed`` removed Use `SeedSequence.spawn`
=================== ============== ============

See `new-or-different` for more information

.. code-block:: python

# As replacement for RandomState(); default_rng() instantiates Generator with
# the default PCG64 BitGenerator.
from numpy.random import default_rng
rg = default_rng()
rg.standard_normal()
rg.bit_generator
See :ref:`new-or-different` for more information

Something like the following code can be used to support both ``RandomState``
and ``Generator``, with the understanding that the interfaces are slightly
Expand All @@ -87,9 +88,9 @@ different
a = rg_integers(1000)

Seeds can be passed to any of the BitGenerators. The provided value is mixed
via `~.SeedSequence` to spread a possible sequence of seeds across a wider
range of initialization states for the BitGenerator. Here `~.PCG64` is used and
is wrapped with a `~.Generator`.
via `SeedSequence` to spread a possible sequence of seeds across a wider
range of initialization states for the BitGenerator. Here `PCG64` is used and
is wrapped with a `Generator`.

.. code-block:: python

Expand All @@ -100,7 +101,7 @@ is wrapped with a `~.Generator`.
Introduction
------------
The new infrastructure takes a different approach to producing random numbers
from the `~.RandomState` object. Random number generation is separated into
from the `RandomState` object. Random number generation is separated into
two components, a bit generator and a random generator.

The `BitGenerator` has a limited set of responsibilities. It manages state
Expand All @@ -113,8 +114,8 @@ distributions, e.g., simulated normal random values. This structure allows
alternative bit generators to be used with little code duplication.

The `Generator` is the user-facing object that is nearly identical to
`.RandomState`. The canonical method to initialize a generator passes a
`~.PCG64` bit generator as the sole argument.
`RandomState`. The canonical method to initialize a generator passes a
`PCG64` bit generator as the sole argument.

.. code-block:: python

Expand All @@ -139,9 +140,9 @@ What's New or Different
The Box-Muller method used to produce NumPy's normals is no longer available
in `Generator`. It is not possible to reproduce the exact random
values using Generator for the normal distribution or any other
distribution that relies on the normal such as the `.RandomState.gamma` or
`.RandomState.standard_t`. If you require bitwise backward compatible
streams, use `.RandomState`.
distribution that relies on the normal such as the `RandomState.gamma` or
`RandomState.standard_t`. If you require bitwise backward compatible
streams, use `RandomState`.

* The Generator's normal, exponential and gamma functions use 256-step Ziggurat
methods which are 2-10 times faster than NumPy's Box-Muller or inverse CDF
Expand All @@ -152,20 +153,20 @@ What's New or Different
* Optional ``out`` argument that allows existing arrays to be filled for
select distributions
* All BitGenerators can produce doubles, uint64s and uint32s via CTypes
(`~.PCG64.ctypes`) and CFFI (`~.PCG64.cffi`). This allows the bit generators
(`PCG64.ctypes`) and CFFI (`PCG64.cffi`). This allows the bit generators
to be used in numba.
* The bit generators can be used in downstream projects via
:ref:`Cython <random_cython>`.
* `~.Generator.integers` is now the canonical way to generate integer
* `Generator.integers` is now the canonical way to generate integer
random numbers from a discrete uniform distribution. The ``rand`` and
``randn`` methods are only available through the legacy `~.RandomState`.
``randn`` methods are only available through the legacy `RandomState`.
The ``endpoint`` keyword can be used to specify open or closed intervals.
This replaces both ``randint`` and the deprecated ``random_integers``.
* `~.Generator.random` is now the canonical way to generate floating-point
random numbers, which replaces `.RandomState.random_sample`,
`.RandomState.sample`, and `.RandomState.ranf`. This is consistent with
* `Generator.random` is now the canonical way to generate floating-point
random numbers, which replaces `RandomState.random_sample`,
`RandomState.sample`, and `RandomState.ranf`. This is consistent with
Python's `random.random`.
* All BitGenerators in numpy use `~SeedSequence` to convert seeds into
* All BitGenerators in numpy use `SeedSequence` to convert seeds into
initialized states.

See :ref:`new-or-different` for a complete list of improvements and
Expand Down Expand Up @@ -202,8 +203,9 @@ Features
c-api
Examples of using Numba, Cython, CFFI <extending>

Original Source
~~~~~~~~~~~~~~~
Original Source of the Generator and BitGenerators
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

This package was developed independently of NumPy and was integrated in version
1.17.0. The original repo is at https://github.com/bashtage/randomgen.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aren't there more bit generators in randomgen? If so, that seems worth mentioning. Otherwise, this subsection is probably best moved to THANKS.txt, it doesn't help the user.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are many more there still.


68 changes: 68 additions & 0 deletions doc/source/reference/random/legacy.rst
Original file line number Diff line number Diff line change
Expand Up @@ -121,3 +121,71 @@ Distributions
~RandomState.wald
~RandomState.weibull
~RandomState.zipf

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add the legacy functions below so they are linked into the intersphinx index

Functions in `numpy.random`
===========================
Many of the RandomState methods above are exported as functions in
`numpy.random` This usage is discouraged, as it is implemented via a gloabl
`RandomState` instance which is not advised on two counts:

- It uses global state, which means results will change as the code changes

- It uses a `RandomState` rather than the more modern `Generator`.

For backward compatible legacy reasons, we cannot change this. See
`random-quick-start`.

.. autosummary::
:toctree: generated/

beta
binomial
bytes
chisquare
choice
dirichlet
exponential
f
gamma
geometric
get_state
gumbel
hypergeometric
laplace
logistic
lognormal
logseries
multinomial
multivariate_normal
negative_binomial
noncentral_chisquare
noncentral_f
normal
pareto
permutation
poisson
power
rand
randint
randn
random
random_integers
random_sample
ranf
rayleigh
sample
seed
set_state
shuffle
standard_cauchy
standard_exponential
standard_gamma
standard_normal
standard_t
triangular
uniform
vonmises
wald
weibull
zipf

14 changes: 14 additions & 0 deletions numpy/random/__init__.pxd
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
cimport numpy as np
from libc.stdint cimport uint32_t, uint64_t

cdef extern from "numpy/random/bitgen.h":
struct bitgen:
void *state
uint64_t (*next_uint64)(void *st) nogil
uint32_t (*next_uint32)(void *st) nogil
double (*next_double)(void *st) nogil
uint64_t (*next_raw)(void *st) nogil

ctypedef bitgen bitgen_t

from numpy.random._bit_generator cimport BitGenerator, SeedSequence
2 changes: 1 addition & 1 deletion numpy/random/_bounded_integers.pxd.in
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ import numpy as np
cimport numpy as np
ctypedef np.npy_bool bool_t

from ._bit_generator cimport bitgen_t
from numpy.random cimport bitgen_t

cdef inline uint64_t _gen_mask(uint64_t max_val) nogil:
"""Mask generator for use in bounded random numbers"""
Expand Down
2 changes: 1 addition & 1 deletion numpy/random/_common.pxd
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ from libc.stdint cimport uint32_t, uint64_t, int32_t, int64_t
import numpy as np
cimport numpy as np

from ._bit_generator cimport bitgen_t
from numpy.random cimport bitgen_t

cdef double POISSON_LAM_MAX
cdef double LEGACY_POISSON_LAM_MAX
Expand Down
2 changes: 1 addition & 1 deletion numpy/random/_examples/cython/extending.pyx
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ import numpy as np
cimport numpy as np
cimport cython

from numpy.random._bit_generator cimport bitgen_t
from numpy.random cimport bitgen_t
from numpy.random import PCG64

np.import_array()
Expand Down
4 changes: 2 additions & 2 deletions numpy/random/_examples/cython/extending_distributions.pyx
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,7 @@ cimport numpy as np
cimport cython
from cpython.pycapsule cimport PyCapsule_IsValid, PyCapsule_GetPointer
from libc.stdint cimport uint16_t, uint64_t
from numpy.random._bit_generator cimport bitgen_t

from numpy.random cimport bitgen_t
from numpy.random import PCG64


Expand Down Expand Up @@ -72,3 +71,4 @@ def uint10_uniforms(Py_ssize_t n):

randoms = np.asarray(random_values)
return randoms

14 changes: 7 additions & 7 deletions numpy/random/_generator.pyx
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ from ._bounded_integers cimport (_rand_bool, _rand_int32, _rand_int64,
_rand_uint8, _gen_mask)
from ._bounded_integers import _integers_types
from ._pcg64 import PCG64
from ._bit_generator cimport bitgen_t
from numpy.random cimport bitgen_t
from ._common cimport (POISSON_LAM_MAX, CONS_POSITIVE, CONS_NONE,
CONS_NON_NEGATIVE, CONS_BOUNDED_0_1, CONS_BOUNDED_GT_0_1,
CONS_GT_1, CONS_POSITIVE_NOT_NAN, CONS_POISSON,
Expand Down Expand Up @@ -1004,19 +1004,19 @@ cdef class Generator:
A floating-point array of shape ``size`` of drawn samples, or a
single sample if ``size`` was not specified.

See Also
--------
normal :
Equivalent function with additional ``loc`` and ``scale`` arguments
for setting the mean and standard deviation.

Notes
-----
For random samples from :math:`N(\\mu, \\sigma^2)`, use one of::

mu + sigma * gen.standard_normal(size=...)
gen.normal(mu, sigma, size=...)

See Also
--------
normal :
Equivalent function with additional ``loc`` and ``scale`` arguments
for setting the mean and standard deviation.

Examples
--------
>>> rng = np.random.default_rng()
Expand Down
2 changes: 1 addition & 1 deletion numpy/random/_mt19937.pyx
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ import numpy as np
cimport numpy as np

from libc.stdint cimport uint32_t, uint64_t
from ._bit_generator cimport BitGenerator, SeedSequence
from numpy.random cimport BitGenerator, SeedSequence

__all__ = ['MT19937']

Expand Down
2 changes: 1 addition & 1 deletion numpy/random/_pcg64.pyx
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ cimport numpy as np

from libc.stdint cimport uint32_t, uint64_t
from ._common cimport uint64_to_double, wrap_int
from ._bit_generator cimport BitGenerator
from numpy.random cimport BitGenerator

__all__ = ['PCG64']

Expand Down
2 changes: 1 addition & 1 deletion numpy/random/_philox.pyx
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ cimport numpy as np

from libc.stdint cimport uint32_t, uint64_t
from ._common cimport uint64_to_double, int_to_array, wrap_int
from ._bit_generator cimport BitGenerator
from numpy.random cimport BitGenerator

__all__ = ['Philox']

Expand Down
2 changes: 1 addition & 1 deletion numpy/random/_sfc64.pyx
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ cimport numpy as np

from libc.stdint cimport uint32_t, uint64_t
from ._common cimport uint64_to_double
from ._bit_generator cimport BitGenerator
from numpy.random cimport BitGenerator

__all__ = ['SFC64']

Expand Down
Loading