Skip to content

DOC: Generator is documented as part of the random c-api, but is missing #16597

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
madarez opened this issue Jun 13, 2020 · 8 comments · Fixed by #16731
Closed

DOC: Generator is documented as part of the random c-api, but is missing #16597

madarez opened this issue Jun 13, 2020 · 8 comments · Fixed by #16731

Comments

@madarez
Copy link

madarez commented Jun 13, 2020

xref gh-14778

This is an issue I noticed following the series of tickets on the random c-api usability. Quoting from the docs on Cython API:

Typed versions of many of the Generator and BitGenerator methods as well as the classes themselves can be accessed directly from Cython via
cimport numpy.random

I could verify cimport-ing BitGenerator in Cython, but on the Generator defined in numpy/random/_generator.pyx, it seems that it is not exposed which could be nice if we declare it in random/__init__.pxd.

Reproducing code example:

%%cython
from numpy.random cimport BitGenerator
from numpy.random cimport Generator

Error message:

Error compiling Cython file:
------------------------------------------------------------
...
from numpy.random cimport BitGenerator
from numpy.random cimport Generator
^
------------------------------------------------------------

/tmp/.ipython/cython/_cython_magic_31304ad03a3c08886c6fdc45ff9df750.pyx:2:0: 'numpy/random/Generator.pxd' not found

Numpy/Python version information:

1.18.1 3.6.10 |Anaconda, Inc.| (default, Jan  7 2020, 15:01:53) 
[GCC 4.2.1 Compatible Clang 4.0.1 (tags/RELEASE_401/final)]
@charris
Copy link
Member

charris commented Jun 14, 2020

I believe it is exposed in the upcoming 1.19. If you want to try there are wheels for the rc2 release on PyPI.

@madarez
Copy link
Author

madarez commented Jun 15, 2020

Thanks for your quick reply. I just tried the above cimport statements with numpy 1.18.5, 1.19.0rc1, and 1.19.0rc2 and received 'numpy/random/Generator.pxd' not found error message. I could verify with the content of numpy/random/__init__.pxd exposes BitGenerator but not Generator (last line).

Numpy/Python version information for the most recent candidate:

1.19.0rc2 3.7.7 (default, May  6 2020, 04:59:01) 
[Clang 4.0.1 (tags/RELEASE_401/final)]

@mattip
Copy link
Member

mattip commented Jun 15, 2020

I think that is a documentation mistake. The functions mentioned on the page you reference come from c_distributions.pxd so the line should read something like

Typed versions of many of the SeedSequence and BitGenerator methods and classes and the c_distribution functions ...

I am curious about the use case for the Generator class from Cython,

@madarez
Copy link
Author

madarez commented Jun 15, 2020

Yes, that absolutely makes sense.

My understanding is that if one wants to use Generators they could use their Python objects, and it's absolutely valid to use default_rng(). However, it could appeal to certain Cython developers to either type name their Generator or to compile their Generator (via cimport-ing) to quickly access higher level methods out of the box. I'd say typing and the compilation of default_rng() are the two cases I could put my finger on.

@zoj613
Copy link
Contributor

zoj613 commented Jun 16, 2020

Yes, that absolutely makes sense.

My understanding is that if one wants to use Generators they could use their Python objects, and it's absolutely valid to use default_rng(). However, it could appeal to certain Cython developers to either type name their Generator or to compile their Generator (via cimport-ing) to quickly access higher level methods out of the box. I'd say typing and the compilation of default_rng() are the two cases I could put my finger on.

I second this

@mattip
Copy link
Member

mattip commented Jun 16, 2020

We are trying to be conservative in adding to the sprawling NumPy API. If a project comes along with a real-world use case we can discuss exposing more, but as of now I think we should turn this into a documentation issue.

@mattip mattip changed the title Exposing Generator along with the new changes on the random c-api DOC: Generator is documented as part of the random c-api, but is missing Jun 16, 2020
@bashtage
Copy link
Contributor

bashtage commented Jul 1, 2020

Generator doesn't expose anything that is usable in lower-level cython. There is no point in making it part of the C-api.

The components are all there as of 1.19 so you can mix pretty much any custom variant generators you need.

@bashtage
Copy link
Contributor

bashtage commented Jul 1, 2020

My understanding is that if one wants to use Generators they could use their Python objects, and it's absolutely valid to use default_rng(). However, it could appeal to certain Cython developers to either type name their Generator or to compile their Generator (via cimport-ing) to quickly access higher level methods out of the box. I'd say typing and the compilation of default_rng() are the two cases I could put my finger on.

Every function from Generator is def not cpdef and there aren't really any interesting c-level attributes available that you can't get from the Python Capsule that is exported from any bit generator.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants