MAINT: add test to prevent new public-looking modules being added #14454

rgommers · 2019-09-08T07:58:53Z

There is more work to do (added a FIXME), but that's for another time. Preventing to add new submodules is a good start. Also added some documentation, and split things into public and private submodules with comments.

The change to numpy.lib.mixins.__all__ shows that people have been working around our import * mess sometimes. Not adding some public object to an __all__ dict is incorrect of course - need a more structural fix by getting rid of * imports.

rgommers · 2019-09-08T08:25:13Z

Hmm, we start with a failure that I can't reproduce locally:

>           raise AssertionError("Found unexpected modules: {}".format(modnames))
E           AssertionError: Found unexpected modules: ['numpy.core.generate_numpy_api']

shows up for an install, but not an in-place build. adding it to the list

eric-wieser · 2019-09-09T03:57:06Z

The change to numpy.lib.mixins.all shows that people have been working around our import * mess sometimes.

Can you elaborate on what you mean by this?

numpy/tests/test_public_api.py

rgommers · 2019-09-09T23:14:43Z

The change to numpy.lib.mixins.all shows that people have been working around our import * mess sometimes.

Can you elaborate on what you mean by this?

The comment explicitly said not to add NDArrayOperatorsMixin because it would then end up in the main numpy namespace. While that goal is good, it's incorrect to have a public function not in the __all__ dict in the file it's in. Also, this effectively has made numpy.lib.mixins public, because the mixin didn't end up in numpy.lib even though there's a from .mixins import * in numpy/lib/__init__.py; I'm not sure what the intent was there.

The root cause is that in our main __init__.py there's a from .lib import *, which makes little sense.

numpy/tests/test_public_api.py

…-ness

Also finish the TODO about figuring out which np.lib.<submodule>'s are public. This is a giant mess ...

[ci skip]

Also address other review comments.

…mpygh-14498

Also remove `numpy.ma.version.py`, it was not importable and served no purpose.

mattip · 2019-09-20T06:02:22Z

Thanks @rgommers. I guess the next step is to identify candidates for namespace deprecation.

rgommers · 2019-09-20T08:33:52Z

I guess the next step is to identify candidates for namespace deprecation.

Before that, lower-hanging fruit may be to remove info.py files, add some __all__ dicts, and clean up existing namespaces (e.g. removing numpy.math really shouldn't require a deprecation).

charris · 2019-09-21T12:31:24Z

numpy/tests/test_public_api.py

+    "random.philox",
+    "random.sfc64",
+    "testing.decorators",
+    "testing.noseclasses",


This broke the wheel builds, I suspect because nose is not present in those builds. Nose doesn't support recent Python3 out of the box.

do you have a link?

nose is optional, and if not present an import error will be raised if a attempt to import it is made.

https://travis-ci.org/MacPython/numpy-wheels/builds/587676702?utm_medium=notification&utm_source=email

thanks. okay that's easy to fix, PR to follow

Removes these files that were deprecated since 1.15.0: - numpy/testing/decorators.py - numpy/testing/noseclasses.py - numpy/testing/nosetester.py This also resolves a failure in the recently introduced tests in `test_public_api.py` (see numpygh-14454). Closes numpygh-14566

rgommers added 05 - Testing 30 - API 62 - Python API Changes or additions to the Python API. Mailing list should usually be notified. labels Sep 8, 2019

rgommers changed the title ~~WIP: TST: add test to prevent new public-looking modules being added~~ MAINT: add test to prevent new public-looking modules being added Sep 8, 2019

rgommers added this to the 1.18.0 release milestone Sep 8, 2019

eric-wieser reviewed Sep 9, 2019

View reviewed changes