I was just wondering: what should be done for the cases where the docs refer to the externals, such as when we talk about model persistence? #12171 reminded me of this.

Should there be public methods in sklearn somewhere, which use then _externals for the case we want to expose some public functionality like model persistence using joblib?

I agree with the idea of making externals private, but we tried to rename joblib to _joblib and failed (see e.g. #11408 (comment)). The same applies to moving externals to _externals. I'm not saying that's not possible, maybe it's easier with Py3 only, but it would probably involve messing with the import mechanism: i.e. making,

from sklearn.externals.some_module import something

work (with a deprecation warning) without sklearn/externals/some_module.py existing on the file system, after it was moved to sklearn/_externals/
This can result in other unexpected issues and I don't think it's worth it.

In practice, arff and pilutils are already put in private modules within externals so what remains is joblib and six, both of which are going to be removed. Having a sklearn.externals that only contains private modules would be a bit equivalent in the end.

The question is how to remove joblib and six,

• for six it was proposed in API Deprecate externals.six #12916 (cc @qinhanmin2014 ) let's move that discussion there.
• for joblib, once we stop using the vendored version, putting a deprecation warning in joblib.__init__, making sure it's not collected by pytest and waiting for a few versions before removal could be enough?

I think this was mostly solved. joblib is removed, and most other modules except for six are private,

ls sklearn/externals/
README  __init__.py  __pycache__  _arff.py  _pilutil.py  conftest.py  joblib  setup.py  six.py

we could rename the externals folder in a few versions as well (for now the only blocker is that sklearn.externals.joblib alias needs to exist to avoid breaking pickles).

@rth I'm happy to close this, what do you think?

Sounds good.