This issue is to track the implementation of PEP-499 which we hope to get into the upcoming 3.8 release. I've made it because cpython PRs want a bpo number in the subject line.

I'll link in the proposed PR once I've filed off a few grammar issues I've just noticed (documentation English grammar).

I've withdrawn the PR; I hadn't run the full test suite and there are things to fix. - Cameron

New PR 12490 attached with a fix for test_pdb. More extensive comments are in the leading comment on the PR itself. - Cameron

[Belatedly updating this issue with the current status as of March]

Cameron's implementation generally looks good, but there are couple of compatibility/migration questions that we need to consider, as spelled out in the PEP update that added me as BDFL-Delegate: https://github.com/python/peps/pull/946/files

• We need a generic porting guide entry to handle projects that turn out to have been relying on their name *not* being bound in sys.modules. For example, adding this preamble:

    if __name__ == "__main__":
        # To prevent inadvertent double imports, the -m
        # switch in Python 3.9+ defaults to aliasing __main__
        # under the executed module's import name. We actually
        # want the double import, so remove the alias if it exists
        import sys
        _main_module = sys.modules.get(__name__)
        _spec_module = sys.modules.get(__spec__.name)
        if _main_module is _spec_module:
            sys.modules.pop(__spec__.name)

We'd test the above snippet by adding it to the pdb module (and reverting the other compatibility changes to that module)

• We need to understand the implications for pickle compatibility, and provide a porting guide snippet, similar to the one above for explicitly requesting the double-import behaviour. For example:

    if __name__ == "__main__":
        # To prevent inadvertent double imports, the -m
        # switch in Python 3.9+ defaults to aliasing __main__
        # under the executed module's import name. We need
        # pickle to use the real module name for objects from
        # __main__ though, so we set the import name here
        _running_as_main = True
        __name__ = __spec__.name

I want to start with an apology. I have become a little swamped by work and didn't let anyone know I've made little time for this since March.

To my naive eye Nick's snippet looks like it would work for pdb; I became a little stalled there trying to understand how pdb managed the main module.

I totally lack the expertise to address pickle; I've never used it and the long history of feature updates sugests to me that I cannot get a deep enough understanding in a meaningful time.

Just a remark about the preamble comment: it reads to me as though PEP-499 is a misfeature. Possibly change:

We actually want the double import, so remove the alias if it exists.

to:

This module is unusual, and actually wants the double import.
Therefore we remove the __spec__.name alias if it exists
so that a subsequent import by name will make a distinct instance.

Verbose, but at least I know the intent.

FWIW, I have some feedback on the PEP. (See msg357448.) Can we discuss here or should I open a mailing list thread?

On 25Nov2019 17:38, Python Bug Reports <report@bugs.python.org> wrote:

Eric Snow <ericsnowcurrently@gmail.com> added the comment:

FWIW, I have some feedback on the PEP. (See msg357448.) Can we discuss here or should I open a mailing list thread?

Let's discuss it here unless it looks like we need wider input. This is
related to bpo-37292 (xxsubinterpreters: Can't unpickle objects
defined in __main_), yes?

With PEP-499, "python -m foo" should have __name__=='__main__' and
__spec__.name=='foo'. Were you thinking the __module__ should come from
__spec__.name (and that the PEP should make that clear)?

Exactly. :)

I'd expect PEP-499 to specify changing __module__ of classes and functions from __main__ to the module name (spec.name). This aligns closely with the whole point of the PEP. :) As a bonus, it will simplify things for pickling (which doesn't inherently deal well with __main__).

Leaving the relationship between pickle and __name__ alone wasn't an oversight, as folks already rely on that to gracefully transition from single-file modules to multi-file packages without breaking pickle compatibility in either direction. The trick is to do a "from ._submodule import *" in the package's __init__.py (exposing all the names), and then a "__name__ = __package__" in the submodule itself.

Thus the second snippet above, as a way to port code that was specifically relying on the double import to provide pickle compatibility without risking introducing other unintended incompatibility problems.

Suggesting to close this together with the linked PR, as PEP-499 is still not accepted. The issue and the patch can be revived if PEP-499 is accepted.