Skip to content

Expose ags_gen and agt_gen in asynchronous generators #76991

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
dabeaz mannequin opened this issue Feb 9, 2018 · 13 comments
Closed

Expose ags_gen and agt_gen in asynchronous generators #76991

dabeaz mannequin opened this issue Feb 9, 2018 · 13 comments
Labels
3.13 bugs and security fixes topic-asyncio type-feature A feature request or enhancement

Comments

@dabeaz
Copy link
Mannequin

dabeaz mannequin commented Feb 9, 2018

BPO 32810
Nosy @giampaolo, @yarko, @njsmith, @asvetlov, @1st1, @xgid, @ZackerySpytz, @oremanj, @GeeTransit
PRs
  • gh-76991: Allow accessing the ags_gen and agt_gen attrs of async generators #11166
  • Files
  • agen.py
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2021-01-02.02:42:35.810>
    created_at = <Date 2018-02-09.19:49:33.075>
    labels = ['type-feature', '3.8', 'expert-asyncio']
    title = 'Expose ags_gen and agt_gen in asynchronous generators'
    updated_at = <Date 2021-01-02.02:42:35.810>
    user = 'https://bugs.python.org/dabeaz'

    bugs.python.org fields:

    activity = <Date 2021-01-02.02:42:35.810>
    actor = 'dabeaz'
    assignee = 'none'
    closed = True
    closed_date = <Date 2021-01-02.02:42:35.810>
    closer = 'dabeaz'
    components = ['asyncio']
    creation = <Date 2018-02-09.19:49:33.075>
    creator = 'dabeaz'
    dependencies = []
    files = ['47434']
    hgrepos = []
    issue_num = 32810
    keywords = ['patch']
    message_count = 7.0
    messages = ['311906', '311946', '311950', '331860', '331903', '335376', '349266']
    nosy_count = 11.0
    nosy_names = ['kdart', 'giampaolo.rodola', 'yarkot', 'njs', 'asvetlov', 'dabeaz', 'yselivanov', 'xgdomingo', 'ZackerySpytz', 'Joshua Oreman', 'GeeTransit']
    pr_nums = ['11166']
    priority = 'normal'
    resolution = None
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'enhancement'
    url = 'https://bugs.python.org/issue32810'
    versions = ['Python 3.8']

    @dabeaz
    Copy link
    Mannequin Author

    dabeaz mannequin commented Feb 9, 2018

    Libraries such as Curio and asyncio provide a debugging facility that allows someone to view the call stack of generators/coroutines. For example, the _task_get_stack() function in asyncio/base_tasks.py. This works by manually walking up the chain of coroutines (by following cr_frame and gi_frame links as appropriate).

    The only problem is that it doesn't work if control flow falls into an async generator because an "async_generator_asend" instance is encountered and there is no meaningful way to proceed any further with stack inspection.

    This problem could be fixed if "async_generator_asend" and "async_generator_athrow" instances exposed the underlying "ags_gen" and "agt_gen" attribute that's held inside the corresponding C structures in Objects/genobject.c.

    Note: I made a quick and dirty "hack" to Python to extract "ags_gen" and verified that having this information would allow me to get complete stack traces in Curio.

    @dabeaz dabeaz mannequin added 3.7 (EOL) end of life type-feature A feature request or enhancement labels Feb 9, 2018
    @asvetlov
    Copy link
    Contributor

    Make sense at first glaze.

    @dabeaz
    Copy link
    Mannequin Author

    dabeaz mannequin commented Feb 10, 2018

    I've attached a file that illustrates the issue.

    (Side thought: this would be nice to have in inspect or traceback)

    @ZackerySpytz
    Copy link
    Mannequin

    ZackerySpytz mannequin commented Dec 14, 2018

    I've created a PR for this issue.

    @ZackerySpytz ZackerySpytz mannequin added 3.8 (EOL) end of life and removed 3.7 (EOL) end of life labels Dec 14, 2018
    @1st1
    Copy link
    Member

    1st1 commented Dec 15, 2018

    Please don't merge the PR. I'll need some time to think about this before giving it a green light.

    @oremanj
    Copy link
    Mannequin

    oremanj mannequin commented Feb 12, 2019

    I also ran into this. My workaround was to use gc.get_referents() on the async_generator_asend object, which returns a one-element list containing the async generator object. I don't know if this is guaranteed or just happened to work in the cases I was using it, but it might be good enough?

    @GeeTransit
    Copy link
    Mannequin

    GeeTransit mannequin commented Aug 8, 2019

    Also, while the PEP first introducing asynchronous generators stated that its .asend() and .athrow() methods will return a "coroutine-like object", it doesn't allow any introspection into its state or parent async generator.

    My workaround was to wrap the .asend() and other methods with another coroutine that just awaits and returns the result. Having a better way to check its state and frame will bring it closer to a coroutine.

    @dabeaz dabeaz mannequin closed this as completed Jan 2, 2021
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    @VincentVanlaer
    Copy link

    This issue seems to have been closed by accident. There associated PR is still open and there is currently no clean way to access the generator. The workaround using gc.get_referents works though.

    @erlend-aasland
    Copy link
    Contributor

    cc. @1st1

    @gvanrossum
    Copy link
    Member

    @erlend-aasland What's going on here? @1st1 is unlikely to react. The PR suddenly gets action. (By you? By the OP?) Maybe you want to reopen this issue? (The last comment suggests it should be reopened.)

    @erlend-aasland
    Copy link
    Contributor

    @gvanrossum: I did an audit of all open issues and PRs by @ZackerySpytz earlier today. The linked PR (#11166, authored by Zackery) was still open, and this issue seemed to have been closed by accident, hence the ping.

    @gvanrossum
    Copy link
    Member

    Yeah, @1st1 really won't look at this, so we need someone else to do so. I'll reopen it so at least it's clear that this was closed by mistake.

    @gvanrossum gvanrossum reopened this Jan 5, 2024
    @gvanrossum
    Copy link
    Member

    I'm closing this. Six years ago @1st1 didn't like the idea of exposing these implementation details, and not too many people have requested this feature since then.

    @gvanrossum gvanrossum closed this as not planned Won't fix, can't repro, duplicate, stale Jan 10, 2024
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.13 bugs and security fixes topic-asyncio type-feature A feature request or enhancement
    Projects
    None yet
    Development

    No branches or pull requests

    6 participants