Skip to content

Add support for IMAP IDLE in imaplib #55454

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
ShayRojansky mannequin opened this issue Feb 18, 2011 · 33 comments
Closed

Add support for IMAP IDLE in imaplib #55454

ShayRojansky mannequin opened this issue Feb 18, 2011 · 33 comments
Labels
stdlib Python modules in the Lib dir topic-email type-feature A feature request or enhancement

Comments

@ShayRojansky
Copy link
Mannequin

ShayRojansky mannequin commented Feb 18, 2011

BPO 11245
Nosy @warsaw, @ericvsmith, @bitdancer, @vadmium, @mitya57, @soltysh, @ankostis, @jdek
Dependencies
  • bpo-18921: In imaplib, cached capabilities may be out of date after login
  • Files
  • imapidle.patch
  • imapidle.patch: New patch
  • 0001-Add-IDLE-support-to-imaplib.patch: IMAP IDLE patch for imaplib
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = None
    created_at = <Date 2011-02-18.20:36:10.487>
    labels = ['type-feature', 'library', 'expert-email', '3.9']
    title = 'Implementation of IMAP IDLE in imaplib?'
    updated_at = <Date 2019-12-19.12:58:09.754>
    user = 'https://bugs.python.org/ShayRojansky'

    bugs.python.org fields:

    activity = <Date 2019-12-19.12:58:09.754>
    actor = 'jdek'
    assignee = 'none'
    closed = False
    closed_date = None
    closer = None
    components = ['Library (Lib)', 'email']
    creation = <Date 2011-02-18.20:36:10.487>
    creator = 'Shay.Rojansky'
    dependencies = ['18921']
    files = ['27400', '37555', '48790']
    hgrepos = []
    issue_num = 11245
    keywords = ['patch']
    message_count = 24.0
    messages = ['128814', '129405', '129407', '171885', '171889', '172365', '172383', '202149', '202342', '202345', '233176', '235972', '236167', '245204', '245246', '245249', '245252', '245253', '245255', '245284', '246235', '246236', '293222', '358678']
    nosy_count = 17.0
    nosy_names = ['barry', 'pierslauder', 'eric.smith', 'piers', 'r.david.murray', 'Shay.Rojansky', 'martin.panter', 'mitya57', 'maciej.szulik', 'nafur', 'dveeden', 'Malina', 'F.Malina', 'ankostis', 'equaeghe', 'ohreally', 'jdek']
    pr_nums = []
    priority = 'normal'
    resolution = None
    stage = 'test needed'
    status = 'open'
    superseder = None
    type = 'enhancement'
    url = 'https://bugs.python.org/issue11245'
    versions = ['Python 3.9']

    Linked PRs

    @ShayRojansky
    Copy link
    Mannequin Author

    ShayRojansky mannequin commented Feb 18, 2011

    IMAP IDLE support is not implemented in the current imaplib. A "drop-in" replacement called imaplib2 exists (), but uses internally managed threads - a heavy solution that is not always appropriate (e.g. when handling many IMAP accounts an asynchronous approach would be more efficient)

    I am about to start implementation of an asynchronous select()-compatible approach, and was wondering if there has been any discussion over IDLE, any specific reasons it hasn't been implemented and if eventual integration into imaplib would be a desirable thing.

    Proposed approach:

    • Addition of a new state 'IDLE'
    • Addition of an idle() method to class IMAP4, which issues the IDLE command to the server and returns immediately. At this point we enter the IDLE state, in which no normal IMAP commands may be issued.
    • Users can now select() or poll() the socket as they wish
    • A method can be called to retrieve any untagged responses (e.g. EXISTS) that have arrived since entering the IDLE state. The function returns immediately and does not modify the state.
    • To end the IDLE state, the user calls a method (done()?) which resumes the previous state.

    Would appreciate any sort of feedback...

    @ShayRojansky ShayRojansky mannequin added stdlib Python modules in the Lib dir type-feature A feature request or enhancement labels Feb 18, 2011
    @terryjreedy
    Copy link
    Member

    imaplib has no particular maintainer and I know little about it.
    Doc says it implements 'a large subset of the IMAP4rev1 client protocol as defined in RFC 2060." I do not remember any discussion on pydev, over the last several years, about imaplib. I presume just the subset was chosen because of some combination of necessity and feasibility, as judged by the implementors. Hence the complement, the unimplemented subset, would be 'not done' rather than 'not wanted'. If your proposed new feature, an IDLE command, is part of this complement, then I would assume that a patch would, in principle, be acceptable.

    I cannot comment on your particular proposal, but I hope the above helps as far as it goes.

    @bitdancer
    Copy link
    Member

    I just wound up doing a bit of research on this for other reasons. Piers Lauder was the original author of the imaplib module, and he is (as far as I can tell) currently maintaining an imaplib2 module that does support IDLE (but not, I think, python3). But it does IDLE (and other things) via threads, and in the email I found announcing it he didn't think it was suitable for stdlib inclusion (because of the threading). Piers hasn't contributed to core in quite a while as far as I can tell, but he was active in a bug report back in 2008 according to google, so I thought I'd add him to nosy and see if he has time for an opinion.

    @nafur
    Copy link
    Mannequin

    nafur mannequin commented Oct 3, 2012

    We have implemented this functionality according to RFC 2177.
    We actually implemented a synchronous idle function that blocks until a timeout occurs or the server sent some event.

    This is not the most flexible way, however it will provide a basic functionality that enables user to use imap idle based notifications.
    Besides, every other solution would require threads or regular polling.

    See attached patch file.

    @bitdancer
    Copy link
    Member

    To fully answer the original question that opened this issue: contributions will be welcomed. While I don't currently have time to work on imaplib myself, I have an interest and will review and if appropriate commit patches.

    I like Shay's proposal, but absent a patch along those lines having blocking IMAP support will definitely be an improvement. An application needing to monitor more than one imap connection could do its own threading.

    Thanks for proposing the patch. Could you please submit a contributor agreement? I probably won't have time to fully consider the proposed patch for a bit, but I've put it on my todo list.

    test_imaplib does have a testing framework now, do you think you could write tests for the new feature?

    @nafur
    Copy link
    Mannequin

    nafur mannequin commented Oct 8, 2012

    I got the confirmation for my agreement.

    I'm not quite sure about the tests, as I'm not really familiar with the way this is done in cpython. The test_imaplib.py seems to cover all ways to connect to some server, but none of the actual imap commands. The patch only implements another commands (whose behaviour is highly/only dependent of other events on the server).

    Hence, I don't see a way to create a meaningfull test case other than just calling the command...

    @bitdancer
    Copy link
    Member

    Yeah writing a good test case for this is a bit tricky, since we'll need some infrastructure (an Event?) so we can prove that the call has blocked without delaying the test for very long.

    I'm not sure when I'll be able to get to this...I'm not going to check it in without a test, so I'll have to find enough time to be able to write the tests.

    @nafur
    Copy link
    Mannequin

    nafur mannequin commented Nov 4, 2013

    I stumbled about this issue again and would really like to see it fixed.

    I see the possibility to create a test case in combination with the first test sequence which creates a temporary mail. Would it be enough, that we just call IDLE in some folder, create a temporary mail in this folder and check if it returns?

    Unfortuantely, I have not been able to write code for such a test case yet, as the whole test routine fails with "[PRIVACYREQUIRED] Plaintext authentication disallowed on non-secure (SSL/TLS) connections". This is using 3.2.3, but I guess it will not be any different with the current release... (as it is the same with 2.7.3)

    @bitdancer
    Copy link
    Member

    What do you mean by the whole test routine failing? The test suite is currently passing on the buildbots, so are you speaking of the new test you are trying to write?

    @bitdancer
    Copy link
    Member

    Hmm. Looking at this again, it appears as though there's no way to interrupt IDLE if you want to, say, send an email. If you are actually using this in code, how are you handling that situation?

    @nafur
    Copy link
    Mannequin

    nafur mannequin commented Dec 29, 2014

    So, let's resurrect this one.

    For the project that lead to the old patch, we did not need this feature.
    However, we now needed are more complete implementation of IDLE.
    Hence, we extended this to return after sending idle() and support polling, leaving idle mode or wait until something happens (like before).

    @Malina
    Copy link
    Mannequin

    Malina mannequin commented Feb 14, 2015

    IMAP polling hurts, just merge imaplib2 into standard library as imaplib.

    Piers Lauder authored imaplib IMAP4 client, part of python standard library, back in December 1997 based on RFC 2060. In 2003 RFC 2060 was made obsolete by RFC 3501 adding important features and Piers released imaplib2 which receives feature updates since.
    Last feature updates to the standard library imaplib were before Piers retired from Sydney University a decade ago.

    imaplib2 presents an almost identical API as that provided by the standard library imaplib, the main difference being that imaplib2 allows parallel execution of commands on the IMAP4 server, and implements the IDLE extension, so NO POLLING IS REQUIRED. IMAP server will push new mail notifications to the client. Imaplib2 also supports COMPRESS, ID, better timeout handling etc. There is 975 more lines of code all doing useful things a modern IMAP client needs.

    imaplib2 can be substituted for imaplib in existing clients with no changes in the code apart from required logout call to shutdown the threads.

    Old imaplib was ported to Python 3 with the rest of the standard library. I am working to port imaplib2 to py3, stuck on receiving bytes v strings.

    References:

    imaplib2 code and docs
    http://sourceforge.net/p/imaplib2/code/ci/master/tree/
    also http://sydney.edu.au/engineering/it/~piers/python/imaplib2.html

    imaplib
    https://hg.python.org/cpython/file/3.4/Lib/imaplib.py

    Ruby stdlib support for idle (not that it hurts python performance, just my pride)
    http://ruby-doc.org/stdlib-2.0.0/libdoc/net/imap/rdoc/Net/IMAP.html#method-i-idle

    @Malina
    Copy link
    Mannequin

    Malina mannequin commented Feb 18, 2015

    Imaplib2 now supports Python 3. Piers and me propose to merge imaplib2 into standard library as imaplib.

    Excerpt from our conversation:

    Piers: ...Thanks for bringing it (this thread) to my attention. I entirely agree with your comments.

    Me: ...I found the criticism of the "threads - a heavy solution"? counterproductive. Not that I know anything about threads...

    Piers: I'm not sure what the whole anti-threads thing was about all those years ago since I always loved using them. Maybe early implementations were
    slow, or, more likely, early adopters were clumsy ("giving threads to
    a novice is like giving a blow torch to a baby" to paraphrade a quote :-)

    @bitdancer
    Copy link
    Member

    Are you volunteering to be maintainer, and/or is Piers? If he's changed his mind about the threading, that's good enough for me (and by now he has a lot more experience with the library in actual use).

    The biggest barrier to inclusion, IMO, is tests and backward compatibility. There have been enough changes that making sure we don't break backward compatibility will be important, and almost certainly requires more tests than we have now. Does imaplib2 have a test suite?

    We would need to get approval from python-dev, though. We have ongoing problems with packages that are maintained outside the stdlib...but updating to imaplib2 may be better than leaving it without a maintainer at all.

    Can we get Piers involved in this conversation directly?

    @FMalina
    Copy link
    Mannequin

    FMalina mannequin commented Jun 12, 2015

    I am in for my part and I emailed Piers to come and join us and he surely
    will when the bug tracker is responsive again.

    Imaplib2 does have an up to date test suite and compatibility wise imaplib2
    can be substituted for imaplib in existing clients with no changes in the
    code.

    On top of that I have a private IDLE test suite for common tasks such as

    • instant bounce processing,
    • email photo upload to web service,
    • unsubscribe processing via list-unsubscribe headers and feedback loops of
      major email providers.

    I am looking to make it part of an external project
    https://github.com/fmalina/emails, but need to extract much of the recipes
    first out of a working application in a reusable manner as I need it in
    other projects and will do for years to come.

    @bitdancer
    Copy link
    Member

    This is great.

    When you say it is fully compatible, though, is that testing against imaplib in python2 or python3? It is the python3 decisions about string/bytes handling where the discrepancies are most likely to arise, unless the python3 port was modeled on the stdlib version.

    @FMalina
    Copy link
    Mannequin

    FMalina mannequin commented Jun 12, 2015

    I just wen’t through my repo looking at relevant commits to double check and I didn’t have to change a line in my user level code when upgrading from python2 to 3. There was only one way to do it.

    @bitdancer
    Copy link
    Member

    Do you have any tests that use non-ascii passwords? I think that was the most significant bug.

    @FMalina
    Copy link
    Mannequin

    FMalina mannequin commented Jun 12, 2015

    I don’t have a test for it, neither has stdlib imaplib. We just need to port over the encode fix.

    Copy over the fixed version of _CRAM_MD5_AUTH.
    from line 599 in python3.5 imaplib
    https://github.com/python/cpython/blob/master/Lib/imaplib.py#L599 <https://github.com/python/cpython/blob/master/Lib/imaplib.py#L599\>
    corresponding to line 884 in imaplib2
    https://github.com/bcoe/imaplib2/blob/master/imaplib2/imaplib2.py#L884 <https://github.com/bcoe/imaplib2/blob/master/imaplib2/imaplib2.py#L884\>

    @piers
    Copy link
    Mannequin

    piers mannequin commented Jun 13, 2015

    Hi, apologies for not responding to the "pierslauder" pings, but i don't own that login, or at least have forgotten all about it, and its email address is invalid (or there is another pierslauder out there).

    I maintain imaplib2 on sourceforge (as piersrlauder) at https://sourceforge.net/projects/imaplib2/
    and that version has just been modified to incorporate the CRAM_MD5_AUTH change from python3.6. It is regularly updated with bug fixes and it also has built-in tests for the IDLE function.

    I originally intended for imaplib2 to be incorporated into pythonlib, leaving the original module in place (a la urllib/2). Then people wouldn't be forced into a switch using threads except by choice.

    Anyway, happy to help.

    @bitdancer
    Copy link
    Member

    Thanks, Piers!

    Sorry for dropping off the map on this, I've been busy.

    I'll post to python-dev about this and see how the community would like to proceed.

    @bitdancer
    Copy link
    Member

    By the way, the pierslauder id points to 'pierslauder@users.sourceforge.net'.

    @ankostis
    Copy link
    Mannequin

    ankostis mannequin commented May 8, 2017

    Before merging imaplib2 please consider making proper use of the Python's standard logging module.

    @jdek
    Copy link
    Mannequin

    jdek mannequin commented Dec 19, 2019

    Hi, I'm new to python but I had a go at implementing this for imaplib(1) using a different approach. It works but it has a couple issues (see patch), I would appreciate any thoughts/improvements.

    @jdek jdek mannequin added the 3.9 only security fixes label Dec 19, 2019
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    @trentbuck
    Copy link

    FYI,

    Here is a bare-minimum version that Works For Me (so far) with python 3.9 and dovecot 1.2.9 (don't ask).
    This is meant to replace a crappy cron job * * * * * nobody curl https://⋯/receiveEmail.php >/dev/null.
    I'm think sure this will eventually OOM, as I never explicitly reap the "still here" continuation responses.

    #!/usr/bin/python3 
    
    import imaplib
    import logging
    
    
    # Enable IDLE support. 
    if 'IDLE' in imaplib.Commands:
    
        # in the unlikely event this feature is fixed upstream... 
        IMAP4_SSL = imaplib.IMAP4_SSL
    
    else:
    
        class IMAP4_SSL_plus_IDLE(imaplib.IMAP4_SSL):
    
            def idle(self):
                if 'IDLE' not in self.capabilities:
                    raise self.error('Server does not support IDLE')
                idle_tag = self._command('IDLE')  # start idling 
                self._get_response()
                while line := self._get_line():
                    if line.endswith(b'EXISTS'):
                        self.send(b'DONE' + imaplib.CRLF)
                        return self._command_complete('IDLE', idle_tag)
    
        imaplib.Commands['IDLE'] = ('AUTH', 'SELECTED')
        IMAP4_SSL = IMAP4_SSL_plus_IDLE
    
    
    with IMAP4_SSL(⋯) as conn:
        conn.login(user=⋯, password=⋯)
        conn.select(mailbox='INBOX', readonly=True)
        conn.debug = 100            # DEBUGGING 
        while True:
            resp = requests.get('https://⋯/receiveEmail.php')
            resp.raise_for_status()
            logging.debug('HTTP GET said %s', resp.text)
            resp = conn.idle()
            logging.debug('IMAP IDLE said %s', resp)
    

    @foresto
    Copy link
    Contributor

    foresto commented Jul 26, 2024

    foresto added a commit to foresto/cpython that referenced this issue Jul 31, 2024
    This extends imaplib with support for the rfc2177 IMAP IDLE command,
    enabling events to be pushed to a client as they occur, rather than
    having to continually poll for mailbox changes.
    
    The interface is a new idle() method, which returns an iterable context
    manager.  Entering the context starts IDLE mode, during which events
    (untagged responses) can be retrieved using the iteration protocol.
    Exiting the context sends DONE to the server, ending IDLE mode.
    
    An optional time limit for the IDLE session is supported, for use with
    servers that impose an inactivity timeout.
    
    The context object also offers a burst() method, designed for programs
    wishing to process events in batch rather than one at a time.
    
    Notable differences from other implementations:
    
    - It's an extension to imaplib, rather than a replacement.
    - It doesn't introduce additional threads.
    - It doesn't impose new requirements on the use of imaplib's existing methods.
    - It passes the unit tests in CPython's test/test_imaplib.py module.
    - It works on Windows, Linux, and other unix-like systems.
    - It makes IDLE available on all of imaplib's client variants
      (including IMAP4_stream).
    - The interface is pythonic and easy to use.
    
    Caveats:
    
    - Due to a Windows limitation, the special case of IMAP4_stream running
      on Windows lacks a duraton/timeout feature. (This is the stdin/stdout
      pipe connection variant; timeouts work fine for socket-based
      connections, even on Windows.) I have documented it where appropriate.
    
    - The file-like imaplib instance attributes are chagned from buffered to
      unbuffered mode. This could potentially break any client code that
      uses those objects directly without expecting partial reads/writes.
      However, these attributes are undocumented. As such, I think (and
      PEP 8 confirms) that they are fair game for changes.
      https://peps.python.org/pep-0008/#public-and-internal-interfaces
    
    Usage examples:
    
    python#55454 (comment)
    
    Original discusion:
    
    https://discuss.python.org/t/gauging-interest-in-my-imap4-idle-implementation-for-imaplib/59272
    
    Earlier requests and suggestions:
    
    https://bugs.python.org/issue11245
    
    python#55454
    
    https://mail.python.org/archives/list/python-ideas@python.org/thread/C4TVEYL5IBESQQPPS5GBR7WFBXCLQMZ2/
    foresto added a commit to foresto/cpython that referenced this issue Jul 31, 2024
    This extends imaplib with support for the rfc2177 IMAP IDLE command,
    enabling events to be pushed to a client as they occur, rather than
    having to continually poll for mailbox changes.
    
    The interface is a new idle() method, which returns an iterable context
    manager.  Entering the context starts IDLE mode, during which events
    (untagged responses) can be retrieved using the iteration protocol.
    Exiting the context sends DONE to the server, ending IDLE mode.
    
    An optional time limit for the IDLE session is supported, for use with
    servers that impose an inactivity timeout.
    
    The context object also offers a burst() method, designed for programs
    wishing to process events in batch rather than one at a time.
    
    Notable differences from other implementations:
    
    - It's an extension to imaplib, rather than a replacement.
    - It doesn't introduce additional threads.
    - It doesn't impose new requirements on the use of imaplib's existing methods.
    - It passes the unit tests in CPython's test/test_imaplib.py module.
    - It works on Windows, Linux, and other unix-like systems.
    - It makes IDLE available on all of imaplib's client variants
      (including IMAP4_stream).
    - The interface is pythonic and easy to use.
    
    Caveats:
    
    - Due to a Windows limitation, the special case of IMAP4_stream running
      on Windows lacks a duraton/timeout feature. (This is the stdin/stdout
      pipe connection variant; timeouts work fine for socket-based
      connections, even on Windows.) I have documented it where appropriate.
    
    - The file-like imaplib instance attributes are chagned from buffered to
      unbuffered mode. This could potentially break any client code that
      uses those objects directly without expecting partial reads/writes.
      However, these attributes are undocumented. As such, I think (and
      PEP 8 confirms) that they are fair game for changes.
      https://peps.python.org/pep-0008/#public-and-internal-interfaces
    
    Usage examples:
    
    python#55454 (comment)
    
    Original discusion:
    
    https://discuss.python.org/t/gauging-interest-in-my-imap4-idle-implementation-for-imaplib/59272
    
    Earlier requests and suggestions:
    
    https://bugs.python.org/issue11245
    
    python#55454
    
    https://mail.python.org/archives/list/python-ideas@python.org/thread/C4TVEYL5IBESQQPPS5GBR7WFBXCLQMZ2/
    foresto added a commit to foresto/cpython that referenced this issue Aug 1, 2024
    This extends imaplib with support for the rfc2177 IMAP IDLE command,
    enabling events to be pushed to a client as they occur, rather than
    having to continually poll for mailbox changes.
    
    The interface is a new idle() method, which returns an iterable context
    manager.  Entering the context starts IDLE mode, during which events
    (untagged responses) can be retrieved using the iteration protocol.
    Exiting the context sends DONE to the server, ending IDLE mode.
    
    An optional time limit for the IDLE session is supported, for use with
    servers that impose an inactivity timeout.
    
    The context object also offers a burst() method, designed for programs
    wishing to process events in batch rather than one at a time.
    
    Notable differences from other implementations:
    
    - It's an extension to imaplib, rather than a replacement.
    - It doesn't introduce additional threads.
    - It doesn't impose new requirements on the use of imaplib's existing methods.
    - It passes the unit tests in CPython's test/test_imaplib.py module.
    - It works on Windows, Linux, and other unix-like systems.
    - It makes IDLE available on all of imaplib's client variants
      (including IMAP4_stream).
    - The interface is pythonic and easy to use.
    
    Caveats:
    
    - Due to a Windows limitation, the special case of IMAP4_stream running
      on Windows lacks a duraton/timeout feature. (This is the stdin/stdout
      pipe connection variant; timeouts work fine for socket-based
      connections, even on Windows.) I have documented it where appropriate.
    
    - The file-like imaplib instance attributes are chagned from buffered to
      unbuffered mode. This could potentially break any client code that
      uses those objects directly without expecting partial reads/writes.
      However, these attributes are undocumented. As such, I think (and
      PEP 8 confirms) that they are fair game for changes.
      https://peps.python.org/pep-0008/#public-and-internal-interfaces
    
    Usage examples:
    
    python#55454 (comment)
    
    Original discusion:
    
    https://discuss.python.org/t/gauging-interest-in-my-imap4-idle-implementation-for-imaplib/59272
    
    Earlier requests and suggestions:
    
    https://bugs.python.org/issue11245
    
    python#55454
    
    https://mail.python.org/archives/list/python-ideas@python.org/thread/C4TVEYL5IBESQQPPS5GBR7WFBXCLQMZ2/
    foresto added a commit to foresto/cpython that referenced this issue Aug 1, 2024
    This extends imaplib with support for the rfc2177 IMAP IDLE command,
    enabling events to be pushed to a client as they occur, rather than
    having to continually poll for mailbox changes.
    
    The interface is a new idle() method, which returns an iterable context
    manager.  Entering the context starts IDLE mode, during which events
    (untagged responses) can be retrieved using the iteration protocol.
    Exiting the context sends DONE to the server, ending IDLE mode.
    
    An optional time limit for the IDLE session is supported, for use with
    servers that impose an inactivity timeout.
    
    The context object also offers a burst() method, designed for programs
    wishing to process events in batch rather than one at a time.
    
    Notable differences from other implementations:
    
    - It's an extension to imaplib, rather than a replacement.
    - It doesn't introduce additional threads.
    - It doesn't impose new requirements on the use of imaplib's existing methods.
    - It passes the unit tests in CPython's test/test_imaplib.py module.
    - It works on Windows, Linux, and other unix-like systems.
    - It makes IDLE available on all of imaplib's client variants
      (including IMAP4_stream).
    - The interface is pythonic and easy to use.
    
    Caveats:
    
    - Due to a Windows limitation, the special case of IMAP4_stream running
      on Windows lacks a duraton/timeout feature. (This is the stdin/stdout
      pipe connection variant; timeouts work fine for socket-based
      connections, even on Windows.) I have documented it where appropriate.
    
    - The file-like imaplib instance attributes are chagned from buffered to
      unbuffered mode. This could potentially break any client code that
      uses those objects directly without expecting partial reads/writes.
      However, these attributes are undocumented. As such, I think (and
      PEP 8 confirms) that they are fair game for changes.
      https://peps.python.org/pep-0008/#public-and-internal-interfaces
    
    Usage examples:
    
    python#55454 (comment)
    
    Original discusion:
    
    https://discuss.python.org/t/gauging-interest-in-my-imap4-idle-implementation-for-imaplib/59272
    
    Earlier requests and suggestions:
    
    https://bugs.python.org/issue11245
    
    python#55454
    
    https://mail.python.org/archives/list/python-ideas@python.org/thread/C4TVEYL5IBESQQPPS5GBR7WFBXCLQMZ2/
    foresto added a commit to foresto/cpython that referenced this issue Aug 1, 2024
    This extends imaplib with support for the rfc2177 IMAP IDLE command,
    as requested in python#55454.  It allows events to be pushed to a client as
    they occur, rather than having to continually poll for mailbox changes.
    
    The interface is a new idle() method, which returns an iterable context
    manager.  Entering the context starts IDLE mode, during which events
    (untagged responses) can be retrieved using the iteration protocol.
    Exiting the context sends DONE to the server, ending IDLE mode.
    
    An optional time limit for the IDLE session is supported, for use with
    servers that impose an inactivity timeout.
    
    The context manager also offers a burst() method, designed for programs
    wishing to process events in batch rather than one at a time.
    
    Notable differences from other implementations:
    
    - It's an extension to imaplib, rather than a replacement.
    - It doesn't introduce additional threads.
    - It doesn't impose new requirements on the use of imaplib's existing methods.
    - It passes the unit tests in CPython's test/test_imaplib.py module.
    - It works on Windows, Linux, and other unix-like systems.
    - It makes IDLE available on all of imaplib's client variants
      (including IMAP4_stream).
    - The interface is pythonic and easy to use.
    
    Caveats:
    
    - Due to a Windows limitation, the special case of IMAP4_stream running
      on Windows lacks a duraton/timeout feature. (This is the stdin/stdout
      pipe connection variant; timeouts work fine for socket-based
      connections, even on Windows.) I have documented it where appropriate.
    
    - The file-like imaplib instance attributes are chagned from buffered to
      unbuffered mode. This could potentially break any client code that
      uses those objects directly without expecting partial reads/writes.
      However, these attributes are undocumented. As such, I think (and
      PEP 8 confirms) that they are fair game for changes.
      https://peps.python.org/pep-0008/#public-and-internal-interfaces
    
    Usage examples:
    
    python#55454 (comment)
    
    Original discusion:
    
    https://discuss.python.org/t/gauging-interest-in-my-imap4-idle-implementation-for-imaplib/59272
    
    Earlier requests and suggestions:
    
    https://bugs.python.org/issue11245
    
    python#55454
    
    https://mail.python.org/archives/list/python-ideas@python.org/thread/C4TVEYL5IBESQQPPS5GBR7WFBXCLQMZ2/
    foresto added a commit to foresto/cpython that referenced this issue Aug 1, 2024
    This extends imaplib with support for the rfc2177 IMAP IDLE command,
    as requested in python#55454.  It allows events to be pushed to a client as
    they occur, rather than having to continually poll for mailbox changes.
    
    The interface is a new idle() method, which returns an iterable context
    manager.  Entering the context starts IDLE mode, during which events
    (untagged responses) can be retrieved using the iteration protocol.
    Exiting the context sends DONE to the server, ending IDLE mode.
    
    An optional time limit for the IDLE session is supported, for use with
    servers that impose an inactivity timeout.
    
    The context manager also offers a burst() method, designed for programs
    wishing to process events in batch rather than one at a time.
    
    Notable differences from other implementations:
    
    - It's an extension to imaplib, rather than a replacement.
    - It doesn't introduce additional threads.
    - It doesn't impose new requirements on the use of imaplib's existing methods.
    - It passes the unit tests in CPython's test/test_imaplib.py module.
    - It works on Windows, Linux, and other unix-like systems.
    - It makes IDLE available on all of imaplib's client variants
      (including IMAP4_stream).
    - The interface is pythonic and easy to use.
    
    Caveats:
    
    - Due to a Windows limitation, the special case of IMAP4_stream running
      on Windows lacks a duration/timeout feature. (This is the stdin/stdout
      pipe connection variant; timeouts work fine for socket-based
      connections, even on Windows.) I have documented it where appropriate.
    
    - The file-like imaplib instance attributes are chagned from buffered to
      unbuffered mode. This could potentially break any client code that
      uses those objects directly without expecting partial reads/writes.
      However, these attributes are undocumented. As such, I think (and
      PEP 8 confirms) that they are fair game for changes.
      https://peps.python.org/pep-0008/#public-and-internal-interfaces
    
    Usage examples:
    
    python#55454 (comment)
    
    Original discusion:
    
    https://discuss.python.org/t/gauging-interest-in-my-imap4-idle-implementation-for-imaplib/59272
    
    Earlier requests and suggestions:
    
    https://bugs.python.org/issue11245
    
    python#55454
    
    https://mail.python.org/archives/list/python-ideas@python.org/thread/C4TVEYL5IBESQQPPS5GBR7WFBXCLQMZ2/
    foresto added a commit to foresto/cpython that referenced this issue Aug 1, 2024
    This extends imaplib with support for the rfc2177 IMAP IDLE command,
    as requested in python#55454.  It allows events to be pushed to a client as
    they occur, rather than having to continually poll for mailbox changes.
    
    The interface is a new idle() method, which returns an iterable context
    manager.  Entering the context starts IDLE mode, during which events
    (untagged responses) can be retrieved using the iteration protocol.
    Exiting the context sends DONE to the server, ending IDLE mode.
    
    An optional time limit for the IDLE session is supported, for use with
    servers that impose an inactivity timeout.
    
    The context manager also offers a burst() method, designed for programs
    wishing to process events in batch rather than one at a time.
    
    Notable differences from other implementations:
    
    - It's an extension to imaplib, rather than a replacement.
    - It doesn't introduce additional threads.
    - It doesn't impose new requirements on the use of imaplib's existing methods.
    - It passes the unit tests in CPython's test/test_imaplib.py module
      (and adds new ones).
    - It works on Windows, Linux, and other unix-like systems.
    - It makes IDLE available on all of imaplib's client variants
      (including IMAP4_stream).
    - The interface is pythonic and easy to use.
    
    Caveats:
    
    - Due to a Windows limitation, the special case of IMAP4_stream running
      on Windows lacks a duration/timeout feature. (This is the stdin/stdout
      pipe connection variant; timeouts work fine for socket-based
      connections, even on Windows.) I have documented it where appropriate.
    
    - The file-like imaplib instance attributes are changed from buffered to
      unbuffered mode. This could potentially break any client code that
      uses those objects directly without expecting partial reads/writes.
      However, these attributes are undocumented. As such, I think (and
      PEP 8 confirms) that they are fair game for changes.
      https://peps.python.org/pep-0008/#public-and-internal-interfaces
    
    Usage examples:
    
    python#55454 (comment)
    
    Original discussion:
    
    https://discuss.python.org/t/gauging-interest-in-my-imap4-idle-implementation-for-imaplib/59272
    
    Earlier requests and suggestions:
    
    https://bugs.python.org/issue11245
    
    python#55454
    
    https://mail.python.org/archives/list/python-ideas@python.org/thread/C4TVEYL5IBESQQPPS5GBR7WFBXCLQMZ2/
    foresto added a commit to foresto/cpython that referenced this issue Aug 1, 2024
    This extends imaplib with support for the rfc2177 IMAP IDLE command,
    as requested in python#55454.  It allows events to be pushed to a client as
    they occur, rather than having to continually poll for mailbox changes.
    
    The interface is a new idle() method, which returns an iterable context
    manager.  Entering the context starts IDLE mode, during which events
    (untagged responses) can be retrieved using the iteration protocol.
    Exiting the context sends DONE to the server, ending IDLE mode.
    
    An optional time limit for the IDLE session is supported, for use with
    servers that impose an inactivity timeout.
    
    The context manager also offers a burst() method, designed for programs
    wishing to process events in batch rather than one at a time.
    
    Notable differences from other implementations:
    
    - It's an extension to imaplib, rather than a replacement.
    - It doesn't introduce additional threads.
    - It doesn't impose new requirements on the use of imaplib's existing methods.
    - It passes the unit tests in CPython's test/test_imaplib.py module
      (and adds new ones).
    - It works on Windows, Linux, and other unix-like systems.
    - It makes IDLE available on all of imaplib's client variants
      (including IMAP4_stream).
    - The interface is pythonic and easy to use.
    
    Caveats:
    
    - Due to a Windows limitation, the special case of IMAP4_stream running
      on Windows lacks a duration/timeout feature. (This is the stdin/stdout
      pipe connection variant; timeouts work fine for socket-based
      connections, even on Windows.) I have documented it where appropriate.
    
    - The file-like imaplib instance attributes are changed from buffered to
      unbuffered mode. This could potentially break any client code that
      uses those objects directly without expecting partial reads/writes.
      However, these attributes are undocumented. As such, I think (and
      PEP 8 confirms) that they are fair game for changes.
      https://peps.python.org/pep-0008/#public-and-internal-interfaces
    
    Usage examples:
    
    python#55454 (comment)
    
    Original discussion:
    
    https://discuss.python.org/t/gauging-interest-in-my-imap4-idle-implementation-for-imaplib/59272
    
    Earlier requests and suggestions:
    
    https://bugs.python.org/issue11245
    
    python#55454
    
    https://mail.python.org/archives/list/python-ideas@python.org/thread/C4TVEYL5IBESQQPPS5GBR7WFBXCLQMZ2/
    foresto added a commit to foresto/cpython that referenced this issue Aug 1, 2024
    This extends imaplib with support for the rfc2177 IMAP IDLE command,
    as requested in python#55454.  It allows events to be pushed to a client as
    they occur, rather than having to continually poll for mailbox changes.
    
    The interface is a new idle() method, which returns an iterable context
    manager.  Entering the context starts IDLE mode, during which events
    (untagged responses) can be retrieved using the iteration protocol.
    Exiting the context sends DONE to the server, ending IDLE mode.
    
    An optional time limit for the IDLE session is supported, for use with
    servers that impose an inactivity timeout.
    
    The context manager also offers a burst() method, designed for programs
    wishing to process events in batch rather than one at a time.
    
    Notable differences from other implementations:
    
    - It's an extension to imaplib, rather than a replacement.
    - It doesn't introduce additional threads.
    - It doesn't impose new requirements on the use of imaplib's existing methods.
    - It passes the unit tests in CPython's test/test_imaplib.py module
      (and adds new ones).
    - It works on Windows, Linux, and other unix-like systems.
    - It makes IDLE available on all of imaplib's client variants
      (including IMAP4_stream).
    - The interface is pythonic and easy to use.
    
    Caveats:
    
    - Due to a Windows limitation, the special case of IMAP4_stream running
      on Windows lacks a duration/timeout feature. (This is the stdin/stdout
      pipe connection variant; timeouts work fine for socket-based
      connections, even on Windows.) I have documented it where appropriate.
    
    - The file-like imaplib instance attributes are changed from buffered to
      unbuffered mode. This could potentially break any client code that
      uses those objects directly without expecting partial reads/writes.
      However, these attributes are undocumented. As such, I think (and
      PEP 8 confirms) that they are fair game for changes.
      https://peps.python.org/pep-0008/#public-and-internal-interfaces
    
    Usage examples:
    
    python#55454 (comment)
    
    Original discussion:
    
    https://discuss.python.org/t/gauging-interest-in-my-imap4-idle-implementation-for-imaplib/59272
    
    Earlier requests and suggestions:
    
    https://bugs.python.org/issue11245
    
    python#55454
    
    https://mail.python.org/archives/list/python-ideas@python.org/thread/C4TVEYL5IBESQQPPS5GBR7WFBXCLQMZ2/
    foresto added a commit to foresto/cpython that referenced this issue Aug 1, 2024
    This extends imaplib with support for the rfc2177 IMAP IDLE command,
    as requested in python#55454.  It allows events to be pushed to a client as
    they occur, rather than having to continually poll for mailbox changes.
    
    The interface is a new idle() method, which returns an iterable context
    manager.  Entering the context starts IDLE mode, during which events
    (untagged responses) can be retrieved using the iteration protocol.
    Exiting the context sends DONE to the server, ending IDLE mode.
    
    An optional time limit for the IDLE session is supported, for use with
    servers that impose an inactivity timeout.
    
    The context manager also offers a burst() method, designed for programs
    wishing to process events in batch rather than one at a time.
    
    Notable differences from other implementations:
    
    - It's an extension to imaplib, rather than a replacement.
    - It doesn't introduce additional threads.
    - It doesn't impose new requirements on the use of imaplib's existing methods.
    - It passes the unit tests in CPython's test/test_imaplib.py module
      (and adds new ones).
    - It works on Windows, Linux, and other unix-like systems.
    - It makes IDLE available on all of imaplib's client variants
      (including IMAP4_stream).
    - The interface is pythonic and easy to use.
    
    Caveats:
    
    - Due to a Windows limitation, the special case of IMAP4_stream running
      on Windows lacks a duration/timeout feature. (This is the stdin/stdout
      pipe connection variant; timeouts work fine for socket-based
      connections, even on Windows.) I have documented it where appropriate.
    
    - The file-like imaplib instance attributes are changed from buffered to
      unbuffered mode. This could potentially break any client code that
      uses those objects directly without expecting partial reads/writes.
      However, these attributes are undocumented. As such, I think (and
      PEP 8 confirms) that they are fair game for changes.
      https://peps.python.org/pep-0008/#public-and-internal-interfaces
    
    Usage examples:
    
    python#55454 (comment)
    
    Original discussion:
    
    https://discuss.python.org/t/gauging-interest-in-my-imap4-idle-implementation-for-imaplib/59272
    
    Earlier requests and suggestions:
    
    https://bugs.python.org/issue11245
    
    python#55454
    
    https://mail.python.org/archives/list/python-ideas@python.org/thread/C4TVEYL5IBESQQPPS5GBR7WFBXCLQMZ2/
    foresto added a commit to foresto/cpython that referenced this issue Aug 1, 2024
    This extends imaplib with support for the rfc2177 IMAP IDLE command,
    as requested in python#55454.  It allows events to be pushed to a client as
    they occur, rather than having to continually poll for mailbox changes.
    
    The interface is a new idle() method, which returns an iterable context
    manager.  Entering the context starts IDLE mode, during which events
    (untagged responses) can be retrieved using the iteration protocol.
    Exiting the context sends DONE to the server, ending IDLE mode.
    
    An optional time limit for the IDLE session is supported, for use with
    servers that impose an inactivity timeout.
    
    The context manager also offers a burst() method, designed for programs
    wishing to process events in batch rather than one at a time.
    
    Notable differences from other implementations:
    
    - It's an extension to imaplib, rather than a replacement.
    - It doesn't introduce additional threads.
    - It doesn't impose new requirements on the use of imaplib's existing methods.
    - It passes the unit tests in CPython's test/test_imaplib.py module
      (and adds new ones).
    - It works on Windows, Linux, and other unix-like systems.
    - It makes IDLE available on all of imaplib's client variants
      (including IMAP4_stream).
    - The interface is pythonic and easy to use.
    
    Caveats:
    
    - Due to a Windows limitation, the special case of IMAP4_stream running
      on Windows lacks a duration/timeout feature. (This is the stdin/stdout
      pipe connection variant; timeouts work fine for socket-based
      connections, even on Windows.) I have documented it where appropriate.
    
    - The file-like imaplib instance attributes are changed from buffered to
      unbuffered mode. This could potentially break any client code that
      uses those objects directly without expecting partial reads/writes.
      However, these attributes are undocumented. As such, I think (and
      PEP 8 confirms) that they are fair game for changes.
      https://peps.python.org/pep-0008/#public-and-internal-interfaces
    
    Usage examples:
    
    python#55454 (comment)
    
    Original discussion:
    
    https://discuss.python.org/t/gauging-interest-in-my-imap4-idle-implementation-for-imaplib/59272
    
    Earlier requests and suggestions:
    
    https://bugs.python.org/issue11245
    
    python#55454
    
    https://mail.python.org/archives/list/python-ideas@python.org/thread/C4TVEYL5IBESQQPPS5GBR7WFBXCLQMZ2/
    foresto added a commit to foresto/cpython that referenced this issue Aug 1, 2024
    This extends imaplib with support for the rfc2177 IMAP IDLE command,
    as requested in python#55454.  It allows events to be pushed to a client as
    they occur, rather than having to continually poll for mailbox changes.
    
    The interface is a new idle() method, which returns an iterable context
    manager.  Entering the context starts IDLE mode, during which events
    (untagged responses) can be retrieved using the iteration protocol.
    Exiting the context sends DONE to the server, ending IDLE mode.
    
    An optional time limit for the IDLE session is supported, for use with
    servers that impose an inactivity timeout.
    
    The context manager also offers a burst() method, designed for programs
    wishing to process events in batch rather than one at a time.
    
    Notable differences from other implementations:
    
    - It's an extension to imaplib, rather than a replacement.
    - It doesn't introduce additional threads.
    - It doesn't impose new requirements on the use of imaplib's existing methods.
    - It passes the unit tests in CPython's test/test_imaplib.py module
      (and adds new ones).
    - It works on Windows, Linux, and other unix-like systems.
    - It makes IDLE available on all of imaplib's client variants
      (including IMAP4_stream).
    - The interface is pythonic and easy to use.
    
    Caveats:
    
    - Due to a Windows limitation, the special case of IMAP4_stream running
      on Windows lacks a duration/timeout feature. (This is the stdin/stdout
      pipe connection variant; timeouts work fine for socket-based
      connections, even on Windows.) I have documented it where appropriate.
    
    - The file-like imaplib instance attributes are changed from buffered to
      unbuffered mode. This could potentially break any client code that
      uses those objects directly without expecting partial reads/writes.
      However, these attributes are undocumented. As such, I think (and
      PEP 8 confirms) that they are fair game for changes.
      https://peps.python.org/pep-0008/#public-and-internal-interfaces
    
    Usage examples:
    
    python#55454 (comment)
    
    Original discussion:
    
    https://discuss.python.org/t/gauging-interest-in-my-imap4-idle-implementation-for-imaplib/59272
    
    Earlier requests and suggestions:
    
    https://bugs.python.org/issue11245
    
    python#55454
    
    https://mail.python.org/archives/list/python-ideas@python.org/thread/C4TVEYL5IBESQQPPS5GBR7WFBXCLQMZ2/
    foresto added a commit to foresto/cpython that referenced this issue Aug 1, 2024
    This extends imaplib with support for the rfc2177 IMAP IDLE command,
    as requested in python#55454.  It allows events to be pushed to a client as
    they occur, rather than having to continually poll for mailbox changes.
    
    The interface is a new idle() method, which returns an iterable context
    manager.  Entering the context starts IDLE mode, during which events
    (untagged responses) can be retrieved using the iteration protocol.
    Exiting the context sends DONE to the server, ending IDLE mode.
    
    An optional time limit for the IDLE session is supported, for use with
    servers that impose an inactivity timeout.
    
    The context manager also offers a burst() method, designed for programs
    wishing to process events in batch rather than one at a time.
    
    Notable differences from other implementations:
    
    - It's an extension to imaplib, rather than a replacement.
    - It doesn't introduce additional threads.
    - It doesn't impose new requirements on the use of imaplib's existing methods.
    - It passes the unit tests in CPython's test/test_imaplib.py module
      (and adds new ones).
    - It works on Windows, Linux, and other unix-like systems.
    - It makes IDLE available on all of imaplib's client variants
      (including IMAP4_stream).
    - The interface is pythonic and easy to use.
    
    Caveats:
    
    - Due to a Windows limitation, the special case of IMAP4_stream running
      on Windows lacks a duration/timeout feature. (This is the stdin/stdout
      pipe connection variant; timeouts work fine for socket-based
      connections, even on Windows.) I have documented it where appropriate.
    
    - The file-like imaplib instance attributes are changed from buffered to
      unbuffered mode. This could potentially break any client code that
      uses those objects directly without expecting partial reads/writes.
      However, these attributes are undocumented. As such, I think (and
      PEP 8 confirms) that they are fair game for changes.
      https://peps.python.org/pep-0008/#public-and-internal-interfaces
    
    Usage examples:
    
    python#55454 (comment)
    
    Original discussion:
    
    https://discuss.python.org/t/gauging-interest-in-my-imap4-idle-implementation-for-imaplib/59272
    
    Earlier requests and suggestions:
    
    https://bugs.python.org/issue11245
    
    python#55454
    
    https://mail.python.org/archives/list/python-ideas@python.org/thread/C4TVEYL5IBESQQPPS5GBR7WFBXCLQMZ2/
    foresto added a commit to foresto/cpython that referenced this issue Aug 2, 2024
    This extends imaplib with support for the rfc2177 IMAP IDLE command,
    as requested in python#55454.  It allows events to be pushed to a client as
    they occur, rather than having to continually poll for mailbox changes.
    
    The interface is a new idle() method, which returns an iterable context
    manager.  Entering the context starts IDLE mode, during which events
    (untagged responses) can be retrieved using the iteration protocol.
    Exiting the context sends DONE to the server, ending IDLE mode.
    
    An optional time limit for the IDLE session is supported, for use with
    servers that impose an inactivity timeout.
    
    The context manager also offers a burst() method, designed for programs
    wishing to process events in batch rather than one at a time.
    
    Notable differences from other implementations:
    
    - It's an extension to imaplib, rather than a replacement.
    - It doesn't introduce additional threads.
    - It doesn't impose new requirements on the use of imaplib's existing methods.
    - It passes the unit tests in CPython's test/test_imaplib.py module
      (and adds new ones).
    - It works on Windows, Linux, and other unix-like systems.
    - It makes IDLE available on all of imaplib's client variants
      (including IMAP4_stream).
    - The interface is pythonic and easy to use.
    
    Caveats:
    
    - Due to a Windows limitation, the special case of IMAP4_stream running
      on Windows lacks a duration/timeout feature. (This is the stdin/stdout
      pipe connection variant; timeouts work fine for socket-based
      connections, even on Windows.) I have documented it where appropriate.
    
    - The file-like imaplib instance attributes are changed from buffered to
      unbuffered mode. This could potentially break any client code that
      uses those objects directly without expecting partial reads/writes.
      However, these attributes are undocumented. As such, I think (and
      PEP 8 confirms) that they are fair game for changes.
      https://peps.python.org/pep-0008/#public-and-internal-interfaces
    
    Usage examples:
    
    python#55454 (comment)
    
    Original discussion:
    
    https://discuss.python.org/t/gauging-interest-in-my-imap4-idle-implementation-for-imaplib/59272
    
    Earlier requests and suggestions:
    
    python#55454
    
    https://mail.python.org/archives/list/python-ideas@python.org/thread/C4TVEYL5IBESQQPPS5GBR7WFBXCLQMZ2/
    foresto added a commit to foresto/cpython that referenced this issue Aug 16, 2024
    This extends imaplib with support for the rfc2177 IMAP IDLE command,
    as requested in python#55454.  It allows events to be pushed to a client as
    they occur, rather than having to continually poll for mailbox changes.
    
    The interface is a new idle() method, which returns an iterable context
    manager.  Entering the context starts IDLE mode, during which events
    (untagged responses) can be retrieved using the iteration protocol.
    Exiting the context sends DONE to the server, ending IDLE mode.
    
    An optional time limit for the IDLE session is supported, for use with
    servers that impose an inactivity timeout.
    
    The context manager also offers a burst() method, designed for programs
    wishing to process events in batch rather than one at a time.
    
    Notable differences from other implementations:
    
    - It's an extension to imaplib, rather than a replacement.
    - It doesn't introduce additional threads.
    - It doesn't impose new requirements on the use of imaplib's existing methods.
    - It passes the unit tests in CPython's test/test_imaplib.py module
      (and adds new ones).
    - It works on Windows, Linux, and other unix-like systems.
    - It makes IDLE available on all of imaplib's client variants
      (including IMAP4_stream).
    - The interface is pythonic and easy to use.
    
    Caveats:
    
    - Due to a Windows limitation, the special case of IMAP4_stream running
      on Windows lacks a duration/timeout feature. (This is the stdin/stdout
      pipe connection variant; timeouts work fine for socket-based
      connections, even on Windows.) I have documented it where appropriate.
    
    - The file-like imaplib instance attributes are changed from buffered to
      unbuffered mode. This could potentially break any client code that
      uses those objects directly without expecting partial reads/writes.
      However, these attributes are undocumented. As such, I think (and
      PEP 8 confirms) that they are fair game for changes.
      https://peps.python.org/pep-0008/#public-and-internal-interfaces
    
    Usage examples:
    
    python#55454 (comment)
    
    Original discussion:
    
    https://discuss.python.org/t/gauging-interest-in-my-imap4-idle-implementation-for-imaplib/59272
    
    Earlier requests and suggestions:
    
    python#55454
    
    https://mail.python.org/archives/list/python-ideas@python.org/thread/C4TVEYL5IBESQQPPS5GBR7WFBXCLQMZ2/
    foresto added a commit to foresto/cpython that referenced this issue Sep 1, 2024
    This extends imaplib with support for the rfc2177 IMAP IDLE command,
    as requested in python#55454.  It allows events to be pushed to a client as
    they occur, rather than having to continually poll for mailbox changes.
    
    The interface is a new idle() method, which returns an iterable context
    manager.  Entering the context starts IDLE mode, during which events
    (untagged responses) can be retrieved using the iteration protocol.
    Exiting the context sends DONE to the server, ending IDLE mode.
    
    An optional time limit for the IDLE session is supported, for use with
    servers that impose an inactivity timeout.
    
    The context manager also offers a burst() method, designed for programs
    wishing to process events in batch rather than one at a time.
    
    Notable differences from other implementations:
    
    - It's an extension to imaplib, rather than a replacement.
    - It doesn't introduce additional threads.
    - It doesn't impose new requirements on the use of imaplib's existing methods.
    - It passes the unit tests in CPython's test/test_imaplib.py module
      (and adds new ones).
    - It works on Windows, Linux, and other unix-like systems.
    - It makes IDLE available on all of imaplib's client variants
      (including IMAP4_stream).
    - The interface is pythonic and easy to use.
    
    Caveats:
    
    - Due to a Windows limitation, the special case of IMAP4_stream running
      on Windows lacks a duration/timeout feature. (This is the stdin/stdout
      pipe connection variant; timeouts work fine for socket-based
      connections, even on Windows.) I have documented it where appropriate.
    
    - The file-like imaplib instance attributes are changed from buffered to
      unbuffered mode. This could potentially break any client code that
      uses those objects directly without expecting partial reads/writes.
      However, these attributes are undocumented. As such, I think (and
      PEP 8 confirms) that they are fair game for changes.
      https://peps.python.org/pep-0008/#public-and-internal-interfaces
    
    Usage examples:
    
    python#55454 (comment)
    
    Original discussion:
    
    https://discuss.python.org/t/gauging-interest-in-my-imap4-idle-implementation-for-imaplib/59272
    
    Earlier requests and suggestions:
    
    python#55454
    
    https://mail.python.org/archives/list/python-ideas@python.org/thread/C4TVEYL5IBESQQPPS5GBR7WFBXCLQMZ2/
    foresto added a commit to foresto/cpython that referenced this issue Sep 11, 2024
    - Add example idle response tuples, to make the minor difference from other
      imaplib response tuples more obvious.
    - Merge the idle context manager's burst() method docs with the IMAP
      object's idle() method docs, for easier understanding.
    - Upgrade the Windows note regarding lack of pipe timeouts to a warning.
    - Rephrase various things for clarity.
    foresto added a commit to foresto/cpython that referenced this issue Sep 11, 2024
    - Add example idle response tuples, to make the minor difference from other
      imaplib response tuples more obvious.
    - Merge the idle context manager's burst() method docs with the IMAP
      object's idle() method docs, for easier understanding.
    - Upgrade the Windows note regarding lack of pipe timeouts to a warning.
    - Rephrase various things for clarity.
    @picnixz picnixz removed the 3.9 only security fixes label Dec 3, 2024
    @picnixz picnixz changed the title Implementation of IMAP IDLE in imaplib? Add support for IMAP IDLE in imaplib Dec 3, 2024
    @serhiy-storchaka
    Copy link
    Member

    I am sorry, but I am not sure the interface proposed in #122542 is the best possible interface. There are some issues with it:

    • Using the iterator protocol means that the polling is blocking. When we are blocked on waiting a response from the server, there is no way to interrupt waiting. It means that an interactive application which displays an IMAP4 folder status at runtime cannot be responsible to user input. We need a non-blocking operation to poll a response if there is one and return immediately if there are no responses. We need also a blocking operation with timeout and a way to interrupt thread before expiration of the timeout.
    • Using the context manager protocol means that DONE is only sent after leaving the with block. After client sent DONE, server can still send status updates. What happens to them? You cannot use Idler after leaving the with block. We need a method to send DONE and read status updates that came after this.
    • Server can send status updates not only for IDLE command, but for other commands. It is common to use the NOOP command for this. There is a large similarity between NOOP and IDLE commands, but is is not seen in the proposed interface. You can read status updates from untagged_responces after the NOOP command, although this is not thread-safe. It would be nice to have unified way to get status updates for IDLE and other commands. It would be nice if it is thread-safe or there is a clear way to use the IMAP4 client in thread-safe way.

    Let's look what other solutions provide. Interesting, there were two different third-party packages named imaplib2, both were proposed for inclusion in the stdlib.

    Piers Lauder's imaplib2 was discussed above. It is available on PyPI. I has a simple blocking idle() method which waits for first server response or expiration of the timeout. Then it send DONE and wait for tagged response for the IDLE command. You can read status updates from untagged_responces after this. It may be not very efficient, because you need to send IDLE again, but it fits perfectly in the line with other commands. This package is compatible with imaplib. The main disadvantage of this package is that it runs internal threads. But I think that there may be possibility to implement it without running internal thread. Instead the user will need to run external threads to utilize the IMAP4 concurrency features. imaplib will still need to use locks and events to make its methods thread-safe.

    Maxim Khitrov's imaplib2 was discussed in Python-ideas. It provides incompatible interface, in particularly concurrent fetch(). Its interface for IDLE is similar to the one proposed in #122542, with support for the context manager and iterator protocols, but it also has the poll() method.

    Looking at support in other programming languages, in Perl, support for IDLE is provided by three methods -- idle, idle_data and done. idle is non-blocking, it simply sends the IDLE command, idle_data is non-blocking or blocking with timeout, it polls server response. done sends DONE.

    @serhiy-storchaka
    Copy link
    Member

    I think that we first should implement support for concurrent execution of other commands (or at least draw a plan for it). Then we can add support for the IDLE command which fits the general design.

    I think that many (if not all) commands should have a non-blocking variant. Either boolean parameter or a method with special suffix. Non-blocking method sends the command but does not wait for the tagged response. It should return something Future-like. You can block a thread waiting on the response for specific command or check if there is response without blocking.

    There should be a thread-safe method for polling non-tagged responses (after IDLE or other commands). It should provide blocking and non-blocking interface. Iterator protocol also can be supported for convenience -- you can iterate only already received non-tagged responses or block to wait on new responses.

    IDLE will also have blocking and non-blocking variants. In blocking variant it will wait on tagged server response (which can only be sent after client sent DONE in other thread), in non-blocking variant it will return a Future-like object (which also supports the context manager protocol and has the done() method). We can also implement auto-deblocking option like in Piers Lauder's imaplib2 -- send DONE after first non-tagged response and wait for tagged response.

    @foresto
    Copy link
    Contributor

    foresto commented Dec 20, 2024

    I am sorry, but I am not sure the interface proposed in #122542 is the best possible interface. There are some issues with it:

    • Using the iterator protocol means that the polling is blocking.

    That's not a requirement of the iterator protocol. It supports stopping immediately without producing any results, much like the read() system call can do with a non-blocking file descriptor. As a matter of fact, #122542 already does this when a limited-duration Idler has expired.

    It does block in other cases, and this is by design, because the whole of imaplib is a blocking API. #122542 respects and preserves that design. It's a small extension to the existing module, not a replacement for it.

    (And just to be clear for the sake of other readers, the IDLE command doesn't poll, at least not in the way we usually mean in computer science. The point of IDLE is to avoid polling, and instead react to events sent by the server as they arrive.)

    When we are blocked on waiting a response from the server, there is no way to interrupt waiting.

    There are two ways. Waiting can be cleanly interrupted by:

    • A duration (time limit) passed to the call that sets up the IDLE context.
    • An exception. For example, a KeyboardInterrupt in an interactive terminal program, or some other exception raised by a timer or signal handler.

    If you're wishing for an API to interrupt a blocking IDLE from another thread, then it is true that no such thing exists at the moment. The same is true for every other command in imaplib.

    Such a thing could be added in the future. One way would be modifying imaplib to use non-bocking sockets, and make blocking commands call select() on both the IMAP socket and one end of a socket pair. Another thread could then write to the other end of the socket pair to interrupt the command. The interface in #122542 would not have to change.

    Of course, moving imaplib to non-blocking sockets and adding calls to interrupt its blocking commands would be additional features, beyond the scope of #122542. This PR simply adds support for IMAP IDLE. It doesn't aim to overhaul or replace imaplib.

    It means that an interactive application which displays an IMAP4 folder status at runtime cannot be responsible to user input.

    Let's keep in mind that introducing IDLE support would't mean all applications have to use it. They can continue using the existing commands, and poll for updates using NOOP, just as they always have.

    When considering applications with asynchronous display updates, let's also remember that all of imaplib is already a blocking API, so the issues around blocking calls are already present. An application can deal with them by running commands in a separate thread. #122542 doesn't change this. The same approach can be used with IDLE.

    Note that IMAP doesn't allow other commands during IDLE, so an application wishing to do both at the same time would have to keep two connections open, presumably in separate threads. That's a limitation imposed by the protocol.

    (I suppose a library could hide this limitation by automatically switching between command mode and IDLE mode, making the switches transparent to application code, but such a library would be considerably more complex than imaplib.)

    We need a non-blocking operation to poll a response if there is one and return immediately if there are no responses.

    This doesn't exist in any of imaplib's commands. I agree that it could be useful for certain programming models, but it's a separate concern from adding IDLE support. I hope we can agree that the two should not be conflated.

    Are you worried that #122542 might get in the way of future non-blocking support in imaplib? I don't believe it would.

    • Using the context manager protocol means that DONE is only sent after leaving the with block. After client sent DONE, server can still send status updates. What happens to them? You cannot use Idler after leaving the with block. We need a method to send DONE and read status updates that came after this.

    Indeed, the server can send untagged responses after DONE is sent. These are already handled in #122542. They can be retrieved with the response() method, just as they can after other commands.

    • Server can send status updates not only for IDLE command, but for other commands. It is common to use the NOOP command for this. There is a large similarity between NOOP and IDLE commands, but is is not seen in the proposed interface.

    Both can produce untagged responses, but their similarity ends there. Notably:

    • The IDLE command requests a temporary inversion of control, which is represented nicely in the proposed interface using a python context manager.
    • The IDLE command produces an unbounded stream of responses with unpredictable timing, which is represented nicely in the proposed interface using python iteration.

    IDLE and NOOP are made for different situations, and their interfaces reflect that.

    You can read status updates from untagged_responces after the NOOP command,

    The untagged_responses attribute is an internal, undocumented interface. Programs should not be using it. The correct way to do this is through the response() method. This still works in #122542, including responses after leaving the IDLE context (as mentioned above).

    It would be nice to have unified way to get status updates for IDLE and other commands.

    This could be done with a callback interface, iteration, or polling. (Although using a polling interface for IDLE would be a bit silly, because its purpose is to avoid polling; that's what NOOP is for.)

    However, it would mean modifying or duplicating every command method in imaplib, not just IDLE, and adding features that are orthogonal to the present issue. I think that would make more sense as a separate feature request. Don't you?

    It would be nice if it is thread-safe or there is a clear way to use the IMAP4 client in thread-safe way.

    I guess you mean sharing a single IMAP4 object and connection among multiple threads. (As far as I know, imaplib is already thread-safe when each thread uses its own IMAP4 object. I don't see any writes to global variables, at least.)

    I think this would be a little tricky even without IDLE, since you would have to multiplex commands from different threads into a single connection, ensure atomic writes to the server, sort out the various tagged and untagged responses, and route them to the threads that wanted them. It would be even harder with IDLE, because IMAP doesn't support command mode and IDLE mode at the same time. You would have to dynamically switch between modes within the library while simulating a single mode from each thread's point of view.

    I suppose that might be possible, but would be a major undertaking, and the result would arguably not be imaplib any more. If it's important to you, I think you should write up a separate issue for it. This one is about adding IDLE support to imaplib.

    Let's look what other solutions provide. Interesting, there were two different third-party packages named imaplib2, both were proposed for inclusion in the stdlib.

    Piers Lauder's imaplib2 was discussed above. It is available on PyPI. I has a simple blocking idle() method which waits for first server response or expiration of the timeout. Then it send DONE and wait for tagged response for the IDLE command. You can read status updates from untagged_responces after this. It may be not very efficient, because you need to send IDLE again, but it fits perfectly in the line with other commands. This package is compatible with imaplib. The main disadvantage of this package is that it runs internal threads. But I think that there may be possibility to implement it without running internal thread. Instead the user will need to run external threads to utilize the IMAP4 concurrency features. imaplib will still need to use locks and events to make its methods thread-safe.

    I've seen that one. #122542 already does all those things, and more, and it does them more elegantly, and without imposing threads on applications.

    Maxim Khitrov's imaplib2 was discussed in Python-ideas. It provides incompatible interface, in particularly concurrent fetch(). Its interface for IDLE is similar to the one proposed in #122542, with support for the context manager and iterator protocols, but it also has the poll() method.

    I've seen that one, and no, its interface for IDLE is not similar to #122542. Note that its iteration feature doesn't work during IDLE. Instead, it forces the application to use a polling loop and make select() or sleep() calls in between. It also has various implementation flaws that will make it misbehave in conditions where #122542 works correctly. Perhaps those could be fixed, but there's a bigger problem with it:

    The interface is incompatible with imaplib, as you pointed out. So adopting it would effectively mean replacing imaplib.

    Looking at support in other programming languages, in Perl, support for IDLE is provided by three methods -- idle, idle_data and done. idle is non-blocking, it simply sends the IDLE command, idle_data is non-blocking or blocking with timeout, it polls server response. done sends DONE.

    My takeaway from everything you've written here is that you would like a non-blocking IMAP client library with polling semantics.

    I can see ways in which that would be useful, but imaplib is not that library, and there is no IDLE design or implementation that can change that. The imaplib API is a blocking design throughout. Getting what you want would require substantial changes all across the library, or else replacing it.

    You're not alone in your wish. An imaplib overhaul or replacement has been suggested every so often for at least 13 years. However, nobody has done the work to make it happen in the standard library, and there is no sign that anyone will have done that work any time soon.

    Meanwhile, we finally have a clean, working, compatible, pythonic, documented, reviewed pull request providing IDLE support, which people have also been requesting for at least 13 years. And as far as I can tell, there is nothing about it that would prevent the non-blocking interface you want from being added later.

    I therefore propose that the request for non-blocking imaplib interfaces (or replacing imaplib entirely) be moved to a new issue.

    gvanrossum added a commit that referenced this issue Feb 7, 2025
    * gh-55454: Add IMAP4 IDLE support to imaplib
    
    This extends imaplib with support for the rfc2177 IMAP IDLE command,
    as requested in #55454.  It allows events to be pushed to a client as
    they occur, rather than having to continually poll for mailbox changes.
    
    The interface is a new idle() method, which returns an iterable context
    manager.  Entering the context starts IDLE mode, during which events
    (untagged responses) can be retrieved using the iteration protocol.
    Exiting the context sends DONE to the server, ending IDLE mode.
    
    An optional time limit for the IDLE session is supported, for use with
    servers that impose an inactivity timeout.
    
    The context manager also offers a burst() method, designed for programs
    wishing to process events in batch rather than one at a time.
    
    Notable differences from other implementations:
    
    - It's an extension to imaplib, rather than a replacement.
    - It doesn't introduce additional threads.
    - It doesn't impose new requirements on the use of imaplib's existing methods.
    - It passes the unit tests in CPython's test/test_imaplib.py module
      (and adds new ones).
    - It works on Windows, Linux, and other unix-like systems.
    - It makes IDLE available on all of imaplib's client variants
      (including IMAP4_stream).
    - The interface is pythonic and easy to use.
    
    Caveats:
    
    - Due to a Windows limitation, the special case of IMAP4_stream running
      on Windows lacks a duration/timeout feature. (This is the stdin/stdout
      pipe connection variant; timeouts work fine for socket-based
      connections, even on Windows.) I have documented it where appropriate.
    
    - The file-like imaplib instance attributes are changed from buffered to
      unbuffered mode. This could potentially break any client code that
      uses those objects directly without expecting partial reads/writes.
      However, these attributes are undocumented. As such, I think (and
      PEP 8 confirms) that they are fair game for changes.
      https://peps.python.org/pep-0008/#public-and-internal-interfaces
    
    Usage examples:
    
    #55454 (comment)
    
    Original discussion:
    
    https://discuss.python.org/t/gauging-interest-in-my-imap4-idle-implementation-for-imaplib/59272
    
    Earlier requests and suggestions:
    
    #55454
    
    https://mail.python.org/archives/list/python-ideas@python.org/thread/C4TVEYL5IBESQQPPS5GBR7WFBXCLQMZ2/
    
    * gh-55454: Clarify imaplib idle() docs
    
    - Add example idle response tuples, to make the minor difference from other
      imaplib response tuples more obvious.
    - Merge the idle context manager's burst() method docs with the IMAP
      object's idle() method docs, for easier understanding.
    - Upgrade the Windows note regarding lack of pipe timeouts to a warning.
    - Rephrase various things for clarity.
    
    * docs: words instead of <=
    
    Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
    
    * docs: improve style in an example
    
    Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
    
    * docs: grammatical edit
    
    Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
    
    * docs consistency
    
    Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
    
    * comment -> docstring
    
    Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
    
    * docs: refer to imaplib as "this module"
    
    Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
    
    * imaplib: simplify & clarify idle debug message
    
    Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
    
    * imaplib: elaborate in idle context manager comment
    
    * imaplib: re-raise BaseException instead of bare except
    
    Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
    
    * imaplib: convert private doc string to comment
    
    * docs: correct mistake in imaplib example
    
    This is a correction to 8077f2e, which
    changed a variable name in only one place and broke the subsequent
    reference to it, departed from the naming convention used in the rest of
    the module, and shadowed the type() builtin along the way.
    
    * imaplib: simplify example code in doc string
    
    This is for consistency with the documentation change in 8077f2e
    and subsequent correction in 013bbf1.
    
    * imaplib: rename _Idler to Idler, update its docs
    
    * imaplib: add comment in Idler._pop()
    
    Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
    
    * imaplib: remove unnecessary blank line
    
    Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
    
    * imaplib: comment on use of unbuffered pipes
    
    * docs: imaplib: use the reStructuredText :class: role
    
    Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
    
    * Revert "docs: imaplib: use the reStructuredText :class: role"
    
    This reverts commit f385e44, because it
    triggers CI failures in the docs by referencing a class that is
    (deliberately) undocumented.
    
    * docs: imaplib: use the reST :class: role, escaped
    
    This is a different approach to f385e44, which was reverted for
    creating dangling link references.
    
    By prefixing the reStructuredText role target with a ! we disable
    conversion to a link, thereby passing continuous integration checks
    even though the referenced class is deliberately absent from the
    documentation.
    
    * docs: refer to IMAP4 IDLE instead of just IDLE
    
    This clarifies that we are referring to the email protocol, not the editor with the same name.
    
    Co-authored-by: Guido van Rossum <gvanrossum@gmail.com>
    
    * imaplib: IDLE -> IMAP4 IDLE in exception message
    
    Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
    
    * docs: imaplib idle() phrasing and linking tweaks
    
    * docs: imaplib: avoid linking to an invalid target
    
    This reverts and rephrases part of a3f21cd
    which created links to a method on a deliberately undocumented class.
    The links didn't work consistently, and caused sphinx warnings that
    broke cpython's continuous integration tests.
    
    * imaplib: update test after recent exception change
    
    This fixes a test that was broken by changing an exception in
    b01de95
    
    * imaplib: rename idle() dur argument to duration
    
    * imaplib: bytes.index() -> bytes.find()
    
    This makes it more obvious which statement triggers the branch.
    
    * imaplib: remove no-longer-necessary statement
    
    Co-authored-by: Martin Panter <vadmium@users.noreply.github.com>
    
    * docs: imaplib: concise & valid method links
    
    The burst() method is a little tricky to link in restructuredText, due
    to quirks of its parent class.  This syntax allows sphinx to generate
    working links without generating warnings (which break continuous
    integration) and without burdening the reader with unimportant namespace
    qualifications.  It makes the reST source ugly, but few people read
    the reST source, so it's a tolerable tradeoff.
    
    * imaplib: note data types present in IDLE responses
    
    * docs: imaplib: add comma to reST changes header
    
    Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
    
    * imaplib: sync doc strings with reST docs
    
    * docs: imaplib: minor Idler clarifications
    
    * imaplib: idle: emit (type, [data, ...]) tuples
    
    This allows our iterator to emit untagged responses that contain literal
    strings in the same way that imaplib's existing methods do, while still
    emitting exactly one whole response per iteration.
    
    * imaplib: while/yield instead of yield from iter()
    
    * imaplib: idle: use deadline idiom when iterating
    
    This simplifies the code, and avoids idle duration drift from time spent
    processing each iteration.
    
    * docs: imaplib: state duration/interval arg types
    
    * docs: imaplib: minor rephrasing of a sentence
    
    * docs: imaplib: reposition a paragraph
    
    This might improve readability, especially when encountering Idler.burst()
    for the first time.
    
    * docs: imaplib: wrap long lines in idle() section
    
    * docs: imaplib: note: Idler objects require 'with'
    
    * docs: imaplib: say that 29 minutes is 1740 seconds
    
    * docs: imaplib: mark a paragraph as a 'tip'
    
    * docs: imaplib: rephrase reference to MS Windows
    
    * imaplib: end doc string titles with a period
    
    * imaplib: idle: socket timeouts instead of select()
    
    IDLE timeouts were originally implemented using select() after
    checking for the presence of already-buffered data.
    That allowed timeouts on pipe connetions like IMAP4_stream.
    However, it seemed possible that SSL data arriving without any
    IMAP data afterward could cause select() to indicate available
    application data when there was none, leading to a read() call
    that would block with no timeout. It was unclear under what
    conditions this would happen in practice. This change switches
    to socket timeouts instead of select(), just to be safe.
    
    This also reverts IMAP4_stream changes that were made to support IDLE
    timeouts, since our new implementation only supports socket connections.
    
    * imaplib: Idler: rename private state attributes
    
    * imaplib: rephrase a comment in example code
    
    * docs: imaplib: idle: use Sphinx code-block:: pycon
    
    * docs: whatsnew: imaplib: reformat IMAP4.idle entry
    
    * imaplib: idle: make doc strings brief
    
    Since we generally rely on the reST/html documentation for details, we
    can keep these doc strings short. This matches the module's existing doc
    string style and avoids having to sync small changes between two files.
    
    * imaplib: Idler: split assert into two statements
    
    * imaplib: Idler: move assignment out of try: block
    
    * imaplib: Idler: move __exit__() for readability
    
    * imaplib: Idler: move __next__() for readability
    
    * imaplib: test: make IdleCmdHandler a global class
    
    * docs: imaplib: idle: collapse double-spaces
    
    * imaplib: warn on use of undocumented 'file' attr
    
    * imaplib: revert import reformatting
    
    Since we no longer import platform or selectors, the original import
    statement style can be restored, reducing the footprint of PR #122542.
    
    * imaplib: restore original exception msg formatting
    
    This reduces the footprint of PR #122542.
    
    * docs: imaplib: idle: versionadded:: next
    
    * imaplib: move import statement to where it's used
    
    This import is only needed if external code tries to use an attribute
    that it shouldn't be using. Making it a local import reduces module
    loading time in supported cases.
    
    * imaplib test: RuntimeWarning on IMAP4.file access
    
    * imaplib: use stacklevel=2 in warnings.warn()
    
    * imaplib test: simplify IMAP4.file warning test
    
    * imaplib test: pre-idle-continuation response
    
    * imaplib test: post-done untagged response
    
    * imaplib: downgrade idle-denied exception to error
    
    This makes it easier for client code to distinguish a temporary
    rejection of the IDLE command from a server responding incorrectly to
    IDLE.
    
    * imaplib: simplify check for socket object
    
    * imaplib: narrow the scope of IDLE socket timeouts
    
    If an IDLE duration or burst() was in use, and an unsolicited response
    contained a literal string, and crossed a packet boundary, and the
    subsequent packet was delayed beyond the IDLE feature's time limit, the
    timeout would leave the incoming protocol stream in a bad state (with
    the tail of that response appearing where the start of a response is
    expected).
    
    This change moves the IDLE socket timeout to cover only the start
    of a response, so it can no longer cause that problem.
    
    * imaplib: preserve partial reads on exception
    
    This ensures that short IDLE durations / burst() intervals
    won't risk corrupting response lines that span multiple packets.
    
    * imaplib: read/readline: save multipart buffer tail
    
    For resilience if read() or readline() ever complete with more than one
    bytes object remaining in the buffer. This is not expected to happen,
    but it seems wise to be prepared for a future change making it possible.
    
    * imaplib: use TimeoutError subclass only if needed
    
    * doc: imaplib: elaborate on IDLE response delivery
    
    * doc: imaplib: elaborate in note re: IMAP4.response
    
    * imaplib: comment on benefit of reading in chunks
    
    Our read() implementation designed to support IDLE replaces the one from
    PR #119514, fixing the same problem it was addressing. The tests that it
    added are preserved.
    
    * imaplib: readline(): treat ConnectionError as EOF
    
    ---------
    
    Co-authored-by: Gregory P. Smith <greg@krypto.org>
    Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
    Co-authored-by: Guido van Rossum <guido@python.org>
    Co-authored-by: Guido van Rossum <gvanrossum@gmail.com>
    Co-authored-by: Martin Panter <vadmium@users.noreply.github.com>
    Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
    @gvanrossum
    Copy link
    Member

    This landed, so closing.

    srinivasreddy pushed a commit to srinivasreddy/cpython that referenced this issue Feb 7, 2025
    * pythongh-55454: Add IMAP4 IDLE support to imaplib
    
    This extends imaplib with support for the rfc2177 IMAP IDLE command,
    as requested in python#55454.  It allows events to be pushed to a client as
    they occur, rather than having to continually poll for mailbox changes.
    
    The interface is a new idle() method, which returns an iterable context
    manager.  Entering the context starts IDLE mode, during which events
    (untagged responses) can be retrieved using the iteration protocol.
    Exiting the context sends DONE to the server, ending IDLE mode.
    
    An optional time limit for the IDLE session is supported, for use with
    servers that impose an inactivity timeout.
    
    The context manager also offers a burst() method, designed for programs
    wishing to process events in batch rather than one at a time.
    
    Notable differences from other implementations:
    
    - It's an extension to imaplib, rather than a replacement.
    - It doesn't introduce additional threads.
    - It doesn't impose new requirements on the use of imaplib's existing methods.
    - It passes the unit tests in CPython's test/test_imaplib.py module
      (and adds new ones).
    - It works on Windows, Linux, and other unix-like systems.
    - It makes IDLE available on all of imaplib's client variants
      (including IMAP4_stream).
    - The interface is pythonic and easy to use.
    
    Caveats:
    
    - Due to a Windows limitation, the special case of IMAP4_stream running
      on Windows lacks a duration/timeout feature. (This is the stdin/stdout
      pipe connection variant; timeouts work fine for socket-based
      connections, even on Windows.) I have documented it where appropriate.
    
    - The file-like imaplib instance attributes are changed from buffered to
      unbuffered mode. This could potentially break any client code that
      uses those objects directly without expecting partial reads/writes.
      However, these attributes are undocumented. As such, I think (and
      PEP 8 confirms) that they are fair game for changes.
      https://peps.python.org/pep-0008/#public-and-internal-interfaces
    
    Usage examples:
    
    python#55454 (comment)
    
    Original discussion:
    
    https://discuss.python.org/t/gauging-interest-in-my-imap4-idle-implementation-for-imaplib/59272
    
    Earlier requests and suggestions:
    
    python#55454
    
    https://mail.python.org/archives/list/python-ideas@python.org/thread/C4TVEYL5IBESQQPPS5GBR7WFBXCLQMZ2/
    
    * pythongh-55454: Clarify imaplib idle() docs
    
    - Add example idle response tuples, to make the minor difference from other
      imaplib response tuples more obvious.
    - Merge the idle context manager's burst() method docs with the IMAP
      object's idle() method docs, for easier understanding.
    - Upgrade the Windows note regarding lack of pipe timeouts to a warning.
    - Rephrase various things for clarity.
    
    * docs: words instead of <=
    
    Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
    
    * docs: improve style in an example
    
    Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
    
    * docs: grammatical edit
    
    Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
    
    * docs consistency
    
    Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
    
    * comment -> docstring
    
    Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
    
    * docs: refer to imaplib as "this module"
    
    Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
    
    * imaplib: simplify & clarify idle debug message
    
    Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
    
    * imaplib: elaborate in idle context manager comment
    
    * imaplib: re-raise BaseException instead of bare except
    
    Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
    
    * imaplib: convert private doc string to comment
    
    * docs: correct mistake in imaplib example
    
    This is a correction to 8077f2e, which
    changed a variable name in only one place and broke the subsequent
    reference to it, departed from the naming convention used in the rest of
    the module, and shadowed the type() builtin along the way.
    
    * imaplib: simplify example code in doc string
    
    This is for consistency with the documentation change in 8077f2e
    and subsequent correction in 013bbf1.
    
    * imaplib: rename _Idler to Idler, update its docs
    
    * imaplib: add comment in Idler._pop()
    
    Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
    
    * imaplib: remove unnecessary blank line
    
    Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
    
    * imaplib: comment on use of unbuffered pipes
    
    * docs: imaplib: use the reStructuredText :class: role
    
    Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
    
    * Revert "docs: imaplib: use the reStructuredText :class: role"
    
    This reverts commit f385e44, because it
    triggers CI failures in the docs by referencing a class that is
    (deliberately) undocumented.
    
    * docs: imaplib: use the reST :class: role, escaped
    
    This is a different approach to f385e44, which was reverted for
    creating dangling link references.
    
    By prefixing the reStructuredText role target with a ! we disable
    conversion to a link, thereby passing continuous integration checks
    even though the referenced class is deliberately absent from the
    documentation.
    
    * docs: refer to IMAP4 IDLE instead of just IDLE
    
    This clarifies that we are referring to the email protocol, not the editor with the same name.
    
    Co-authored-by: Guido van Rossum <gvanrossum@gmail.com>
    
    * imaplib: IDLE -> IMAP4 IDLE in exception message
    
    Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
    
    * docs: imaplib idle() phrasing and linking tweaks
    
    * docs: imaplib: avoid linking to an invalid target
    
    This reverts and rephrases part of a3f21cd
    which created links to a method on a deliberately undocumented class.
    The links didn't work consistently, and caused sphinx warnings that
    broke cpython's continuous integration tests.
    
    * imaplib: update test after recent exception change
    
    This fixes a test that was broken by changing an exception in
    b01de95
    
    * imaplib: rename idle() dur argument to duration
    
    * imaplib: bytes.index() -> bytes.find()
    
    This makes it more obvious which statement triggers the branch.
    
    * imaplib: remove no-longer-necessary statement
    
    Co-authored-by: Martin Panter <vadmium@users.noreply.github.com>
    
    * docs: imaplib: concise & valid method links
    
    The burst() method is a little tricky to link in restructuredText, due
    to quirks of its parent class.  This syntax allows sphinx to generate
    working links without generating warnings (which break continuous
    integration) and without burdening the reader with unimportant namespace
    qualifications.  It makes the reST source ugly, but few people read
    the reST source, so it's a tolerable tradeoff.
    
    * imaplib: note data types present in IDLE responses
    
    * docs: imaplib: add comma to reST changes header
    
    Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
    
    * imaplib: sync doc strings with reST docs
    
    * docs: imaplib: minor Idler clarifications
    
    * imaplib: idle: emit (type, [data, ...]) tuples
    
    This allows our iterator to emit untagged responses that contain literal
    strings in the same way that imaplib's existing methods do, while still
    emitting exactly one whole response per iteration.
    
    * imaplib: while/yield instead of yield from iter()
    
    * imaplib: idle: use deadline idiom when iterating
    
    This simplifies the code, and avoids idle duration drift from time spent
    processing each iteration.
    
    * docs: imaplib: state duration/interval arg types
    
    * docs: imaplib: minor rephrasing of a sentence
    
    * docs: imaplib: reposition a paragraph
    
    This might improve readability, especially when encountering Idler.burst()
    for the first time.
    
    * docs: imaplib: wrap long lines in idle() section
    
    * docs: imaplib: note: Idler objects require 'with'
    
    * docs: imaplib: say that 29 minutes is 1740 seconds
    
    * docs: imaplib: mark a paragraph as a 'tip'
    
    * docs: imaplib: rephrase reference to MS Windows
    
    * imaplib: end doc string titles with a period
    
    * imaplib: idle: socket timeouts instead of select()
    
    IDLE timeouts were originally implemented using select() after
    checking for the presence of already-buffered data.
    That allowed timeouts on pipe connetions like IMAP4_stream.
    However, it seemed possible that SSL data arriving without any
    IMAP data afterward could cause select() to indicate available
    application data when there was none, leading to a read() call
    that would block with no timeout. It was unclear under what
    conditions this would happen in practice. This change switches
    to socket timeouts instead of select(), just to be safe.
    
    This also reverts IMAP4_stream changes that were made to support IDLE
    timeouts, since our new implementation only supports socket connections.
    
    * imaplib: Idler: rename private state attributes
    
    * imaplib: rephrase a comment in example code
    
    * docs: imaplib: idle: use Sphinx code-block:: pycon
    
    * docs: whatsnew: imaplib: reformat IMAP4.idle entry
    
    * imaplib: idle: make doc strings brief
    
    Since we generally rely on the reST/html documentation for details, we
    can keep these doc strings short. This matches the module's existing doc
    string style and avoids having to sync small changes between two files.
    
    * imaplib: Idler: split assert into two statements
    
    * imaplib: Idler: move assignment out of try: block
    
    * imaplib: Idler: move __exit__() for readability
    
    * imaplib: Idler: move __next__() for readability
    
    * imaplib: test: make IdleCmdHandler a global class
    
    * docs: imaplib: idle: collapse double-spaces
    
    * imaplib: warn on use of undocumented 'file' attr
    
    * imaplib: revert import reformatting
    
    Since we no longer import platform or selectors, the original import
    statement style can be restored, reducing the footprint of PR python#122542.
    
    * imaplib: restore original exception msg formatting
    
    This reduces the footprint of PR python#122542.
    
    * docs: imaplib: idle: versionadded:: next
    
    * imaplib: move import statement to where it's used
    
    This import is only needed if external code tries to use an attribute
    that it shouldn't be using. Making it a local import reduces module
    loading time in supported cases.
    
    * imaplib test: RuntimeWarning on IMAP4.file access
    
    * imaplib: use stacklevel=2 in warnings.warn()
    
    * imaplib test: simplify IMAP4.file warning test
    
    * imaplib test: pre-idle-continuation response
    
    * imaplib test: post-done untagged response
    
    * imaplib: downgrade idle-denied exception to error
    
    This makes it easier for client code to distinguish a temporary
    rejection of the IDLE command from a server responding incorrectly to
    IDLE.
    
    * imaplib: simplify check for socket object
    
    * imaplib: narrow the scope of IDLE socket timeouts
    
    If an IDLE duration or burst() was in use, and an unsolicited response
    contained a literal string, and crossed a packet boundary, and the
    subsequent packet was delayed beyond the IDLE feature's time limit, the
    timeout would leave the incoming protocol stream in a bad state (with
    the tail of that response appearing where the start of a response is
    expected).
    
    This change moves the IDLE socket timeout to cover only the start
    of a response, so it can no longer cause that problem.
    
    * imaplib: preserve partial reads on exception
    
    This ensures that short IDLE durations / burst() intervals
    won't risk corrupting response lines that span multiple packets.
    
    * imaplib: read/readline: save multipart buffer tail
    
    For resilience if read() or readline() ever complete with more than one
    bytes object remaining in the buffer. This is not expected to happen,
    but it seems wise to be prepared for a future change making it possible.
    
    * imaplib: use TimeoutError subclass only if needed
    
    * doc: imaplib: elaborate on IDLE response delivery
    
    * doc: imaplib: elaborate in note re: IMAP4.response
    
    * imaplib: comment on benefit of reading in chunks
    
    Our read() implementation designed to support IDLE replaces the one from
    PR python#119514, fixing the same problem it was addressing. The tests that it
    added are preserved.
    
    * imaplib: readline(): treat ConnectionError as EOF
    
    ---------
    
    Co-authored-by: Gregory P. Smith <greg@krypto.org>
    Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
    Co-authored-by: Guido van Rossum <guido@python.org>
    Co-authored-by: Guido van Rossum <gvanrossum@gmail.com>
    Co-authored-by: Martin Panter <vadmium@users.noreply.github.com>
    Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
    cmaloney pushed a commit to cmaloney/cpython that referenced this issue Feb 8, 2025
    * pythongh-55454: Add IMAP4 IDLE support to imaplib
    
    This extends imaplib with support for the rfc2177 IMAP IDLE command,
    as requested in python#55454.  It allows events to be pushed to a client as
    they occur, rather than having to continually poll for mailbox changes.
    
    The interface is a new idle() method, which returns an iterable context
    manager.  Entering the context starts IDLE mode, during which events
    (untagged responses) can be retrieved using the iteration protocol.
    Exiting the context sends DONE to the server, ending IDLE mode.
    
    An optional time limit for the IDLE session is supported, for use with
    servers that impose an inactivity timeout.
    
    The context manager also offers a burst() method, designed for programs
    wishing to process events in batch rather than one at a time.
    
    Notable differences from other implementations:
    
    - It's an extension to imaplib, rather than a replacement.
    - It doesn't introduce additional threads.
    - It doesn't impose new requirements on the use of imaplib's existing methods.
    - It passes the unit tests in CPython's test/test_imaplib.py module
      (and adds new ones).
    - It works on Windows, Linux, and other unix-like systems.
    - It makes IDLE available on all of imaplib's client variants
      (including IMAP4_stream).
    - The interface is pythonic and easy to use.
    
    Caveats:
    
    - Due to a Windows limitation, the special case of IMAP4_stream running
      on Windows lacks a duration/timeout feature. (This is the stdin/stdout
      pipe connection variant; timeouts work fine for socket-based
      connections, even on Windows.) I have documented it where appropriate.
    
    - The file-like imaplib instance attributes are changed from buffered to
      unbuffered mode. This could potentially break any client code that
      uses those objects directly without expecting partial reads/writes.
      However, these attributes are undocumented. As such, I think (and
      PEP 8 confirms) that they are fair game for changes.
      https://peps.python.org/pep-0008/#public-and-internal-interfaces
    
    Usage examples:
    
    python#55454 (comment)
    
    Original discussion:
    
    https://discuss.python.org/t/gauging-interest-in-my-imap4-idle-implementation-for-imaplib/59272
    
    Earlier requests and suggestions:
    
    python#55454
    
    https://mail.python.org/archives/list/python-ideas@python.org/thread/C4TVEYL5IBESQQPPS5GBR7WFBXCLQMZ2/
    
    * pythongh-55454: Clarify imaplib idle() docs
    
    - Add example idle response tuples, to make the minor difference from other
      imaplib response tuples more obvious.
    - Merge the idle context manager's burst() method docs with the IMAP
      object's idle() method docs, for easier understanding.
    - Upgrade the Windows note regarding lack of pipe timeouts to a warning.
    - Rephrase various things for clarity.
    
    * docs: words instead of <=
    
    Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
    
    * docs: improve style in an example
    
    Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
    
    * docs: grammatical edit
    
    Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
    
    * docs consistency
    
    Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
    
    * comment -> docstring
    
    Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
    
    * docs: refer to imaplib as "this module"
    
    Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
    
    * imaplib: simplify & clarify idle debug message
    
    Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
    
    * imaplib: elaborate in idle context manager comment
    
    * imaplib: re-raise BaseException instead of bare except
    
    Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
    
    * imaplib: convert private doc string to comment
    
    * docs: correct mistake in imaplib example
    
    This is a correction to 8077f2e, which
    changed a variable name in only one place and broke the subsequent
    reference to it, departed from the naming convention used in the rest of
    the module, and shadowed the type() builtin along the way.
    
    * imaplib: simplify example code in doc string
    
    This is for consistency with the documentation change in 8077f2e
    and subsequent correction in 013bbf1.
    
    * imaplib: rename _Idler to Idler, update its docs
    
    * imaplib: add comment in Idler._pop()
    
    Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
    
    * imaplib: remove unnecessary blank line
    
    Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
    
    * imaplib: comment on use of unbuffered pipes
    
    * docs: imaplib: use the reStructuredText :class: role
    
    Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
    
    * Revert "docs: imaplib: use the reStructuredText :class: role"
    
    This reverts commit f385e44, because it
    triggers CI failures in the docs by referencing a class that is
    (deliberately) undocumented.
    
    * docs: imaplib: use the reST :class: role, escaped
    
    This is a different approach to f385e44, which was reverted for
    creating dangling link references.
    
    By prefixing the reStructuredText role target with a ! we disable
    conversion to a link, thereby passing continuous integration checks
    even though the referenced class is deliberately absent from the
    documentation.
    
    * docs: refer to IMAP4 IDLE instead of just IDLE
    
    This clarifies that we are referring to the email protocol, not the editor with the same name.
    
    Co-authored-by: Guido van Rossum <gvanrossum@gmail.com>
    
    * imaplib: IDLE -> IMAP4 IDLE in exception message
    
    Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
    
    * docs: imaplib idle() phrasing and linking tweaks
    
    * docs: imaplib: avoid linking to an invalid target
    
    This reverts and rephrases part of a3f21cd
    which created links to a method on a deliberately undocumented class.
    The links didn't work consistently, and caused sphinx warnings that
    broke cpython's continuous integration tests.
    
    * imaplib: update test after recent exception change
    
    This fixes a test that was broken by changing an exception in
    b01de95
    
    * imaplib: rename idle() dur argument to duration
    
    * imaplib: bytes.index() -> bytes.find()
    
    This makes it more obvious which statement triggers the branch.
    
    * imaplib: remove no-longer-necessary statement
    
    Co-authored-by: Martin Panter <vadmium@users.noreply.github.com>
    
    * docs: imaplib: concise & valid method links
    
    The burst() method is a little tricky to link in restructuredText, due
    to quirks of its parent class.  This syntax allows sphinx to generate
    working links without generating warnings (which break continuous
    integration) and without burdening the reader with unimportant namespace
    qualifications.  It makes the reST source ugly, but few people read
    the reST source, so it's a tolerable tradeoff.
    
    * imaplib: note data types present in IDLE responses
    
    * docs: imaplib: add comma to reST changes header
    
    Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
    
    * imaplib: sync doc strings with reST docs
    
    * docs: imaplib: minor Idler clarifications
    
    * imaplib: idle: emit (type, [data, ...]) tuples
    
    This allows our iterator to emit untagged responses that contain literal
    strings in the same way that imaplib's existing methods do, while still
    emitting exactly one whole response per iteration.
    
    * imaplib: while/yield instead of yield from iter()
    
    * imaplib: idle: use deadline idiom when iterating
    
    This simplifies the code, and avoids idle duration drift from time spent
    processing each iteration.
    
    * docs: imaplib: state duration/interval arg types
    
    * docs: imaplib: minor rephrasing of a sentence
    
    * docs: imaplib: reposition a paragraph
    
    This might improve readability, especially when encountering Idler.burst()
    for the first time.
    
    * docs: imaplib: wrap long lines in idle() section
    
    * docs: imaplib: note: Idler objects require 'with'
    
    * docs: imaplib: say that 29 minutes is 1740 seconds
    
    * docs: imaplib: mark a paragraph as a 'tip'
    
    * docs: imaplib: rephrase reference to MS Windows
    
    * imaplib: end doc string titles with a period
    
    * imaplib: idle: socket timeouts instead of select()
    
    IDLE timeouts were originally implemented using select() after
    checking for the presence of already-buffered data.
    That allowed timeouts on pipe connetions like IMAP4_stream.
    However, it seemed possible that SSL data arriving without any
    IMAP data afterward could cause select() to indicate available
    application data when there was none, leading to a read() call
    that would block with no timeout. It was unclear under what
    conditions this would happen in practice. This change switches
    to socket timeouts instead of select(), just to be safe.
    
    This also reverts IMAP4_stream changes that were made to support IDLE
    timeouts, since our new implementation only supports socket connections.
    
    * imaplib: Idler: rename private state attributes
    
    * imaplib: rephrase a comment in example code
    
    * docs: imaplib: idle: use Sphinx code-block:: pycon
    
    * docs: whatsnew: imaplib: reformat IMAP4.idle entry
    
    * imaplib: idle: make doc strings brief
    
    Since we generally rely on the reST/html documentation for details, we
    can keep these doc strings short. This matches the module's existing doc
    string style and avoids having to sync small changes between two files.
    
    * imaplib: Idler: split assert into two statements
    
    * imaplib: Idler: move assignment out of try: block
    
    * imaplib: Idler: move __exit__() for readability
    
    * imaplib: Idler: move __next__() for readability
    
    * imaplib: test: make IdleCmdHandler a global class
    
    * docs: imaplib: idle: collapse double-spaces
    
    * imaplib: warn on use of undocumented 'file' attr
    
    * imaplib: revert import reformatting
    
    Since we no longer import platform or selectors, the original import
    statement style can be restored, reducing the footprint of PR python#122542.
    
    * imaplib: restore original exception msg formatting
    
    This reduces the footprint of PR python#122542.
    
    * docs: imaplib: idle: versionadded:: next
    
    * imaplib: move import statement to where it's used
    
    This import is only needed if external code tries to use an attribute
    that it shouldn't be using. Making it a local import reduces module
    loading time in supported cases.
    
    * imaplib test: RuntimeWarning on IMAP4.file access
    
    * imaplib: use stacklevel=2 in warnings.warn()
    
    * imaplib test: simplify IMAP4.file warning test
    
    * imaplib test: pre-idle-continuation response
    
    * imaplib test: post-done untagged response
    
    * imaplib: downgrade idle-denied exception to error
    
    This makes it easier for client code to distinguish a temporary
    rejection of the IDLE command from a server responding incorrectly to
    IDLE.
    
    * imaplib: simplify check for socket object
    
    * imaplib: narrow the scope of IDLE socket timeouts
    
    If an IDLE duration or burst() was in use, and an unsolicited response
    contained a literal string, and crossed a packet boundary, and the
    subsequent packet was delayed beyond the IDLE feature's time limit, the
    timeout would leave the incoming protocol stream in a bad state (with
    the tail of that response appearing where the start of a response is
    expected).
    
    This change moves the IDLE socket timeout to cover only the start
    of a response, so it can no longer cause that problem.
    
    * imaplib: preserve partial reads on exception
    
    This ensures that short IDLE durations / burst() intervals
    won't risk corrupting response lines that span multiple packets.
    
    * imaplib: read/readline: save multipart buffer tail
    
    For resilience if read() or readline() ever complete with more than one
    bytes object remaining in the buffer. This is not expected to happen,
    but it seems wise to be prepared for a future change making it possible.
    
    * imaplib: use TimeoutError subclass only if needed
    
    * doc: imaplib: elaborate on IDLE response delivery
    
    * doc: imaplib: elaborate in note re: IMAP4.response
    
    * imaplib: comment on benefit of reading in chunks
    
    Our read() implementation designed to support IDLE replaces the one from
    PR python#119514, fixing the same problem it was addressing. The tests that it
    added are preserved.
    
    * imaplib: readline(): treat ConnectionError as EOF
    
    ---------
    
    Co-authored-by: Gregory P. Smith <greg@krypto.org>
    Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
    Co-authored-by: Guido van Rossum <guido@python.org>
    Co-authored-by: Guido van Rossum <gvanrossum@gmail.com>
    Co-authored-by: Martin Panter <vadmium@users.noreply.github.com>
    Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    stdlib Python modules in the Lib dir topic-email type-feature A feature request or enhancement
    Projects
    None yet
    Development

    No branches or pull requests

    7 participants