class io.BufferedIOBase states "In addition, those methods [read(), readinto() and write()] can raise BlockingIOError if the underlying raw stream is in non-blocking mode and cannot take or give enough data; unlike their RawIOBase counterparts, they will never return None."

However, class.io.BufferedReader (inheriting from io.BufferedIOBase) *does* return None in this case. Admittedly, io.BufferedReader does says it is overriding the inherited method, but I'm surprised that change in behaviour declared for buffered objects, is reverted to the RarIOBase behaviour on a more specific class.

The attached file (a little long - sorry), simulates a slow non-blocking raw file, which it wraps in a BufferReader to test the behaviour defined in BufferedIOBase.

The description of read in io.BufferedReader.read function states "Read and return size bytes, or if size is not given or negative, until EOF or if the read call would block in non-blocking mode." This does mention the non-block mode scenario, but I can't parse this sentence.

This is covered by bpo-13322. There are a few other BufferedReader methods that contradict the documentation for non-blocking mode.

A while ago I posted a patch to change the implementation to match the documentation, but nobody reviewed it or gave their opinion. These days I would prefer to just documentat the reality: the methods might raise an exception rather than returning None, or perhaps no particular behaviour at all is expected in general in the non-blocking case. But I don’t spend much time on Python now, so you might have to find someone else to move this forward.

Regarding the entry for “BufferedReader.read”, it would make more sense to remove the last “if”: “if ‘size’ is not given . . ., until EOF, or the ‘read’ call would block”. But I don’t think even that is complete, because it should also say the read stops and returns short in the non-blocking case even when “size” is positive.

I looked at changing this and found three cases reading through the BufferedReader code:

1. BufferedReader.read(size=10) underlying raw.read(size=10) returns None
2. BufferedReader.read() without size, raw implements readall() which returns None
3. BufferedReader.read() without size, raw doesn't have readall so uses a loop that calls raw.read() which returns None.

All three of these cases consistently return None from the BufferedReader. Definitely could change that behavior, but worry that in the 6 years since this issue was opened more code exists that expects None than have implemented and tested behavior for a BlockingIOError.

Could potentially add a new mode/flag which would cause it to raise, but Blocking I/O behavior is not well specified in the io library above the "Raw I/O" layer (https://peps.python.org/pep-3116/#non-blocking-i-o). Could definitely be evolved, but I think that would need first a Python Ideas discussion.

I do think the documentation should be updated to include the None return to reduce surprise. I'm actively working on that.

@vadmium , @steverpalmer the docs are now updated to match implementation behavior, can this be closed?

In short - maybe.

I understand why the preference is not to change the behaviour of the code given its flying hours and exposure. It may well be that some code out in the wild has been written around the actual behaviour rather than the previously described behaviour. The corollary to this though is that the description can become complex and there needs to be caution in offering simplified summaries.

(I'm quoting from https://docs.python.org/3/library/io.html#io.BufferedIOBase)

The summary description in io.BufferedIOBase states:

In addition, if the underlying raw stream is in non-blocking mode, when the system returns would block write() will raise BlockingIOError with BlockingIOError.characters_written and read() will return data read so far or None if no data is available.

The detailed description in io.BufferedIOBase.read(...) states:

Note: When the underlying raw stream is non-blocking, implementations may either raise BlockingIOError or return None if no data is available. io implementations return None.

The first quote seems to suggest that, in the case under consideration, the read function will return None is no data is available, but the later quote extends this to allow that it might alternatively raise a BlockingIOError, but not by the io module. The two quotes taken together might be confusing.

I wondered about editing the first quote to match the second, but that felt like duplication and too much detail for a summary. It also seems superfluous to change the first quote (summary) to refer to the second quote (detail). I then wondered about removing the first quote completely - what does it really add? I suspect that it was originally written about a perfectly reasonable, arguably better design intention, but as we now know this intention was carried inconsistently to the implementation. Oddly, the one thing that the first quote does add, not included in v3.12 of the document and that isn't detailed later in the function detail, is that the io.BufferedIOBase.write function BlockingIOError exception includes the charaters_written field.

My suggestion, then. is to delete the first summary quote entirely; the actual behaviour being described is too complex to simplify in a summary. I would also suggest that the io.BufferedIOBase.write function be extended to reference the BlockingIOError.charaters_written field (as it has been in io.BufferedWriter.write). Perhaps:

When in non-blocking mode, a BlockingIOError with BlockingIOError.characters_written is raised if the data needed to be written to the raw stream but it couldn’t accept all the data without blocking.

Nevertheless, the new text is better than the v3.12 text which is simply incorrect. Additionally. the reference from io.BufferedReader.read back to io.BufferedIOBasse.read is also better. If I am being pedantic, please feel free to ignore me.

re: write, slowly working my way through different docs functions :). FileIO and RawIOBase have some corner cases for just .read that don't currently match the docs (gh-129011), but also the behavior has not precisely matched the docs since the first release of the io code shipped over 15 years ago as far as I can tell in my repository spelunking. Without general agreement on what the code actually does, and what is an understandable, precise, and concise way of documenting it I don't see rewriting more being valuable. To me need to get consensus on one as a model for more :).

re: BufferedIOBase.read I get it's annoying to write both behaviors in the detailed doc still. Unfortunately things which followed the docs as they were before may exist, and there is nothing CPython can really do about that; Python doesn't enforce abstract API / interface shapes in that way. The note is purely a "the old docs said this, and others may have implemented it". Definitely get frustrating, but removing entirely (pretending the docs never said that) doesn't seem right to me nor does including in the main body of the doc (CPython implementation never did that). If you're using CPython and reading CPython's docs, the main paragraph behavior says what the code actually does, the notes give you details on how to be extra defensive in your programming to handle all possible cases that may exist in the Python ecosystem. My hope is the common case (you're just using read with a normal file and blocking I/O) is the very first thing documented and the rest is "skippable" details. For someone new to Python not documenting read() at all when people get it commonly from open(filename, 'rb') doesn't seem like enough... Could maybe drop blocking section entirely, but it's behavior which is used/relied upon today (even by code in CPython for specific cases).

In any case, glad you find better. Will experiment some more on improving further.

Thank you for your consideration of my comments. Best of luck with improving the documentation. I'm happy if you want to close this issue. Cheers.