DOC: Add examples for setbufsize & getbufsize #22727
Conversation
There was a problem hiding this comment.
The results of these are dependent on the OS, and CI failures are indicative of this. MacOS Python 3.8 on Azure reports:
numpy.core.setbufsize
---------------------
File "build/testenv/lib/python3.8/site-packages/numpy/__init__.py", line 199, in setbufsize
Failed example:
np.getbufsize()
Expected:
8192
Got:
20000
File "build/testenv/lib/python3.8/site-packages/numpy/__init__.py", line 201, in setbufsize
Failed example:
np.setbufsize(4096)
Expected:
8192
Got:
20000
numpy.core.getbufsize
---------------------
File "build/testenv/lib/python3.8/site-packages/numpy/__init__.py", line 232, in getbufsize
Failed example:
np.getbufsize()
Expected:
8192
Got:
4096
|
I didn't realize the examples are actually tested, thanks for pointing it out. |
Rather than closing this perhaps just marking the outputs as "to be skipped", and mention the OS and indicative output in a comment? refguide can be signalled to skip testing code by adding
``#doctest: +SKIP`` to the end of the line. If the output varies or is
random, add ``# may vary`` or ``# random`` to the comment. for example
>>> plt.plot(...) # doctest: +SKIP
>>> random.randint(0,10)
5 # random |
|
I looked at this PR as I've often wondered whether the default buffer sizes are close to optimal for some of my more performance critical code, and was a little disappointed here to only see examples added of usage, which is really quite trivial (though good to mention it has to be a multiple of 16 bytes). Maybe it is still useful, but that in that case probably best to set back the buffer size in the example -- as is, after the testing code passes through this example, the buffer size is now different for all further tests it does. |
| >>> original_size = np.getbufsize() | ||
| >>> np.setbufsize(4096) == original_size | ||
| True | ||
| >>> np.getbufsize() | ||
| 4096 |
There was a problem hiding this comment.
| >>> original_size = np.getbufsize() | |
| >>> np.setbufsize(4096) == original_size | |
| True | |
| >>> np.getbufsize() | |
| 4096 | |
| >>> original_size = np.getbufsize() | |
| >>> np.setbufsize(4096) | |
| >>> np.getbufsize() | |
| 4096 | |
| >>> np.setbufsize(original_size) |
There was a problem hiding this comment.
The == test doesn't add anything and is potentially confusing. Also resets the size as @mhvk suggested.
| Examples | ||
| -------- | ||
| >>> np.getbufsize() # doctest: +SKIP | ||
| 8192 # may vary |
There was a problem hiding this comment.
| 8192 # may vary | |
| 8192 # varies by OS |
There was a problem hiding this comment.
It doesn't vary by OS, the variations seem due to the test run itself, I think. The typing tests are slow and don't run everwywhere, and set the bufsize to 4096 without setting it back. The 20000 is/was maybe also the doc tests running twice or so?
I am not sure when to modify buffer sizes best. There is an older issue I saw recently, that suggested we should just make them generally smaller, also to better use CPU caches (I guess times change)...
There was a problem hiding this comment.
I realize I don't know enough about the use cases of these functions to meaningfully improve the docs atm. If I manage to find an example that demonstrates the impact on performance, I'll open another PR. Feel free to close this one, unless you want to discuss this further.
|
i think this would be more convincing with guidance when to change the buffer size. Is there some explanation elsewhere as to why one would want to change it? |
|
Its too bad, but I suppose we should close this eve though there are some nice starts smaller things in here. If anyone wants to continue, please do not hesitate, or I am happy to reopen again. (Which btw. could also lead to improving the buffer sizes ;)). |
|
@jnclt Thank you for giving it a go, Lefteris! Feel free to work on another “sprintable” issue that peaks your interest. |
Add examples for getbufsize and setbufsize as requested by #22266.
Written as part of PyData 2022 sprint.