bpo-12887 and bpo-14345: Document the availability and uses of the SO_ constants. Document SOL_SOCKET #3072

ammaraskar · 2017-08-11T05:20:35Z

The main thing I'm looking for feedback on is how to present information on the main OSs these options are available on. My current approach is to visualize it in a table that looks like this:

The alternative is to document it with each constant which looks like this:

However I think this starts to get very repetitive and doesn't look nearly as nice as the table. It does have the advantage that you don't have to scroll back up to the table to see if a certain option is available on your OS, you can see the availability as you read the documentation for the option. (I've commented those out until I make a decision on which approach is better, table or inline)

I've added fairly brief descriptions of each constant, they're ordered in the order they appear in https://github.com/python/cpython/blob/master/Modules/socketmodule.c
The descriptions have been written by consulting man pages and experimentation and any feedback on improving them would be appreciated.

Some of them have vary scant documentation online such as SO_PASSSEC so I've opted to keep those super brief. Hopefully they can be improved over time by the people who added them or otherwise.

https://bugs.python.org/issue12887

bpo-14345: Document SOL_SOCKET

vadmium · 2017-08-12T02:53:42Z

Doc/library/socket.rst

+
+   When a socket is set to linger. A call to :meth:`socket.close` or
+   :meth:`socket.shutdown` will not return until all queued messages
+   for the socket have been successfully sent or the linger timeout


First sentence does not make sense. Perhaps you should have a comma (,) instead of the full stop (.)?

vadmium · 2017-08-12T02:56:00Z

Doc/library/socket.rst

+.. data:: SO_DONTROUTE
+
+   Indicates that packets sent through this socket should be routed
+   through the interface its bound to.


It’s or it is

vadmium · 2017-08-12T03:29:11Z

Doc/library/socket.rst

+
+   SO_RCVLOWAT sets the minimum number of bytes that must be present in
+   the socket's internal receive buffer before they are passed on to able
+   read call. SO_SNDLOWAT similiary sets the minimum bytes before data is


I don’t understand “passed on to able read call”. Perhaps do you mean something like “before the socket becomes readable” or “before they are returned by a read”?

Spelling: similarly

vadmium · 2017-08-12T03:49:42Z

Doc/library/socket.rst

+   SO_SNDLOWAT is read-only on Linux and SO_RCVLOWAT is read-only on
+   Linux versions below 2.4.
+
+   Both these values default to 1.


According to Posix, the SNDLOWAT default depends on the implementation and protocol.

vadmium · 2017-08-12T04:05:29Z

Doc/library/socket.rst

+          SO_SNDTIMEO
+
+   Specifies the amount of time send and receive calls for this socket willl
+   block before timing out. The default timeout of zero means that operations


Extra l in will.

Is it worth clarifying that these timeouts (which raise EAGAIN etc) are independent of the settimeout mechanism (which raises socket.timeout for non-zero timeouts)?

vadmium · 2017-08-12T05:13:02Z

Doc/library/socket.rst

+   When bound, only packets received from that particular device are
+   processsed by the socket.
+
+   .. versionadded:: 3.1


Extra s in processed.

According to my patch at https://bugs.python.org/issue27409 this was only added in 3.3 (or did I make a mistake?). See also https://hg.python.org/cpython/rev/6159311f0f44.

Oh I came to the 3.1 number by going to https://github.com/python/cpython/blob/master/Modules/socketmodule.c

Clicking the blame button and checking the tag in the commit. The tag says 3.1 but it looks like that is incorrect, since the README at the time says: This is Python version 3.3 alpha 0

vadmium · 2017-08-12T05:43:02Z

Doc/library/socket.rst

+
+   Passing ``SO_DOMAIN`` to :meth:`socket.getsockopt` allows for the retrival
+   of the ``family`` value as defined in the :func:`socket.socket` function.
+   ``SO_PROTOCOL`` returns the ``proto`` value. Both these options are read only.


Actually it can return the real protocol, e.g. IPPROTO_TCP, even when proto is zero.

vadmium · 2017-08-12T05:48:54Z

Doc/library/socket.rst

+   ``SO_PROTOCOL`` returns the ``proto`` value. Both these options are read only.
+
+   The value returned for the ``family`` is an integer and not one of the
+   friendly constants above like :const:`AF_INET`. In order to get a constant


The integer value normally does equal one of those enumeration members; it is just that the integer object is never an enumeration member.

vadmium · 2017-08-12T07:00:54Z

Doc/library/socket.rst

+          SO_PEERCRED
+
+   Allows for the passing of SCM credentials over unix sockets.
+   See the end of :func:`socket.recvmsg` for details.


I’m not familiar with either API, but it seems this only applies to PASSCRED, and PEERCRED returns credentials straight away.

There seems to be something missing with the reference to recvmsg. The end of that entry covers Python versions, platforms, and before that, receiving file descriptors with SCM_RIGHTS. But as I understand it, when using PASSCRED you would look for SCM_CREDENTIALS messages, with a different data structure. The first couple of paragraphs, about receiving anciliary messages in general, are more relevant.

Yeah I'm not familiar with this API either, I'm just going to add an easily searchable keyword of ancillary message and point to recvmsg

vadmium · 2017-08-12T11:13:16Z

Doc/library/socket.rst

+
+   .. versionadded:: 3.6
+
+..   Availability: Linux >= 2.6.13


Where does this come from? What I found was SO_PASSSEC added to Linux 2.6.18 (https://repo.or.cz/linux-2.6-linus.git/commitdiff/877ce7c), and SO_PEERSEC to 2.6.2 (schwabe/tglx-history@da6e57a).

Aah that's my mistake. I think 2.6.2 is the right number to include here. I came up with 2.6.13 because I was looking at the blame for sock.c here: https://github.com/torvalds/linux/blame/add459aa1afe05472abc96f6a29aefd0c84e73d6/net/core/sock.c

I scrolled to

case SO_PEERSEC: return security_socket_getpeersec_stream(sock, optval, optlen, len);

and saw "Linux-2.6.12-rc2" in the commit message. But it looks like that was a big commit for when Linux switched from bitkeeper to git.

But if we look at the commit that adds

case SO_PASSSEC: v.val = test_bit(SOCK_PASSSEC, &sock->flags) ? 1 : 0; break;

the commit message says "[AF_UNIX]: Datagram getpeersec" and we can find the same commit in the 2.6.2 changelogs here: https://www.kernel.org/pub/linux/kernel/v2.6/ChangeLog-2.6.2

So I think its safe to say both of these options are available on >= 2.6.2

pitrou · 2017-08-24T15:55:23Z

Hi @ammaraskar! I appreciate the level of work you've put into this. On the other hand, I wonder about the maintenance cost of updating this each time an OS gains support for one those options.
(also, the OS list by far not exhaustive, for example we would certainly want to add an OS X column -- or is it "macOS" these days?)

My personal take would be that people who use the (rather low-level) socket module should know how to check man pages and/or the MSDN documentation for the presence and meaning of specific options. Higher-level alternatives are the various networking frameworks (blocking or non-blocking) and the existing implementations of applicative protocols such as http.

Not sure what others think about this.

ammaraskar · 2017-08-27T18:22:46Z

On the other hand, I wonder about the maintenance cost of updating this each time an OS gains support for one those options.

I think this will be far and in between, the core socket options are usually a fairly stable OS level API. With the current ones documented, it should hopefully encourage maintainers to document newly added ones and then check their availability accordingly.

As far as an OS gaining support, I think that will be fairly rare but in that case having the documentation being slightly outdated wouldn't be the worst of situations.

There is some older discussion about this on the bpo thread.

My personal take would be that people who use the (rather low-level) socket module should know how to check man pages and/or the MSDN documentation for the presence and meaning of specific options.

I agree, however there is no real documentation on what constants python exposes from the underlying socket library. At the very minimum I think documenting the list of options and brief descriptions of them would be a good idea. Short of doing dir(socket) there is no way to examine what is available.

matrixise · 2018-01-30T10:11:13Z

Hi,

@pitrou you are right about the list of the options, we have to check in the official documentation (man pages or MSDN site) but this table with some references can be useful for the beginners, and also for the maintainer, because we can check if an option is directly supported by Python or not.

@ammaraskar
Maybe you could add "This list is incomplete exhaustive and you should to read the documentation of your operating system" and also, add a column for macOS.
But don't forget that we support some operating systems.

ammaraskar · 2018-01-30T17:04:22Z

At the very least, I think adding documentation for the list of options that socketmodule exports and getsockopt/setsockopt would be useful. I agree that the compatibility table is a bit of a pain to maintain.

matrixise · 2018-11-04T09:28:37Z

@ammaraskar How did you find the entries for your table. I propose another solution, for the most used entries, we could keep them in the tables. Add a small paragraph with the link to the official documentation of macOS, Windows and for the man pages for POSIX systems.

ammaraskar · 2018-11-04T16:20:18Z

I pretty much just went down the list here:

cpython/Modules/socketmodule.c

Line 7217 in e359bc2

#ifdef SOCK_RAW

Then I looked them up in the man page here for Linux: http://man7.org/linux/man-pages/man7/socket.7.html

FreeBSD here: https://www.freebsd.org/cgi/man.cgi?query=setsockopt&sektion=2&apropos=0&manpath=FreeBSD+7.0-RELEASE

and finally Windows here: https://docs.microsoft.com/en-us/windows/desktop/WinSock/sol-socket-socket-options

The availabilities took a bit more digging up but that was the basic idea.

vstinner · 2019-01-10T16:00:57Z

I removed the " needs backport to 3.6" label, the 3.6 branch no longer accept bugfixes (only security fixes are accepted): https://devguide.python.org/#status-of-python-branches

JulienPalard · 2019-09-10T12:16:53Z

At the very least, I think adding documentation for the list of options that socketmodule exports and getsockopt/setsockopt would be useful. I agree that the compatibility table is a bit of a pain to maintain.

I agree. In one hand I like the table (not having to search the info for an OS you don't know is nice), but it's probably too much work to maintain. Let's just start by documenting the exposed options. @ammaraskar can you remove the compatibility list and fix the previous reviews about?

csabella · 2020-02-04T01:00:48Z

@ammaraskar, please address the review comments.

github-actions · 2022-02-21T00:08:33Z

This PR is stale because it has been open for 30 days with no activity.

JelleZijlstra · 2022-05-08T21:22:47Z

Closing as it's been more than two years since changes were requested. Feel free to reopen or submit a new PR.

the-knights-who-say-ni added the CLA signed label Aug 11, 2017

ammaraskar force-pushed the socket_docs branch from b72a80d to ebab99f Compare August 11, 2017 05:25

bpo-12887: Document the availability and uses of the SO_ constants.

73cc260

bpo-14345: Document SOL_SOCKET

ammaraskar force-pushed the socket_docs branch from ebab99f to 73cc260 Compare August 11, 2017 05:31

ammaraskar mentioned this pull request Aug 11, 2017

Only run CI checks when appropriate files have changed python/core-workflow#14

Closed

vadmium reviewed Aug 12, 2017

View reviewed changes

ammaraskar added 2 commits August 12, 2017 09:34

Fix simple issues found in code review

32ee0de

Fix Linux availability version for SO_PEERSEC

86cece8

ammaraskar force-pushed the socket_docs branch from 90ef897 to 86cece8 Compare August 12, 2017 20:54

Mariatta added needs backport to 3.6 docs Documentation in the Doc dir labels Aug 22, 2017

brettcannon added the awaiting review label Jan 12, 2018

serhiy-storchaka added the needs backport to 3.7 label Dec 10, 2018

vstinner removed the needs backport to 3.6 label Jan 10, 2019

csabella added awaiting changes needs backport to 3.8 and removed awaiting review labels Feb 4, 2020

ned-deily removed the needs backport to 3.7 label Jul 3, 2020

ambv removed the needs backport to 3.8 label May 4, 2021

github-actions bot added the stale Stale PR or inactive for long period of time. label Feb 21, 2022

JelleZijlstra closed this May 8, 2022

ukarroum mentioned this pull request Apr 20, 2025

gh-45048: Document the availability and uses of the SO_ constants #132752

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

bpo-12887 and bpo-14345: Document the availability and uses of the SO_ constants. Document SOL_SOCKET #3072

bpo-12887 and bpo-14345: Document the availability and uses of the SO_ constants. Document SOL_SOCKET #3072

ammaraskar commented Aug 11, 2017 •

edited

Loading

vadmium Aug 12, 2017

vadmium Aug 12, 2017

vadmium Aug 12, 2017

vadmium Aug 12, 2017

vadmium Aug 12, 2017

vadmium Aug 12, 2017

ammaraskar Aug 12, 2017

vadmium Aug 12, 2017

vadmium Aug 12, 2017

vadmium Aug 12, 2017

ammaraskar Aug 12, 2017

vadmium Aug 12, 2017

ammaraskar Aug 12, 2017

pitrou commented Aug 24, 2017 •

edited

Loading

ammaraskar commented Aug 27, 2017

matrixise commented Jan 30, 2018

ammaraskar commented Jan 30, 2018

matrixise commented Nov 4, 2018

ammaraskar commented Nov 4, 2018

vstinner commented Jan 10, 2019

JulienPalard commented Sep 10, 2019

csabella commented Feb 4, 2020

github-actions bot commented Feb 21, 2022

JelleZijlstra commented May 8, 2022

bpo-12887 and bpo-14345: Document the availability and uses of the SO_ constants. Document SOL_SOCKET #3072

bpo-12887 and bpo-14345: Document the availability and uses of the SO_ constants. Document SOL_SOCKET #3072

Conversation

ammaraskar commented Aug 11, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pitrou commented Aug 24, 2017 • edited Loading

ammaraskar commented Aug 27, 2017

matrixise commented Jan 30, 2018

ammaraskar commented Jan 30, 2018

matrixise commented Nov 4, 2018

ammaraskar commented Nov 4, 2018

vstinner commented Jan 10, 2019

JulienPalard commented Sep 10, 2019

csabella commented Feb 4, 2020

github-actions bot commented Feb 21, 2022

JelleZijlstra commented May 8, 2022

ammaraskar commented Aug 11, 2017 •

edited

Loading

pitrou commented Aug 24, 2017 •

edited

Loading