Skip to content

gh-61460: Add a comment describing the multiprocessing.connection protocol #99623

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Nov 20, 2022
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
68 changes: 68 additions & 0 deletions Lib/multiprocessing/connection.py
Original file line number Diff line number Diff line change
Expand Up @@ -728,6 +728,74 @@ def PipeClient(address):
WELCOME = b'#WELCOME#'
FAILURE = b'#FAILURE#'

# multiprocessing.connection Authentication Handshake Protocol Description
# (as documented for reference after reading the existing code)
# =============================================================================
#
# On Windows: native pipes with "overlapped IO" are used to send the bytes,
# instead of the length prefix SIZE scheme described below. (ie: the OS deals
# with message sizes for us)
#
# Protocol error behaviors:
#
# On POSIX, any failure to receive the length prefix into SIZE, for SIZE greater
# than the requested maxsize to receive, or receiving fewer than SIZE bytes
# results in the connection being closed and auth to fail.
#
# On Windows, receiving too few bytes is never a low level _recv_bytes read
# error, receiving too many will trigger an error only if receive maxsize
# value was larger than 128 OR the if the data arrived in smaller pieces.
#
# Serving side Client side
# ------------------------------ ---------------------------------------
# 0. Open a connection on the pipe.
# 1. Accept connection.
# 2. New random 20 bytes -> MESSAGE
# 3. send 4 byte length (net order)
# prefix followed by:
# b'#CHALLENGE#' + MESSAGE
# 4. Receive 4 bytes, parse as network byte
# order integer. If it is -1, receive an
# additional 8 bytes, parse that as network
# byte order. The result is the length of
# the data that follows -> SIZE.
# 5. Receive min(SIZE, 256) bytes -> M1
# 6. Assert that M1 starts with:
# b'#CHALLENGE#'
# 7. Strip that prefix from M1 into -> M2
# 8. Compute HMAC-MD5 of AUTHKEY, M2 -> C_DIGEST
# 9. Send 4 byte length prefix (net order)
# followed by C_DIGEST bytes.
# 10. Compute HMAC-MD5 of AUTHKEY,
# MESSAGE into -> M_DIGEST.
# 11. Receive 4 or 4+8 byte length
# prefix (#4 dance) -> SIZE.
# 12. Receive min(SIZE, 256) -> C_D.
# 13. Compare M_DIGEST == C_D:
# 14a: Match? Send length prefix &
# b'#WELCOME#'
# <- RETURN
# 14b: Mismatch? Send len prefix &
# b'#FAILURE#'
# <- CLOSE & AuthenticationError
# 15. Receive 4 or 4+8 byte length prefix (net
# order) again as in #4 into -> SIZE.
# 16. Receive min(SIZE, 256) bytes -> M3.
# 17. Compare M3 == b'#WELCOME#':
# 17a. Match? <- RETURN
# 17b. Mismatch? <- CLOSE & AuthenticationError
#
# If this RETURNed, the connection remains open: it has been authenticated.
#
# Length prefixes are used consistently even though every step so far has
# always been a singular specific fixed length. This may help us evolve
# the protocol in the future without breaking backwards compatibility.
#
# Similarly the initial challenge message from the serving side has always
# been 20 bytes, but clients can accept a 100+ so using the length of the
# opening challenge message as an indicator of protocol version may work.


def deliver_challenge(connection, authkey):
import hmac
if not isinstance(authkey, bytes):
Expand Down