You are running multiple instances of the Python interpreter, possibly on different ma‐

chines, and you would like to exchange data between interpreters using messages.

It is easy to communicate between interpreters if you use the multiprocessing.con

from multiprocessing.connection import Listener

import traceback

def echo_client(conn):

try:

while True:

msg = conn.recv()

conn.send(msg)

except EOFError:

print('Connection closed')

def echo_server(address, authkey):

serv = Listener(address, authkey=authkey)

while True:

client = serv.accept()

echo_client(client)

except Exception:

traceback.print_exc()

echo_server(('', 25000), authkey=b'peekaboo')

Here is a simple example of a client connecting to the server and sending various

messages:

>>> from multiprocessing.connection import Client

>>> c = Client(('localhost', 25000), authkey=b'peekaboo')

>>> c.send('hello')

>>> c.recv()

>>> c.send(42)

>>> c.recv()

>>> c.send([1, 2, 3, 4, 5])

>>> c.recv()

>>>

Unlike a low-level socket, messages are kept intact (each object sent using send() is

received in its entirety with recv()). In addition, objects are serialized using pickle.

So, any object compatible with pickle can be sent or received over the connection.

There are many packages and libraries related to implementing various forms of mes‐

sage passing, such as ZeroMQ, Celery, and so forth. As an alternative, you might also

be inclined to implement a message layer on top of low-level sockets. However, some‐

times you just want a simple solution. The multiprocessing.connection library is just

that—using a few simple primitives, you can easily connect interpreters together and

have them exchange messages.

If you know that the interpreters are going to be running on the same machine, you can

use alternative forms of networking, such as UNIX domain sockets or Windows named

pipes. To create a connection using a UNIX domain socket, simply change the address

to a filename such as this:

s = Listener('/tmp/myconn', authkey=b'peekaboo')

To create a connection using a Windows named pipe, use a filename such as this:

s = Listener(r'\\.\pipe\myconn', authkey=b'peekaboo')

As a general rule, you would not be using multiprocessing to implement public-facing

services. The authkey parameter to Client() and Listener() is there to help authen‐

ticate the end points of the connection. Connection attempts with a bad key raise an

exception. In addition, the module is probably best suited for long-running connections

(not a large number of short connections). For example, two interpreters might establish

a connection at startup and keep the connection active for the entire duration of a

problem.

Don’t use multiprocessing if you need more low-level control over aspects of the con‐

nection. For example, if you needed to support timeouts, nonblocking I/O, or anything

similar, you’re probably better off using a different library or implementing such features

on top of sockets instead.