RPC is easy to implement by encoding function requests, arguments, and return values

using pickle, and passing the pickled byte strings between interpreters. Here is an

example of a simple RPC handler that could be incorporated into a server:

# rpcserver.py

import pickle

class RPCHandler:

def __init__(self):

self._functions = { }

def register_function(self, func):

self._functions[func.__name__] = func

def handle_connection(self, connection):

try:

while True:

# Receive a message

func_name, args, kwargs = pickle.loads(connection.recv())

# Run the RPC and send a response

try:

r = self._functions[func_name](*args,**kwargs)

To use this handler, you need to add it into a messaging server. There are many possible

choices, but the multiprocessing library provides a simple option. Here is an example

RPC server:

from multiprocessing.connection import Listener

from threading import Thread

def rpc_server(handler, address, authkey):

sock = Listener(address, authkey=authkey)

while True:

client = sock.accept()

t = Thread(target=handler.handle_connection, args=(client,))

t.daemon = True

t.start()

# Some remote functions

def add(x, y):

return x + y

def sub(x, y):

return x - y

# Register with a handler

handler = RPCHandler()

handler.register_function(add)

handler.register_function(sub)

# Run the server

rpc_server(handler, ('localhost', 17000), authkey=b'peekaboo')

To access the server from a remote client, you need to create a corresponding RPC proxy

class that forwards requests. For example:

import pickle

class RPCProxy:

def __init__(self, connection):

self._connection = connection

def __getattr__(self, name):

def do_rpc(*args, **kwargs):

To use the proxy, you wrap it around a connection to the server. For example:

>>> from multiprocessing.connection import Client

>>> c = Client(('localhost', 17000), authkey=b'peekaboo')

>>> proxy = RPCProxy(c)

>>> proxy.add(2, 3)

5

>>> proxy.sub(2, 3)

>>> proxy.sub([1, 2], 4)

>>>

It should be noted that many messaging layers (such as multiprocessing) already se‐

rialize data using pickle. If this is the case, the pickle.dumps() and pickle.loads()

calls can be eliminated.

The general idea of the RPCHandler and RPCProxy classes is relatively simple. If a client

wants to call a remote function, such as foo(1, 2, z=3), the proxy class creates a tuple

('foo', (1, 2), {'z': 3}) that contains the function name and arguments. This

tuple is pickled and sent over the connection. This is performed in the do_rpc() closure

that’s returned by the __getattr__() method of RPCProxy. The server receives and

unpickles the message, looks up the function name to see if it’s registered, and executes

it with the given arguments. The result (or exception) is then pickled and sent back.

As shown, the example relies on multiprocessing for communication. However, this

approach could be made to work with just about any other messaging system. For ex‐

ample, if you want to implement RPC over ZeroMQ, just replace the connection objects

with an appropriate ZeroMQ socket object.

Given the reliance on pickle, security is a major concern (because a clever hacker can

create messages that make arbitrary functions execute during unpickling). In particular,

you should never allow RPC from untrusted or unauthenticated clients. In particular,

you definitely don’t want to allow access from just any machine on the Internet—this

should really only be used internally, behind a firewall, and not exposed to the rest of

the world.

As an alternative to pickle, you might consider the use of JSON, XML, or some other

data encoding for serialization. For example, this recipe is fairly easy to adapt to JSON

encoding

if you simply replace pickle.loads() and pickle.dumps() with

json.loads() and json.dumps(). For example:

# jsonrpcserver.py

import json

class RPCHandler:

def __init__(self):

self._functions = { }

def register_function(self, func):

self._functions[func.__name__] = func

def handle_connection(self, connection):

try:

while True:

# Receive a message

func_name, args, kwargs = json.loads(connection.recv())

# Run the RPC and send a response

try:

r = self._functions[func_name](*args,**kwargs)

# jsonrpcclient.py

import json

class RPCProxy:

def __init__(self, connection):

self._connection = connection

def __getattr__(self, name):

def do_rpc(*args, **kwargs):

One complicated factor in implementing RPC is how to handle exceptions. At the very

least, the server shouldn’t crash if an exception is raised by a method. However, the

means by which the exception gets reported back to the client requires some study. If

you’re using pickle, exception instances can often be serialized and reraised in the

client. If you’re using some other protocol, you might have to think of an alternative

approach. At the very least, you would probably want to return the exception string in

the response. This is the approach taken in the JSON example.

For another example of an RPC implementation, it can be useful to look at the imple‐

mentation of the SimpleXMLRPCServer and ServerProxy classes used in XML-RPC, as

described in Recipe 11.6.