5/30/24, 8:05 PM Web3.
py Patterns: Customization
Web3.py
Web3.py Patterns:
Customization
Marc Garreau
May 25, 2022 • 3 min read
If you're looking to make Web3.py do something outside of its native
functionality, you've got at least a few options: middleware, custom methods,
external modules, and custom providers. This post will walk through what each of
those are, when you might reach for them, and how to get started.
1. Middleware
What
Middleware allows you to add some behavior to existing methods prior to making
a request or after a result is received.
When
Reach for middleware when you want something to happen every time a certain
RPC call or set of calls are performed, e.g., logging, data visualization, data
conversion, etc.
How
A set of default middleware comes standard in Web3.py, along with more
optional middleware to pull in. However, If you need to write some custom
middleware, you have a couple of syntax options: functions or classes. The
function syntax is more typical for straightforward use cases.
def example_middleware(make_request, w3):
# do one-time setup operations here
def middleware(method, params):
# do pre-processing here
# perform the RPC request, getting the response
response = make_request(method, params)
# do post-processing here
# finally return the response
return response
return middleware
A middleware template utilizing the function syntax
Middleware is executed in a particular order, so the API allows you to add your
new middleware to the end of the list, inject , replace or remove a layer, or
clear the whole middleware stack.
2. Custom Methods
What
Arbitrary RPC methods may be added to existing modules.
When
Registering custom methods can be handy if you're working with a client with
nonstandard RPC commands or are testing some custom functionality within a
forked client.
Custom methods can also be used to overwrite existing methods, if you want to
apply your own request or result formatter.
How
The attach_methods function is available on every module and accepts a
dictionary with method names and corresponding Method :
https://snakecharmers.ethereum.org/web3-py-patterns-customizations/ 1/3
5/30/24, 8:05 PM Web3.py Patterns: Customization
from web3.method import Method
w3.eth.attach_methods({"create_access_list": Method("eth_createAccessList")})
w3.eth.create_access_list(...)
You may optionally include custom handlers for input munging, request, and
result formatters.
from web3.method import Method
w3.eth.attach_methods({
"example": Method(
"eth_example",
mungers=[...],
request_formatters=[...],
result_formatters=[...],
is_property=False,
),
})
w3.eth.example()
Adding a custom eth_example method to the Eth module
If you prefer, you can add a property instead of a method by setting
is_property to True .
3. External Modules
What
External modules offer still more flexibility by allowing you to import groups of
APIs under one banner. Think: plugins.
When
External modules may be useful for introducing an entire L2 API or several
nonstandard RPC methods supported by one client, e.g., Erigon-specific methods
like erigon_getHeaderByHash , erigon_getHeaderByNumber , and so on.
How
Modules need only be classes and can reference the parent Web3 instance.
Configure your external modules at the time of Web3 instantiation using the
external_modules keyword argument, or at any point via the attach_modules
method:
# add modules at instantiation:
w3 = Web3(
HTTPProvider(...),
external_modules={"example": ExampleModule}
)
# add modules after instantiation:
w3.attach_modules({"example": ExampleModule})
# invoking external modules:
w3.example.example_method()
More context, including a nested module example, available here.
4. Custom Providers
What
At its core, a provider defines how requests are performed.
When
Building a custom provider is only relevant for the rare occasions that you're
plugging into a custom testing framework, or something in that vein. If you're
just looking to connect to another EVM blockchain, sidechain, or rollup, typically
you can just configure one of the existing options: HTTPProvider, IPCProvider,
or WebsocketProvider.
How
Providers only require two methods, make_request and isConnected , and a
definition of middlewares .
from web3.providers.base import BaseProvider
class CustomProvider(BaseProvider):
middlewares = ()
https://snakecharmers.ethereum.org/web3-py-patterns-customizations/ 2/3
5/30/24, 8:05 PM Web3.py Patterns: Customization
def make_request(self, method, params):
return {"result": {"welp": "lol"}}
def isConnected(self):
print(True)
w3 = Web3(CustomProvider())
w3.eth.get_block("latest")
# AttributeDict({"welp": "lol"})
Wrapping up
The options above are ordered roughly from least to most flexibility they offer. In
practice, I expect middleware and external modules to get the most mileage,
particularly once trusted external modules become commonplace.
Deliberately not included in this post is monkey patching. If you've gone down
that path... is everything okay? Seriously though, open an issue if you have
another vector you'd like to customize that requires monkey patching today.
Want new posts in your inbox?
Enter your email Subscribe
web3.py Patterns: Bloom Filters
Have you ever queried an Ethereum block and wondered what that "logsBloom" was? Have you gone
looking for the most efficient way to find an event within a block? If you answered "yes" to either of
the above, then you're in for a good
Apr 24, 2024 4 min read
Snake Charmers © 2024
Powered by Ghost
https://snakecharmers.ethereum.org/web3-py-patterns-customizations/ 3/3
