5/30/24, 8:07 PM web3.

py Patterns: Intro to Async

Web3.py

web3.py Patterns: Intro to

Async
Marc Garreau
Mar 14, 2023 • 3 min read

Asynchronous support has long been one of the most requested features in
web3.py. The AsyncHTTPProvider was first introduced in v5.20.0, but has
received polish in most releases since. There are a couple edge cases to go for full
support, but the freshly released v6 is as good a time as any to give the async
provider a try!

Why async?
If you're satisfied with the performance of your web3.py dapp or script today,
you're under no obligation to update it. However, dapps very commonly spend
much of their runtime waiting on requests to make the round trip to and from a
remote node to query block data. Asynchronous programming patterns can make
much more efficient use of your computing resources, dispatching additional
requests while still waiting for earlier ones to return, for example. If your
application performs a lot of data queries, you may have a good candidate for a
provider refactor.

A code sample
Let's skip straight to the good stuff. Here's a simple, contrived example that
makes 50 requests – one for each of the first 50 blocks of the Ethereum
blockchain – and logs out the block number as each request returns.

import asyncio
from web3 import AsyncWeb3, AsyncHTTPProvider

w3 = AsyncWeb3(AsyncHTTPProvider('https://...'))

async def fetch_blocks(n):

for result in asyncio.as_completed(
[w3.eth.get_block(num) for num in range(n)]
):
b = await result
print(b.number)

asyncio.run(fetch_blocks(50))
# 30
# 8
# 24
# 13
# ...

Here we define an asynchronous function and make use of asyncio to execute it

via the asyncio.run method. Note that web3.py utilizes aiohttp under the
hood; asyncio is the only supported async library for the moment.

asyncio has a several usage patterns, but we leverage the as_completed

method to handle each request as they return. For each coroutine (request for a
block), we await the result, then print out that block number. As a result, and
subject to whatever conditions exist in the network, block numbers are logged out
in a fairly random order.

If you need a random number generator, there are certainly more efficient ways
to go about that. Hopefully though, this is a helpful illustration of asynchronous
behavior. Depending on your use case, you may need to reorder the data before
you make use of it. The gather , wait , or callback methods may help with that;
a deep rabbit hole of learning awaits those who need more sophisticated patterns.

AsyncENS
For the uninitiated, an ENS module exists within web3.py. It too has been given
asynchronous superpowers via the AsyncENS class:

from web3 import AsyncWeb3, AsyncHTTPProvider

from ens import AsyncENS

w3 = AsyncWeb3(AsyncHTTPProvider('https://...'))

https://snakecharmers.ethereum.org/web3-py-patterns-intro-async/ 1/3
5/30/24, 8:07 PM web3.py Patterns: Intro to Async
ns = AsyncENS.from_web3(w3)

names = ['shaq.eth', 'vitalik.eth', 'parishilton.eth'] * 10

async def fetch_addresses():

for result in asyncio.as_completed(
[ns.address(name) for name in names]
):
print(await result)

asyncio.run(fetch_addresses())

The asyncio.as_completed pattern should look familiar; it's utilized here to

print out the resolved address as soon as the response is received from the
remote node.

Performance
Your optimization gains will vary greatly depending on your use case. Again, you
will benefit most in cases where you spend much of your runtime waiting on
responses from remote nodes. To give you a sense of the scale, here are some
benchmarks while using a free remote node service.

Method (50 calls) | HTTPProvider | AsyncHTTProvider

----------------------------------------------------------
eth_gasPrice | 5.95 secs | 0.52 secs
eth_blockNumber | 6.93 secs | 0.44 secs
eth_getBlock | 7.75 secs | 0.73 secs

very approximate numbers; beating up on a free remote node service

Local benchmarks for AsyncENS produced even greater gains:

Method (30 calls) | ENS | AsyncENS

-------------------------------------------------------
ns.address | 40.13 secs | 2.78 secs
ns.name | 57.13 secs | 2.96 secs

very approximate numbers; beating up on a free remote node service

Can't use the AsyncHTTPProvider ?

If your use case can benefit from a little concurrency, but for whatever reason
asyncio is not an option for you, you may need to reach for threads. An
introduction to multithreaded usage of web3.py can be found here.

If your preference or requirement is to use the Websockets or IPC providers,

async support for those providers is on the roadmap, but without a firm timeline.

Wrapping up
While continuing to receive polish, the AsyncHTTPProvider is ready for use
today. You can expect your query-intensive applications to gain some serious
performance improvements by adopting these asynchronous programming
patterns. Looking for support or to share what you're building? Join the
Ethereum Python Community Discord.

Want new posts in your inbox?

Enter your email Subscribe

web3.py Patterns: Bloom Filters

Have you ever queried an Ethereum block and wondered what that "logsBloom" was? Have you gone
looking for the most efficient way to find an event within a block? If you answered "yes" to either of
the above, then you're in for a good

Apr 24, 2024 4 min read

https://snakecharmers.ethereum.org/web3-py-patterns-intro-async/ 2/3
5/30/24, 8:07 PM web3.py Patterns: Intro to Async

Snake Charmers © 2024

Powered by Ghost

https://snakecharmers.ethereum.org/web3-py-patterns-intro-async/ 3/3