trio-asyncio: A re-implementation of the asyncio mainloop on top of Trio

trio-asyncio is the library of choice for a Python program that contains both Trio and asyncio code.

Trio has native concepts of tasks and task cancellation. Asyncio is based on callbacks and chaining Futures, albeit with nicer syntax, which make handling of failures and timeouts fundamentally less reliable, especially in larger programs. Thus, you really want to use Trio in your project.

On the other hand, there are quite a few robust libraries that have been implemented using asyncio, while Trio’s ecosystem is relatively younger. You really don’t want to re-invent any wheels in your project.

Thus, being able to use asyncio libraries from Trio is useful. trio-asyncio enables you to do that, and more.

With trio-asyncio, you can:

  • Incrementally convert an asyncio application to Trio. Start with a Trio mainloop, call your existing asyncio code, then successively convert procedures to Trio calling conventions.

  • Use any asyncio-capable library in a Trio application.

  • Use trio-asyncio as a building block for convincing other async-ish libraries (Twisted, Promise, …) to be compatible with Trio.

trio-asyncio is tested against the complete asyncio test suite as shipped with each supported version of Python, although a small number of tests fail due to our limited support for starting and stopping the same event loop multiple times. It has also passed the test suite of some complex asyncio libraries such as home-assistant.

Note

trio-asyncio is most useful for applications: it works best when you control the code that starts the event loop (such as the call to asyncio.run()). If you’re writing a library and want to adopt a Trio-ish worldview without sacrificing asyncio compatibility, you might find anyio helpful.

Helpful facts:

  • Supported environments: Linux, MacOS, or Windows running some kind of Python 3.7-or-better (either CPython or PyPy3 is fine). *BSD and illumOS likely work too, but are untested.

  • Install: python3 -m pip install -U trio-asyncio (or on Windows, maybe py -3 -m pip install -U trio-asyncio). No compiler needed.

  • Tutorial and reference manual: https://trio-asyncio.readthedocs.io

  • Bug tracker and source code: https://github.com/python-trio/trio-asyncio

Inherited from Trio:

trio-asyncio manual

Principles

Async function “flavors”

As you might recall from the discussion of async “sandwiches” in the Trio tutorial, every async function ultimately must do its useful work by directly or indirectly calling back into the same async library (such as asyncio or Trio) that’s managing the currently running event loop. If a function invoked within a trio.run() calls asyncio.sleep(), or a function invoked within an asyncio.run() calls trio.sleep(), the sleep function will send a message to the event loop that the event loop doesn’t know how to handle, and some sort of error will result.

In a program that uses trio-asyncio, you probably have some async functions implemented in terms of Trio calls and some implemented in terms of asyncio calls. In order to keep track of which is which, we’ll call these “Trio-flavored” and “asyncio-flavored” functions, respectively. It is critical that you understand which functions in your program are Trio-flavored and which are asyncio-flavored, just like it’s critical that you understand which functions are synchronous and which ones are async. Unfortunately, there’s no syntactic marker for flavor: both Trio-flavored and asyncio-flavored functions are defined with async def fn() and call other async functions with await other_fn(). You’ll have to keep track of it some other way. To help you out, every function in trio-asyncio documents its flavor, and we recommend that you follow this convention in your own programs too.

The general rules that determine flavor are as follows:

  • Every async function in the trio module is Trio-flavored. Every async function in the asyncio module is asyncio-flavored.

  • Flavor is transitive: if async function foo() calls await bar(), then foo() has bar()’s flavor. (If foo() calls await baz() too, then bar() and baz() had better have the same flavor.)

  • trio-asyncio gives you the ability to call functions whose flavor is different than your own, but you must be explicit about it. trio_asyncio.aio_as_trio() takes an asyncio-flavored function and returns a Trio-flavored wrapper for it; trio_as_aio() takes a Trio-flavored function and returns an asyncio-flavored wrapper for it.

If you don’t keep track of your function flavors correctly, you might get exceptions like the following:

  • If you call a Trio function where an asyncio function is expected: RuntimeError: Task got bad yield: followed by either WaitTaskRescheduled(abort_func=...) or <class 'trio._core._traps.CancelShieldedCheckpoint'>

  • If you call an asyncio function where a Trio function is expected: TypeError: trio.run received unrecognized yield message <Future ...>.

Other errors are possible too.

Flavor versus context

The concept of function flavor is distinct from the concept of “asyncio context” or “Trio context”. You’re in Trio context if you’re (indirectly) inside a call to trio.run(). You’re in asyncio context if asyncio.get_running_loop() returns a valid event loop. In a trio-asyncio program, you will frequently be in both Trio context and asyncio context at the same time, but each async function is either Trio-flavored or asyncio-flavored (not both).

Most synchronous asyncio or Trio functions (trio.Event.set(), asyncio.StreamWriter.close(), etc) only require you to be in asyncio or Trio context, and work equally well regardless of the flavor of function calling them. The exceptions are functions that access the current task (asyncio.current_task(), trio.lowlevel.current_task(), and anything that calls them), because there’s only a meaningful concept of the current foo task when a foo-flavored function is executing. For example, this means context managers that set a timeout on their body (with async_timeout.timeout(N):, with trio.move_on_after(N):) must be run from within the correct flavor of function.

Flavor transitions are explicit

As mentioned above, trio-asyncio does not generally allow you to transparently call await trio.something() from asyncio code, nor vice versa; you need to use aio_as_trio() or trio_as_aio() when calling a function whose flavor is different than yours. This is certainly more frustrating than having it “just work”. Unfortunately, semantic differences between Trio and asyncio (such as how to signal cancellation) need to be resolved at each boundary between asyncio and Trio, and we haven’t found a way to do this with acceptable performance and robustness unless those boundaries are marked.

If you insist on living on the wild side, trio-asyncio does provide allow_asyncio() which allows limited, experimental, and slow mixing of Trio-flavored and asyncio-flavored calls in the same Trio-flavored function.

trio-asyncio’s place in the asyncio stack

At its base, asyncio doesn’t know anything about futures or coroutines, nor does it have any concept of a task. All of these features are built on top of the simpler interfaces provided by the event loop. The event loop itself has little functionality beyond executing synchronous functions submitted with call_soon() and call_later() and invoking I/O availability callbacks registered using add_reader() and add_writer() at the appropriate times.

trio-asyncio provides an asyncio event loop implementation which performs these basic operations using Trio APIs. Everything else in asyncio (futures, tasks, cancellation, and so on) is ultimately implemented in terms of calls to event loop methods, and thus works “magically” with Trio once the trio-asyncio event loop is installed. This strategy provides a high level of compatibility with asyncio libraries, but it also means that asyncio-flavored code running under trio-asyncio doesn’t benefit much from Trio’s more structured approach to concurrent programming: cancellation, causality, and exception propagation in asyncio-flavored code are just as error-prone under trio-asyncio as they are under the default asyncio event loop. (Of course, your Trio-flavored code will still benefit from all the usual Trio guarantees.)

If you look at a Trio task tree, you’ll see only one Trio task for the entire asyncio event loop. The distinctions between different asyncio tasks are erased, because they’ve all been merged into a single pot of callback soup by the time they get to trio-asyncio. Similarly, context variables will only work properly in asyncio-flavored code when running Python 3.7 or later (where they’re supported natively), even though Trio supports them on earlier Pythons using a backport package.

Event loop implementations

A stock asyncio event loop may be interrupted and restarted at any time, simply by making repeated calls to run_until_complete(). Trio, however, requires one long-running main loop. trio-asyncio bridges this gap by providing two event loop implementations.

  • The preferred option is to use an “async loop”: inside a Trio-flavored async function, write async with trio_asyncio.open_loop() as loop:. Within the async with block (and anything it calls, and any tasks it starts, and so on), asyncio.get_event_loop() and asyncio.get_running_loop() will return loop. You can’t manually start and stop an async loop. Instead, it starts when you enter the async with block and stops when you exit the block.

  • The other option is a “sync loop”. If you’ve imported trio-asyncio but aren’t in Trio context, and you haven’t installed a custom event loop policy, calling asyncio.new_event_loop() (including the implicit call made by the first asyncio.get_event_loop() in the main thread) will give you an event loop that transparently runs in a separate greenlet in order to support multiple calls to run_until_complete(), run_forever(), and stop(). Sync loops are intended to allow trio-asyncio to run the existing test suites of large asyncio libraries, which often call run_until_complete() on the same loop multiple times. Using them for other purposes is not recommended (it is better to refactor so you can use an async loop) but will probably work.

Usage

Using trio from asyncio, or vice versa, requires two steps:

Because Trio and asyncio differ in some key semantics, most notably how they handle tasks and cancellation, usually their domains are strictly separated – i.e., you need to call a wrapper that translates from one to the other. While trio-asyncio includes a wrapper that allows you ignore that separation in some limited cases, you probably should not use it in non-trivial programs.

Startup and shutdown

Trio main loop

Typically, you start with a Trio program which you need to extend with asyncio code.

Before:

import trio

trio.run(async_main, *args)

After:

import trio
import trio_asyncio

trio_asyncio.run(async_main, *args)

Note that async_main here still must be a Trio-flavored async function! trio_asyncio.run() is trio.run() plus an additional asyncio context (which you can take advantage of using aio_as_trio()).

Equivalently, wrap your main function (or any other code that needs to talk to asyncio) in a trio_asyncio.open_loop() block:

import trio
import trio_asyncio

async def async_main_wrapper(*args):
    async with trio_asyncio.open_loop() as loop:
        assert loop == asyncio.get_event_loop()
        await async_main(*args)

trio.run(async_main_wrapper, *args)

In either case, within async_main, calls to asyncio.get_event_loop() will return the currently-running TrioEventLoop instance. (Since asyncio code uses the result of get_event_loop() as the default event loop in effectively all cases, this means you don’t need to pass loop= arguments around explicitly.)

TrioEventLoop has a few trio-asyncio specific methods in addition to the usual asyncio.AbstractEventLoop interface; these are documented in the appropriate sections below. In general you don’t need to care about any of them, though, as you can just use aio_as_trio() to run asyncio code. (See below for more details.)

async with trio_asyncio.open_loop(queue_len=None)

Returns a Trio-flavored async context manager which provides an asyncio event loop running on top of Trio.

The context manager evaluates to a new TrioEventLoop object.

Entering the context manager is not enough on its own to immediately run asyncio code; it just provides the context that makes running that code possible. You additionally need to wrap any asyncio functions that you want to run in aio_as_trio().

If you provide a queue_len, then any attempt to enqueue more than that many asyncio callbacks near-simultaneously (including, for example, new task creations) will fail. There is no way to backpressure asyncio callback registration, so the best we can do if the queue length is exceeded is raise an exception (trio.WouldBlock), which is likely to crash your whole program. It is suggested to leave the queue_len at its default of None (unlimited) unless you need to enforce hard constraints on memory use.

Exiting the context manager will attempt to do an orderly shutdown of the tasks it contains, analogously to asyncio.run(). Both asyncio-flavored tasks and Trio-flavored tasks (the latter started using trio_as_future(), run_trio_task(), trio_as_aio(), etc) are cancelled simultaneously, and the loop waits for them to exit in response to this cancellation before proceeding. All call_soon() callbacks that are submitted before exiting the context manager will run before starting this shutdown sequence, and all callbacks that are submitted before the last task exits will run before the loop closes. The exact point at which the loop stops running callbacks is not specified.

Warning

As with asyncio.run(), asyncio-flavored tasks that are started after exiting the context manager (such as by another task as it unwinds) may or may not be cancelled, and will be abandoned if they survive the shutdown sequence. This may lead to unclosed resources, stderr spew about “coroutine ignored GeneratorExit”, etc. Trio-flavored tasks do not have this hazard.

Example usage:

async def async_main(*args):
    async with trio_asyncio.open_loop() as loop:
        # async part of your main program here
        await trio.sleep(1)
        await trio_asyncio.aio_as_trio(asyncio.sleep)(2)
trio_asyncio.run(proc, *args, queue_len=None, **trio_run_options)

Run a Trio-flavored async function in a context that has an asyncio event loop also available.

This is exactly equivalent to using trio.run() plus wrapping the body of proc in async with trio_asyncio.open_loop():. See the documentation of open_loop() for more on the queue_len argument, which should usually be left at its default of None.

Stopping

The asyncio mainloop will be stopped automatically when the code within async with open_loop() / trio_asyncion.run() exits. trio-asyncio will process all outstanding callbacks and terminate. As in asyncio, callbacks which are added during this step will be ignored.

You cannot restart the loop, nor would you want to. You can always make another loop if you need one.

Asyncio main loop

Sometimes you instead start with asyncio code which you wish to extend with some Trio portions. The best-supported approach here is to wrap your entire asyncio program in a Trio event loop. In other words, you should transform this code:

def main():
    loop = asyncio.get_event_loop()
    loop.run_until_complete(async_main())

or (Python 3.7 and later):

def main():
    asyncio.run(async_main())

to this:

def main():
    trio_asyncio.run(trio_asyncio.aio_as_trio(async_main))

If your program makes multiple calls to run_until_complete() and/or run_forever(), or if the call to asyncio.run() is hidden inside a library you’re using, then this may be a somewhat challenging transformation. In such cases, you can instead keep the old approach (get_event_loop() + run_until_complete()) unchanged, and if you’ve imported trio_asyncio (and not changed the asyncio event loop policy) you’ll still be able to use trio_as_aio() to run Trio code from within your asyncio-flavored functions. This is referred to internally as a “sync loop” (SyncTrioEventLoop), as contrasted with the “async loop” that you use when you start from an existing Trio run. The sync loop is implemented using the greenlet library to switch out of a Trio run that has not yet completed, so it is less well-supported than the approach where you start in Trio. But as of trio-asyncio 0.14.0, we do think it should generally work.

Compatibility issues

Loop implementations

There are replacement event loops for asyncio, such as uvloop. trio-asyncio is not compatible with them.

Multithreading

trio-asyncio monkey-patches asyncio’s loop policy to be thread-local. This lets you use uvloop in one thread while running trio_asyncio in another.

Interrupting the asyncio loop

A trio-asyncio event loop created with open_loop() does not support run_until_complete or run_forever. If you need these features, you might be able to get away with using a “sync loop” as explained above, but it’s better to refactor your program so all of its async code runs within a single event loop invocation. For example, you might replace:

async def setup():
    pass # … start your services
async def shutdown():
    pass # … terminate services and clean up
    loop.stop()

loop = asyncio.get_event_loop()
loop.run_until_complete(setup)
loop.run_forever()

with:

stopped_event = trio.Event()
async def setup():
    pass # … start your services
async def cleanup():
    pass # … terminate services and clean up
async def shutdown():
    stopped_event.set()

async def async_main():
    await aio_as_trio(setup)()
    await stopped_event.wait()
    await aio_as_trio(cleanup)()
trio_asyncio.run(async_main)
Detecting the current function’s flavor

sniffio.current_async_library() correctly reports “asyncio” or “trio” when called from a trio-asyncio program, based on the flavor of function that’s calling it.

However, this feature should generally not be necessary, because you should know whether each function in your program is asyncio-flavored or Trio-flavored. (The two have different semantics, especially surrounding cancellation.) It’s provided mainly so that your trio-asyncio program can safely depend on libraries that use sniffio to support both flavors. It can also be helpful if you want to assert that you’re in the mode you think you’re in, using

assert sniffio.current_async_library() == "trio"

(or "asyncio") to detect mismatched flavors while porting code from asyncio to Trio.

Cross-calling

First, a bit of background.

For historical reasons, calling an async function (of any flavor) is a two-step process – that is, given

async def proc():
    pass

a call to await proc() does two things:

  • proc() returns an awaitable, i.e. something that has an __await__ method.

  • await proc() then hooks this awaitable up to your event loop, so that it can do whatever combination of execution and cooperative blocking it desires. (Technically, __await__() returns an iterable, which is iterated until it has been exhausted, and each yielded object is sent through to the event loop.)

asyncio traditionally uses awaitables for indirect procedure calls, so you often see the pattern:

async def some_code():
    pass
async def run(proc):
    await proc
await run(some_code())

This method has a problem: it decouples creating the awailable from running it. If you decide to add code to run that retries running proc when it encounters a specific error, you’re out of luck.

Trio, in contrast, uses (async) callables:

async def some_code():
    pass
async def run(proc):
    await proc()
await run(some_code)

Here, calling proc multiple times from within run is not a problem.

trio-asyncio adheres to Trio conventions, but the asyncio way is also supported when possible.

Calling asyncio from Trio

Wrap the callable, awaitable, generator, or iterator in trio_asyncio.aio_as_trio().

Thus, you can call an asyncio function from Trio as follows:

async def aio_sleep(sec=1):
    await asyncio.sleep(sec)
async def trio_sleep(sec=2):
    await aio_as_trio(aio_sleep)(sec)
trio_asyncio.run(trio_sleep, 3)

or use as a decorator to pre-wrap:

@aio_as_trio
async def trio_sleep(sec=1):
    await asyncio.sleep(sec)
trio_asyncio.run(trio_sleep, 3)

or pass an awaitable:

async def aio_sleep(sec=1):
    await asyncio.sleep(sec)
async def trio_sleep(sec=2):
    await aio_as_trio(aio_sleep(sec))
trio_asyncio.run(trio_sleep, 3)

If you have a choice between aio_as_trio(foo)(bar) and aio_as_trio(foo(bar)), choose the former. If foo() is an async function defined with async def, it doesn’t matter; they behave equivalently. But if foo() is a synchronous wrapper that does anything before delegating to an async function, the first approach will let the synchronous part of foo() determine the current asyncio task, and the second will not. The difference is relevant in practice for popular libraries such as aiohttp.

aio_as_trio() also accepts asyncio.Futures:

async def aio_sleep(sec=1):
    await asyncio.sleep(sec)
    return 42
async def trio_sleep(sec=2):
    f = aio_sleep(1)
    f = asyncio.ensure_future(f)
    r = await aio_as_trio(f)
    assert r == 42
trio_asyncio.run(trio_sleep, 3)

as well as async iterators (such as async generators):

async def aio_slow():
    n = 0
    while True:
        await asyncio.sleep(n)
        yield n
        n += 1
async def printer():
    async for n in aio_as_trio(aio_slow()):
        print(n)
trio_asyncio.run(printer)

and async context managers:

class AsyncCtx:
    async def __aenter__(self):
        await asyncio.sleep(1)
        return self
    async def __aexit__(self, *tb):
        await asyncio.sleep(1)
    async def delay(self, sec=1):
        await asyncio.sleep(sec)
async def trio_ctx():
    async with aio_as_trio(AsyncCtx()) as ctx:
        print("within")
        await aio_as_trio(ctx.delay)(2)
trio_asyncio.run(trio_ctx)

As you can see from the above example, aio_as_trio() handles wrapping the context entry and exit, but it doesn’t know anything about async methods that may exist on the object to which the context evaluates. You still need to treat them as asyncio methods and wrap them appropriately when you call them.

Note that creating the async context manager or async iterator is not itself an asynchronous process; i.e., AsyncCtx.__init__ or __aiter__ is a normal synchronous procedure. Only the __aenter__ and __aexit__ methods of an async context manager, or the __anext__ method of an async iterator, are asynchronous. This is why you need to wrap the context manager or iterator itself – unlike with a simple procedure call, you cannot wrap the call for generating the context handler.

Thus, the following code will not work:

async def trio_ctx():
    async with aio_as_trio(AsyncCtx)() as ctx:  # no!
        print("within")
    async for n in aio_as_trio(aio_slow)():  # also no!
        print(n)
trio_asyncio.aio_as_trio(proc, *, loop=None)

Return a Trio-flavored wrapper for an asyncio-flavored awaitable, async function, async context manager, or async iterator.

Alias: asyncio_as_trio()

This is the primary interface for calling asyncio code from Trio code. You can also use it as a decorator on an asyncio-flavored async function; the decorated function will be callable from Trio-flavored code without additional boilerplate.

Note that while adapting coroutines, i.e.:

await aio_as_trio(proc(*args))

is supported (because asyncio uses them a lot) they’re not a good idea because setting up the coroutine won’t run within an asyncio context. If possible, use:

await aio_as_trio(proc)(*args)

instead.

Too complicated?

There’s also a somewhat-magic wrapper (trio_asyncio.allow_asyncio()) which, as the name implies, allows you to directly call asyncio-flavored functions from a function that is otherwise Trio-flavored.

async def hybrid():
    await trio.sleep(1)
    await asyncio.sleep(1)
    print("Well, that worked")
trio_asyncio.run(trio_asyncio.allow_asyncio, hybrid)

This method works for one-off code. However, there are a couple of semantic differences between asyncio and Trio which trio_asyncio.allow_asyncio() is unable to account for. Additionally, the transparency support is only one-way; you can’t transparently call Trio from a function that’s used by asyncio callers. Thus, you really should not use it for “real” programs or libraries.

await trio_asyncio.allow_asyncio(fn, *args)

Execute await fn(*args) in a context that allows fn to call both Trio-flavored and asyncio-flavored functions without marking which ones are which.

This is a Trio-flavored async function. There is no asyncio-flavored equivalent.

This wrapper allows you to indiscrimnately mix trio and asyncio functions, generators, or iterators:

import trio
import asyncio
import trio_asyncio

async def hello(loop):
    await asyncio.sleep(1)
    print("Hello")
    await trio.sleep(1)
    print("World")

async def main():
    with trio_asyncio.open_loop() as loop:
        await trio_asyncio.allow_asyncio(hello, loop)
trio.run(main)

Unfortunately, there are issues with cancellation (specifically, asyncio function will see trio.Cancelled instead of concurrent.futures.CancelledError). Thus, this mode is not the default.

Calling Trio from asyncio

Wrap the callable, generator, or iterator in trio_asyncio.trio_as_aio().

Thus, you can call a Trio function from asyncio as follows:

async def trio_sleep(sec=1):
    await trio.sleep(sec)
async def aio_sleep(sec=2):
    await trio_as_aio(trio_sleep)(sec)
trio_asyncio.run(aio_as_trio, aio_sleep, 3)

or use a decorator to pre-wrap:

@trio_as_aio
async def aio_sleep(sec=2):
    await trio.sleep(sec)
trio_asyncio.run(aio_as_trio, aio_sleep, 3)

In contrast to aio_as_trio(), using an awaitable is not supported because that’s not an idiom Trio uses.

Calling a function wrapped with trio_as_aio() returns a regular asyncio.Future. Thus, you can call it from a synchronous context (e.g. a callback hook). Of course, you’re responsible for catching any errors – either arrange to await the future, or use add_done_callback():

async def trio_sleep(sec=1):
    await trio.sleep(sec)
    return 42
def cb(f):
    assert f.result == 42
async def aio_sleep(sec=2):
    f = trio_as_aio(trio_sleep)(1)
    f.add_done_callback(cb)
    r = await f
    assert r == 42
trio_asyncio.run(aio_as_trio, aio_sleep, 3)

You can wrap async context managers and async iterables just like with aio_as_trio().

trio_asyncio.trio_as_aio(proc, *, loop=None)

Return an asyncio-flavored wrapper for a Trio-flavored async function, async context manager, or async iterator.

Alias: trio_as_asyncio()

This is the primary interface for calling Trio code from asyncio code. You can also use it as a decorator on a Trio-flavored async function; the decorated function will be callable from asyncio-flavored code without additional boilerplate.

Note that adapting coroutines, i.e.:

await trio_as_aio(proc(*args))

is not supported, because Trio does not expose the existence of coroutine objects in its API. Instead, use:

await trio_as_aio(proc)(*args)

Or if you already have proc(*args) as a single object coro for some reason:

await trio_as_aio(lambda: coro)()

Warning

Be careful when using this to wrap an async context manager. There is currently no mechanism for running the entry and exit in the same Trio task, so if the async context manager wraps a nursery, havoc is likely to result. That is, instead of:

async def some_aio_func():
    async with trio_asyncio.trio_as_aio(trio.open_nursery()) as nursery:
        # code that uses nursery -- this will blow up

do something like:

async def some_aio_func():
    @trio_asyncio.aio_as_trio
    async def aio_body(nursery):
        # code that uses nursery -- this will work

    @trio_asyncio.trio_as_aio
    async def trio_body():
        async with trio.open_nursery() as nursery:
            await aio_body(nursery)

    await trio_body()
Trio background tasks

If you want to start a Trio task that should be monitored by trio_asyncio (i.e. an uncaught error will propagate to, and terminate, the asyncio event loop) instead of having its result wrapped in a asyncio.Future, use run_trio_task().

Multiple asyncio loops

trio-asyncio supports running multiple concurrent asyncio loops in different Trio tasks in the same thread. You may even nest them.

This means that you can write a trio-ish wrapper around an asyncio-using library without regard to whether the main loop or another library also use trio-asyncio.

You can use the event loop’s autoclose() method to tell trio-asyncio to auto-close a file descriptor when the loop terminates. This setting only applies to file descriptors that have been submitted to a loop’s add_reader() or add_writer() methods. As such, this method is mainly useful for servers and should be used to supplement, rather than replace, a finally: handler or a with closing(...): block.

Errors and cancellations

Errors and cancellations are propagated almost-transparently.

For errors, this is straightforward: if a cross-called function terminates with an exception, it continues to propagate out of the cross-call.

Cancellations are also propagated whenever possible. This means

  • the task started with run_trio() is cancelled when you cancel the future which run_trio() returns

  • if the task started with run_trio() is cancelled, the future gets cancelled

  • the future passed to run_aio_future() is cancelled when the Trio code calling it is cancelled

  • However, when the future passed to run_aio_future() is cancelled (i.e., when the task associated with it raises asyncio.CancelledError), that exception is passed along unchanged.

    This asymmetry is intentional since the code that waits for the future often is not within the cancellation context of the part that created it. Cancelling the future would thus impact the wrong (sub)task.

asyncio feature support notes

Deferred calls

call_soon() and friends work as usual.

Worker threads

run_in_executor() works as usual.

There is one caveat: the executor must be either None or an instance of trio_asyncio.TrioExecutor.

class trio_asyncio.TrioExecutor(limiter=None, thread_name_prefix=None, max_workers=None)

An executor that runs its job in a Trio worker thread.

Bases: concurrent.futures.ThreadPoolExecutor

Parameters:

  • limiter (trio.CapacityLimiter or None) – If specified, use this capacity limiter to control the number of threads in which this exeuctor can be running jobs.

  • thread_name_prefix – unused

  • max_workers (int or None) – If specified and limiter is not specified, create a new trio.CapacityLimiter with this value as its limit, and use that as the limiter.

File descriptors

add_reader() and add_writer() work as usual, if you really need them. Behind the scenes, these calls create a Trio task which waits for readability/writability and then runs the callback.

You might consider converting code using these calls to native Trio tasks.

Signals

add_signal_handler() works as usual.

Subprocesses

create_subprocess_exec() and create_subprocess_shell() work as usual.

You might want to convert these calls to use native Trio subprocesses.

Custom child watchers are not supported.

Low-level API reference

class trio_asyncio.BaseTrioEventLoop(queue_len=None)

An asyncio event loop that runs on top of Trio.

Bases: asyncio.SelectorEventLoop

All event loops created by trio-asyncio are of a type derived from BaseTrioEventLoop.

Parameters:

queue_len – The maximum length of the internal event queue. The default of None means unlimited. A limit should be specified only if you would rather crash your program than use too much memory, because it’s not feasible to enforce graceful backpressure here.

staticmethod run_aio_future(fut)

Alias for trio_asyncio.run_aio_future().

This is a Trio-flavored async function.

await run_aio_coroutine(coro)

Schedule an asyncio-flavored coroutine for execution on this loop by wrapping it in an asyncio.Task. Wait for it to complete, then return or raise its result.

Cancelling the current Trio scope will cancel the coroutine, which will throw a single asyncio.CancelledError into the coroutine (just like the usual asyncio behavior). If the coroutine then exits with a CancelledError exception, the call to run_aio_coroutine() will raise trio.Cancelled. But if it exits with CancelledError when the current Trio scope was not cancelled, the CancelledError will be passed along unchanged.

This is a Trio-flavored async function.

trio_as_future(proc, *args)

Start a new Trio task to run await proc(*args) asynchronously. Return an asyncio.Future that will resolve to the value or exception produced by that call.

Errors raised by the Trio call will only be used to resolve the returned Future; they won’t be propagated in any other way. Thus, if you want to notice exceptions, you had better not lose track of the returned Future. The easiest way to do this is to immediately await it in an asyncio-flavored function: await loop.trio_as_future(trio_func, *args).

Note that it’s the awaiting of the returned future, not the call to trio_as_future() itself, that’s asyncio-flavored. You can call trio_as_future() in a Trio-flavored function or even a synchronous context, as long as you plan to do something with the returned Future other than immediately awaiting it.

Cancelling the future will cancel the Trio task running your function, or prevent it from starting if that is still possible. If the Trio task exits due to this cancellation, the future will resolve to an asyncio.CancelledError.

Parameters:

  • proc – a Trio-flavored async function

  • args – arguments for proc

Returns:

an asyncio.Future which will resolve to the result of the call to proc

run_trio_task(proc, *args)

Start a new Trio task to run await proc(*args) asynchronously. If it raises an exception, allow the exception to propagate out of the trio-asyncio event loop (thus terminating it).

Parameters:

  • proc – a Trio-flavored async function

  • args – arguments for proc

Returns:

an asyncio.Handle which can be used to cancel the background task

await synchronize()

Suspend execution until all callbacks previously scheduled using call_soon() have been processed.

This is a Trio-flavored async function.

From asyncio, call await trio_as_aio(loop.synchronize)() instead of await asyncio.sleep(0) if you need to process all queued callbacks.

autoclose(fd)

Mark a file descriptor so that it’s auto-closed along with this loop.

This is a safety measure. You also should use appropriate finalizers.

Calling this method twice on the same file descriptor has no effect.

Parameters:

fd – Either an integer (Unix file descriptor) or an object with a fileno method providing one.

no_autoclose(fd)

Un-mark a file descriptor so that it’s no longer auto-closed along with this loop.

Call this method either before closing the file descriptor, or when passing it to code out of this loop’s scope.

Parameters:

fd – Either an integer (Unix file descriptor) or an object with a fileno() method providing one.

Raises:

KeyError – if the descriptor is not marked to be auto-closed.

await wait_stopped()

Wait until the event loop has stopped.

This is a Trio-flavored async function. You should call it from somewhere outside the async with open_loop() block to avoid a deadlock (the event loop can’t stop until all Trio tasks started within its scope have exited).

class trio_asyncio.TrioEventLoop(queue_len=None)

Bases: BaseTrioEventLoop

An asyncio event loop that runs on top of Trio, opened from within Trio code using open_loop().

trio_asyncio.current_loop

A contextvars.ContextVar whose value is the TrioEventLoop created by the nearest enclosing async with open_loop(): block. This is the same event loop that will be returned by calls to asyncio.get_event_loop(). If current_loop’s value is None, then asyncio.get_event_loop() will raise an error in Trio context. (Outside Trio context its value is always None and asyncio.get_event_loop() uses differnet logic.)

It is OK to modify this if you want the current scope to use a different trio-asyncio event loop, but make sure not to let your modifications leak past their intended scope.

await trio_asyncio.run_aio_future(future)

Wait for an asyncio-flavored future to become done, then return or raise its result.

Cancelling the current Trio scope will cancel the future. If this results in the future resolving to an asyncio.CancelledError exception, the call to run_aio_future() will raise trio.Cancelled. But if the future resolves to CancelledError when the current Trio scope was not cancelled, the CancelledError will be passed along unchanged.

This is a Trio-flavored async function.

async for ... in trio_asyncio.run_aio_generator(loop, async_generator)

Return a Trio-flavored async iterator which wraps the given asyncio-flavored async iterator (usually an async generator, but doesn’t have to be). The asyncio tasks that perform each iteration of async_generator will run in loop.

await trio_asyncio.run_aio_coroutine(coro)

Alias for a call to run_aio_coroutine() on the event loop returned by asyncio.get_event_loop().

This is a Trio-flavored async function which takes an asyncio-flavored coroutine object.

trio_asyncio.run_trio(proc, *args)

Alias for a call to trio_as_future() on the event loop returned by asyncio.get_event_loop().

This is a synchronous function which takes a Trio-flavored async function and returns an asyncio Future.

trio_asyncio.run_trio_task(proc, *args)

Alias for a call to run_trio_task() on the event loop returned by asyncio.get_event_loop().

This is a synchronous function which takes a Trio-flavored async function and returns nothing (the handle returned by BaseTrioEventLoop.run_trio_task is discarded). An uncaught error will propagate to, and terminate, the trio-asyncio loop.

exception trio_asyncio.TrioAsyncioDeprecationWarning

Warning emitted if you use deprecated trio-asyncio functionality.

This inherits from FutureWarning, not DeprecationWarning, for the same reasons described for trio.TrioDeprecationWarning.

Release history

trio-asyncio 0.15.0 (2024-04-24)

Features

  • trio-asyncio now properly finalizes asyncio-flavored async generators upon closure of the event loop. Previously, Trio’s async generator finalizers would try to finalize all async generators in Trio mode, regardless of their flavor, which could lead to spurious errors. (#92)

Bugfixes

  • trio-asyncio no longer raises a spurious “Event loop stopped before Future completed!” exception if a function passed to asyncio.run() calls sys.exit(). (#149)

trio-asyncio 0.14.1 (2024-04-18)

Bugfixes

Miscellaneous

  • Updated test suite to cope with Trio 0.25.0 and later defaulting strict_exception_groups to True. Trio 0.25.0 is now required to run the tests, although trio-asyncio itself still supports older versions. (#146)

trio-asyncio 0.14.0 (2024-02-07)

Features

  • trio-asyncio now implements its synchronous event loop (which is used when the top-level of your program is an asyncio call such as asyncio.run(), rather than a Trio call such as trio.run()) using the greenlet library rather than a separate thread. This provides some better theoretical grounding and fixes various edge cases around signal handling and other integrations; in particular, recent versions of IPython will no longer crash when importing trio-asyncio. Synchronous event loops have been un-deprecated with this change, though we still recommend using an async loop (async with trio_asyncio.open_loop(): from inside a Trio run) where possible. (#137)

  • trio-asyncio now better respects cancellation semantics for asyncio-to-Trio transitions. The asyncio caller now will not propagate cancellation until the Trio callee actually terminates, and only if the Trio callee terminates by cancellation (rather than, for example, finishing normally because the cancel arrived too late). Additionally, we no longer immediately cancel all Trio tasks started from asyncio context if the entire open_loop() is cancelled; they only become cancelled when their asyncio caller is cancelled, or when the body of the open_loop() terminates. (#140)

Bugfixes

  • trio-asyncio no longer applies a limit to the maximum number of asyncio callbacks (including new task creations) that can be enqueued near-simultaneously. Previously the default limit was 10,000. A limit may still be requested using the queue_len parameter to open_loop(), but this is discouraged because there is no way to fail gracefully upon exceeding the limit; it will most likely just crash your program. (#130)

  • Fix an issue where a call to TrioEventLoop.call_exception_handler() after the loop was closed would attempt to call a method on None. This pattern can be encountered if an aiohttp session is garbage-collected without being properly closed, for example. (#134)

trio-asyncio 0.13.0 (2023-12-01)

Features

  • Exiting an async with trio_asyncio.open_loop(): block now cancels any asyncio tasks that are still running in the background, like asyncio.run() does, so that they have a chance to clean up resources by running async context managers and finally blocks. Previously such tasks would simply be abandoned to the garbage collector, resulting in potential deadlocks and stderr spew. Note that, like asyncio.run(), we do still abandon any tasks that are started during this finalization phase and outlive the existing tasks. (#91)

Bugfixes

  • A deadlock will no longer occur if trio_asyncio.open_loop() is cancelled before its first checkpoint. We also now cancel and wait on all asyncio tasks even if open_loop() terminates due to an exception that was raised within the async with block. (#115)

  • Uncaught exceptions from asyncio tasks will now propagate out of the trio_asyncio.open_loop() call. This has always been the documented behavior, but didn’t actually work before. (#121)

  • Use of loop.add_reader() or loop.add_writer() with a socket object (rather than a file descriptor) will no longer potentially produce spurious uncaught exceptions if the socket is closed in the reader/writer callback. (#121)

Miscellaneous

  • trio-asyncio now requires Trio 0.22 and does not produce deprecation warnings. Python 3.12 is now supported. Python 3.6 and 3.7 are no longer supported. (#121)

  • trio-asyncio now indicates its presence to sniffio using the sniffio.thread_local interface that is preferred since sniffio v1.3.0. This should be less likely than the previous approach to cause sniffio.current_async_library() to return incorrect results due to unintended inheritance of contextvars. (#123)

trio-asyncio 0.12.0 (2021-01-07)

Bugfixes

  • trio-asyncio now cancels any Trio tasks that were started inside a trio-asyncio loop (using e.g. trio_as_aio()) before it allows the trio-asyncio loop to close. This should resolve some cases of deadlocks and “RuntimeError: Event loop is closed” when an async with open_loop(): block is cancelled. (#89)

  • asyncio.get_running_loop() will now return the trio-asyncio event loop (if running), instead of failing with RuntimeError. (#99)

  • On Python versions with native contextvars support (3.7+), a Trio task started from asyncio context (using trio_as_aio(), trio_as_future(), etc) will now properly inherit the contextvars of its caller. Also, if the entire trio-asyncio loop is cancelled, such tasks will no longer let trio.Cancelled exceptions leak into their asyncio caller. (#76)

  • Previously, cancelling the context surrounding an open_loop() block might cause a deadlock in some cases. The ordering of operations during loop teardown has been improved, so this shouldn’t happen anymore. (#81)

Deprecations and Removals

A number of functions deprecated since 0.10.0 are now removed:

Removed

Replacement

wrap_generator(proc, *args)

aio_as_trio(proc(*args))

run_iterator(aiter)

aio_as_trio(aiter)

trio2aio

aio_as_trio()

aio2trio

trio_as_aio()

run_future TrioEventLoop.run_future

run_aio_future()

run_coroutine TrioEventLoop.run_coroutine

run_aio_coroutine()

wrap_trio_context(ctx)

trio_as_aio(ctx)

TrioEventLoop.run_trio(proc, *args)

trio_as_aio(proc)(*args)

run_asyncio(proc, *args)

aio_as_trio(proc)(*args)

Miscellaneous

  • trio-asyncio now requires Trio 0.15. Support for Python < 3.6 has been removed. (#82)

  • No more TrioDeprecationWarning about trio.hazmat. (#82)

trio-asyncio 0.11.0 (2020-03-09)

Features

  • Substantially reorganize monkeypatching for asyncio event loop and event loop policy accessors, fixing support for Python 3.8. Also, stop using features deprecated in Trio 0.12. (#66)

Bugfixes

  • Calling loop.stop manually no longer causes a deadlock when exiting the context of trio_asyncio.open_loop (#58)

  • trio_asyncio.run() now properly returns whatever was returned by the async function it ran, like trio.run() does. (#57)

  • Replace uses of deprecated trio.open_cancel_scope() with trio.CancelScope.

Deprecations and Removals

  • The non-underscore-prefixed names of trio-asyncio submodules (trio_asyncio.loop, trio_asyncio.adapter, etc) have been deprecated; public names should be imported from trio_asyncio directly.

    trio_asyncio.current_policy, trio_asyncio.TrioChildWatcher, and trio_asyncio.TrioPolicy have been deprecated with no replacement. current_policy is no longer used at all, and the other two are singletons that can’t be customized so there’s no reason to make them publicly visible.

    A number of functions which were already documented as deprecated now raise the new TrioAsyncioDeprecationWarning where previously they provided either no runtime warning or a generic DeprecationWarning. (#64)

trio-asyncio 0.10.0 (2018-12-09)

Bugfixes

  • Replace deprecated trio.Queue with new channels, requiring Trio 0.9 or later. (#49)

trio-asyncio 0.9.1 (2018-09-06)

Bugfixes

  • Defer creating an asyncio coroutine until asyncio mode is actually entered. (#40)

trio-asyncio 0.9.0 (2018-08-31)

Features

Deprecations and Removals

  • run_asyncio() is deprecated: replace with a aio_as_trio() wrapper.

    trio2aio() is deprecated: replace with aio_as_trio().

    run_future() and TrioEventLoop.run_future() are deprecated: replace with run_aio_future().

    run_coroutine() and TrioEventLoop.run_coroutine() are deprecated: replace with run_aio_coroutine().

    TrioEventLoop.wrap_generator() is deprecated: replace with a aio_as_trio() wrapper.

    TrioEventLoop.run_iterator() is deprecated: replace with aio_as_trio(). (#36)

trio-asyncio 0.8.2 (2018-08-25)

Features

trio-asyncio 0.8.1 (2018-08-25)

Features

  • trio_asyncio now contains an allow_asyncio() wrapper which allows you to seamlessly mix asyncio and trio semantics:

    import asyncio
import trio
from trio_asyncio import run, allow_asyncio
async def main():
    print("Sleeping 1")
    await asyncio.sleep(1)
    print("Sleeping 2")
    await trio.sleep(1)
run(allow_asyncio, main)

    The problem with this solution is that Trio’s cancellations will no longer be converted to asyncio’s CancelledError within asyncio code. This may or may not be an issue for your code. (#30)

  • While the test suite still requires Python 3.6, trio_asyncio itself now works on Python 3.5.3 and later. (#33)

  • TrioEventLoop.run_asyncio() now supports wrapping async iterators and async context managers, in addition to its existing support for async functions.

trio-asyncio 0.8.0 (2018-08-03)

  • Add TrioEventLoop.run_iterator() as an alias for run_generator().

  • Add support for Python 3.7 via a monkey-patch to asyncio.set_event_loop_policy(). (#23)

  • Deprecate the use of “compatibility mode” / “sync event loops”, except as a tool for running the test suites of existing asyncio projects.

trio-asyncio 0.7.5 (2018-07-23)

  • Use a contextvar to represent the current trio-asyncio loop, rather than the deprecated trio.TaskLocal.

  • Use the outcome library rather than the deprecated trio.hazmat.Result.

  • Better handle errors in wrapped async generators.

trio-asyncio 0.7.0 (2018-03-27)

  • The @trio2aio and @aio2trio decorators now can be used to decorate both async generator functions and async functions that take keyword arguments.

  • open_loop() now takes an optional queue_len= parameter to specify the length of the internal callback queue (for performance tuning).

  • Add BaseTrioEventLoop.synchronize().

  • Monkey-patch asyncio.get_event_loop_policy() and asyncio.get_event_loop() so trio-asyncio works correctly in multithreaded programs that use a different asyncio event loop in other threads.

  • Add wrap_generator() and run_generator() which adapt an asyncio-flavored async generator to be used from Trio code.

trio-asyncio 0.5.0 (2018-02-20)

  • Support contextvars on Python 3.7 and later.

  • Support waiting for child processes even though Trio doesn’t (yet).

trio-asyncio 0.4.2 (2018-02-12)

  • Add trio_asyncio.run().

  • Fix a deadlock in SyncTrioEventLoop. Encourage people to use the async TrioEventLoop instead.

trio-asyncio 0.4.1 (2018-02-08)

  • Add TrioEventLoop.run_task() as an entry point for running Trio code in a context that allows asyncio calls, for use when neither a Trio nor an asyncio event loop is running.

trio-asyncio 0.4.0 (2018-02-07)

  • Add support for async loops (open_loop()) and encourage their use. Numerous functions renamed.

trio-asyncio 0.3.0 (2017-10-17)

  • Initial release.

Indices and tables