.. currentmodule:: asyncio
This section outlines high-level asyncio APIs to work with coroutines and Tasks.
:term:`Coroutines <coroutine>` declared with the async/await syntax is the preferred way of writing asyncio applications. For example, the following snippet of code prints "hello", waits 1 second, and then prints "world":
>>> import asyncio
>>> async def main():
... print('hello')
... await asyncio.sleep(1)
... print('world')
>>> asyncio.run(main())
hello
world
Note that simply calling a coroutine will not schedule it to be executed:
>>> main() <coroutine object main at 0x1053bb7c8>
To actually run a coroutine, asyncio provides three main mechanisms:
The :func:`asyncio.run` function to run the top-level entry point "main()" function (see the above example.)
Awaiting on a coroutine. The following snippet of code will print "hello" after waiting for 1 second, and then print "world" after waiting for another 2 seconds:
import asyncio import time async def say_after(delay, what): await asyncio.sleep(delay) print(what) async def main(): print(f"started at {time.strftime('%X')}") await say_after(1, 'hello') await say_after(2, 'world') print(f"finished at {time.strftime('%X')}") asyncio.run(main())Expected output:
started at 17:13:52 hello world finished at 17:13:55
The :func:`asyncio.create_task` function to run coroutines concurrently as asyncio :class:`Tasks <Task>`.
Let's modify the above example and run two
say_aftercoroutines concurrently:async def main(): task1 = asyncio.create_task( say_after(1, 'hello')) task2 = asyncio.create_task( say_after(2, 'world')) print(f"started at {time.strftime('%X')}") # Wait until both tasks are completed (should take # around 2 seconds.) await task1 await task2 print(f"finished at {time.strftime('%X')}")Note that expected output now shows that the snippet runs 1 second faster than before:
started at 17:14:32 hello world finished at 17:14:34
We say that an object is an awaitable object if it can be used in an :keyword:`await` expression. Many asyncio APIs are designed to accept awaitables.
There are three main types of awaitable objects: coroutines, Tasks, and Futures.
Coroutines
Python coroutines are awaitables and therefore can be awaited from other coroutines:
import asyncio
async def nested():
return 42
async def main():
# Nothing happens if we just call "nested()".
# A coroutine object is created but not awaited,
# so it *won't run at all*.
nested()
# Let's do it differently now and await it:
print(await nested()) # will print "42".
asyncio.run(main())
Important
In this documentation the term "coroutine" can be used for two closely related concepts:
- a coroutine function: an :keyword:`async def` function;
- a coroutine object: an object returned by calling a coroutine function.
asyncio also supports legacy :ref:`generator-based <asyncio_generator_based_coro>` coroutines.
Tasks
Tasks are used to schedule coroutines concurrently.
When a coroutine is wrapped into a Task with functions like :func:`asyncio.create_task` the coroutine is automatically scheduled to run soon:
import asyncio
async def nested():
return 42
async def main():
# Schedule nested() to run soon concurrently
# with "main()".
task = asyncio.create_task(nested())
# "task" can now be used to cancel "nested()", or
# can simply be awaited to wait until it is complete:
await task
asyncio.run(main())
Futures
A :class:`Future` is a special low-level awaitable object that represents an eventual result of an asynchronous operation.
When a Future object is awaited it means that the coroutine will wait until the Future is resolved in some other place.
Future objects in asyncio are needed to allow callback-based code to be used with async/await.
Normally there is no need to create Future objects at the application level code.
Future objects, sometimes exposed by libraries and some asyncio APIs, can be awaited:
async def main():
await function_that_returns_a_future_object()
# this is also valid:
await asyncio.gather(
function_that_returns_a_future_object(),
some_python_coroutine()
)
A good example of a low-level function that returns a Future object is :meth:`loop.run_in_executor`.
.. function:: run(coro, *, debug=False)
Execute the :term:`coroutine` *coro* and return the result.
This function runs the passed coroutine, taking care of
managing the asyncio event loop, *finalizing asynchronous
generators*, and closing the threadpool.
This function cannot be called when another asyncio event loop is
running in the same thread.
If *debug* is ``True``, the event loop will be run in debug mode.
This function always creates a new event loop and closes it at
the end. It should be used as a main entry point for asyncio
programs, and should ideally only be called once.
Example::
async def main():
await asyncio.sleep(1)
print('hello')
asyncio.run(main())
.. versionadded:: 3.7
.. versionchanged:: 3.9
Updated to use :meth:`loop.shutdown_default_executor`.
.. note::
The source code for ``asyncio.run()`` can be found in
:source:`Lib/asyncio/runners.py`.
.. function:: create_task(coro, *, name=None)
Wrap the *coro* :ref:`coroutine <coroutine>` into a :class:`Task`
and schedule its execution. Return the Task object.
If *name* is not ``None``, it is set as the name of the task using
:meth:`Task.set_name`.
The task is executed in the loop returned by :func:`get_running_loop`,
:exc:`RuntimeError` is raised if there is no running loop in
current thread.
.. important::
Save a reference to the result of this function, to avoid
a task disappearing mid execution.
.. versionadded:: 3.7
.. versionchanged:: 3.8
Added the ``name`` parameter.
.. coroutinefunction:: sleep(delay, result=None, *, loop=None)
Block for *delay* seconds.
If *result* is provided, it is returned to the caller
when the coroutine completes.
``sleep()`` always suspends the current task, allowing other tasks
to run.
Setting the delay to 0 provides an optimized path to allow other
tasks to run. This can be used by long-running functions to avoid
blocking the event loop for the full duration of the function call.
.. deprecated-removed:: 3.8 3.10
The *loop* parameter.
.. _asyncio_example_sleep:
Example of coroutine displaying the current date every second
for 5 seconds::
import asyncio
import datetime
async def display_date():
loop = asyncio.get_running_loop()
end_time = loop.time() + 5.0
while True:
print(datetime.datetime.now())
if (loop.time() + 1.0) >= end_time:
break
await asyncio.sleep(1)
asyncio.run(display_date())
.. awaitablefunction:: gather(*aws, loop=None, return_exceptions=False)
Run :ref:`awaitable objects <asyncio-awaitables>` in the *aws*
sequence *concurrently*.
If any awaitable in *aws* is a coroutine, it is automatically
scheduled as a Task.
If all awaitables are completed successfully, the result is an
aggregate list of returned values. The order of result values
corresponds to the order of awaitables in *aws*.
If *return_exceptions* is ``False`` (default), the first
raised exception is immediately propagated to the task that
awaits on ``gather()``. Other awaitables in the *aws* sequence
**won't be cancelled** and will continue to run.
If *return_exceptions* is ``True``, exceptions are treated the
same as successful results, and aggregated in the result list.
If ``gather()`` is *cancelled*, all submitted awaitables
(that have not completed yet) are also *cancelled*.
If any Task or Future from the *aws* sequence is *cancelled*, it is
treated as if it raised :exc:`CancelledError` -- the ``gather()``
call is **not** cancelled in this case. This is to prevent the
cancellation of one submitted Task/Future to cause other
Tasks/Futures to be cancelled.
.. deprecated-removed:: 3.8 3.10
The *loop* parameter.
.. _asyncio_example_gather:
Example::
import asyncio
async def factorial(name, number):
f = 1
for i in range(2, number + 1):
print(f"Task {name}: Compute factorial({number}), currently i={i}...")
await asyncio.sleep(1)
f *= i
print(f"Task {name}: factorial({number}) = {f}")
return f
async def main():
# Schedule three calls *concurrently*:
L = await asyncio.gather(
factorial("A", 2),
factorial("B", 3),
factorial("C", 4),
)
print(L)
asyncio.run(main())
# Expected output:
#
# Task A: Compute factorial(2), currently i=2...
# Task B: Compute factorial(3), currently i=2...
# Task C: Compute factorial(4), currently i=2...
# Task A: factorial(2) = 2
# Task B: Compute factorial(3), currently i=3...
# Task C: Compute factorial(4), currently i=3...
# Task B: factorial(3) = 6
# Task C: Compute factorial(4), currently i=4...
# Task C: factorial(4) = 24
# [2, 6, 24]
.. note::
If *return_exceptions* is False, cancelling gather() after it
has been marked done won't cancel any submitted awaitables.
For instance, gather can be marked done after propagating an
exception to the caller, therefore, calling ``gather.cancel()``
after catching an exception (raised by one of the awaitables) from
gather won't cancel any other awaitables.
.. versionchanged:: 3.7
If the *gather* itself is cancelled, the cancellation is
propagated regardless of *return_exceptions*.
.. awaitablefunction:: shield(aw, *, loop=None)
Protect an :ref:`awaitable object <asyncio-awaitables>`
from being :meth:`cancelled <Task.cancel>`.
If *aw* is a coroutine it is automatically scheduled as a Task.
The statement::
res = await shield(something())
is equivalent to::
res = await something()
*except* that if the coroutine containing it is cancelled, the
Task running in ``something()`` is not cancelled. From the point
of view of ``something()``, the cancellation did not happen.
Although its caller is still cancelled, so the "await" expression
still raises a :exc:`CancelledError`.
If ``something()`` is cancelled by other means (i.e. from within
itself) that would also cancel ``shield()``.
If it is desired to completely ignore cancellation (not recommended)
the ``shield()`` function should be combined with a try/except
clause, as follows::
try:
res = await shield(something())
except CancelledError:
res = None
.. deprecated-removed:: 3.8 3.10
The *loop* parameter.
.. coroutinefunction:: wait_for(aw, timeout, *, loop=None)
Wait for the *aw* :ref:`awaitable <asyncio-awaitables>`
to complete with a timeout.
If *aw* is a coroutine it is automatically scheduled as a Task.
*timeout* can either be ``None`` or a float or int number of seconds
to wait for. If *timeout* is ``None``, block until the future
completes.
If a timeout occurs, it cancels the task and raises
:exc:`asyncio.TimeoutError`.
To avoid the task :meth:`cancellation <Task.cancel>`,
wrap it in :func:`shield`.
The function will wait until the future is actually cancelled,
so the total wait time may exceed the *timeout*. If an exception
happens during cancellation, it is propagated.
If the wait is cancelled, the future *aw* is also cancelled.
.. deprecated-removed:: 3.8 3.10
The *loop* parameter.
.. _asyncio_example_waitfor:
Example::
async def eternity():
# Sleep for one hour
await asyncio.sleep(3600)
print('yay!')
async def main():
# Wait for at most 1 second
try:
await asyncio.wait_for(eternity(), timeout=1.0)
except asyncio.TimeoutError:
print('timeout!')
asyncio.run(main())
# Expected output:
#
# timeout!
.. versionchanged:: 3.7
When *aw* is cancelled due to a timeout, ``wait_for`` waits
for *aw* to be cancelled. Previously, it raised
:exc:`asyncio.TimeoutError` immediately.
.. coroutinefunction:: wait(aws, *, loop=None, timeout=None,\
return_when=ALL_COMPLETED)
Run :ref:`awaitable objects <asyncio-awaitables>` in the *aws*
iterable concurrently and block until the condition specified
by *return_when*.
The *aws* iterable must not be empty.
Returns two sets of Tasks/Futures: ``(done, pending)``.
Usage::
done, pending = await asyncio.wait(aws)
*timeout* (a float or int), if specified, can be used to control
the maximum number of seconds to wait before returning.
Note that this function does not raise :exc:`asyncio.TimeoutError`.
Futures or Tasks that aren't done when the timeout occurs are simply
returned in the second set.
*return_when* indicates when this function should return. It must
be one of the following constants:
.. tabularcolumns:: |l|L|
+-----------------------------+----------------------------------------+
| Constant | Description |
+=============================+========================================+
| :const:`FIRST_COMPLETED` | The function will return when any |
| | future finishes or is cancelled. |
+-----------------------------+----------------------------------------+
| :const:`FIRST_EXCEPTION` | The function will return when any |
| | future finishes by raising an |
| | exception. If no future raises an |
| | exception then it is equivalent to |
| | :const:`ALL_COMPLETED`. |
+-----------------------------+----------------------------------------+
| :const:`ALL_COMPLETED` | The function will return when all |
| | futures finish or are cancelled. |
+-----------------------------+----------------------------------------+
Unlike :func:`~asyncio.wait_for`, ``wait()`` does not cancel the
futures when a timeout occurs.
.. deprecated:: 3.8
If any awaitable in *aws* is a coroutine, it is automatically
scheduled as a Task. Passing coroutines objects to
``wait()`` directly is deprecated as it leads to
:ref:`confusing behavior <asyncio_example_wait_coroutine>`.
.. deprecated-removed:: 3.8 3.10
The *loop* parameter.
.. _asyncio_example_wait_coroutine:
.. note::
``wait()`` schedules coroutines as Tasks automatically and later
returns those implicitly created Task objects in ``(done, pending)``
sets. Therefore the following code won't work as expected::
async def foo():
return 42
coro = foo()
done, pending = await asyncio.wait({coro})
if coro in done:
# This branch will never be run!
Here is how the above snippet can be fixed::
async def foo():
return 42
task = asyncio.create_task(foo())
done, pending = await asyncio.wait({task})
if task in done:
# Everything will work as expected now.
.. deprecated-removed:: 3.8 3.11
Passing coroutine objects to ``wait()`` directly is
deprecated.
.. function:: as_completed(aws, *, loop=None, timeout=None)
Run :ref:`awaitable objects <asyncio-awaitables>` in the *aws*
iterable concurrently. Return an iterator of coroutines.
Each coroutine returned can be awaited to get the earliest next
result from the iterable of the remaining awaitables.
Raises :exc:`asyncio.TimeoutError` if the timeout occurs before
all Futures are done.
.. deprecated-removed:: 3.8 3.10
The *loop* parameter.
Example::
for coro in as_completed(aws):
earliest_result = await coro
# ...
.. coroutinefunction:: to_thread(func, /, *args, **kwargs)
Asynchronously run function *func* in a separate thread.
Any \*args and \*\*kwargs supplied for this function are directly passed
to *func*. Also, the current :class:`contextvars.Context` is propagated,
allowing context variables from the event loop thread to be accessed in the
separate thread.
Return a coroutine that can be awaited to get the eventual result of *func*.
This coroutine function is primarily intended to be used for executing
IO-bound functions/methods that would otherwise block the event loop if
they were ran in the main thread. For example::
def blocking_io():
print(f"start blocking_io at {time.strftime('%X')}")
# Note that time.sleep() can be replaced with any blocking
# IO-bound operation, such as file operations.
time.sleep(1)
print(f"blocking_io complete at {time.strftime('%X')}")
async def main():
print(f"started main at {time.strftime('%X')}")
await asyncio.gather(
asyncio.to_thread(blocking_io),
asyncio.sleep(1))
print(f"finished main at {time.strftime('%X')}")
asyncio.run(main())
# Expected output:
#
# started main at 19:50:53
# start blocking_io at 19:50:53
# blocking_io complete at 19:50:54
# finished main at 19:50:54
Directly calling `blocking_io()` in any coroutine would block the event loop
for its duration, resulting in an additional 1 second of run time. Instead,
by using `asyncio.to_thread()`, we can run it in a separate thread without
blocking the event loop.
.. note::
Due to the :term:`GIL`, `asyncio.to_thread()` can typically only be used
to make IO-bound functions non-blocking. However, for extension modules
that release the GIL or alternative Python implementations that don't
have one, `asyncio.to_thread()` can also be used for CPU-bound functions.
.. versionadded:: 3.9
.. function:: run_coroutine_threadsafe(coro, loop)
Submit a coroutine to the given event loop. Thread-safe.
Return a :class:`concurrent.futures.Future` to wait for the result
from another OS thread.
This function is meant to be called from a different OS thread
than the one where the event loop is running. Example::
# Create a coroutine
coro = asyncio.sleep(1, result=3)
# Submit the coroutine to a given loop
future = asyncio.run_coroutine_threadsafe(coro, loop)
# Wait for the result with an optional timeout argument
assert future.result(timeout) == 3
If an exception is raised in the coroutine, the returned Future
will be notified. It can also be used to cancel the task in
the event loop::
try:
result = future.result(timeout)
except asyncio.TimeoutError:
print('The coroutine took too long, cancelling the task...')
future.cancel()
except Exception as exc:
print(f'The coroutine raised an exception: {exc!r}')
else:
print(f'The coroutine returned: {result!r}')
See the :ref:`concurrency and multithreading <asyncio-multithreading>`
section of the documentation.
Unlike other asyncio functions this function requires the *loop*
argument to be passed explicitly.
.. versionadded:: 3.5.1
.. function:: current_task(loop=None) Return the currently running :class:`Task` instance, or ``None`` if no task is running. If *loop* is ``None`` :func:`get_running_loop` is used to get the current loop. .. versionadded:: 3.7
.. function:: all_tasks(loop=None) Return a set of not yet finished :class:`Task` objects run by the loop. If *loop* is ``None``, :func:`get_running_loop` is used for getting current loop. .. versionadded:: 3.7
Note
Support for generator-based coroutines is deprecated and is removed in Python 3.11.
Generator-based coroutines predate async/await syntax. They are
Python generators that use yield from expressions to await
on Futures and other coroutines.
Generator-based coroutines should be decorated with :func:`@asyncio.coroutine <asyncio.coroutine>`, although this is not enforced.
.. decorator:: coroutine
Decorator to mark generator-based coroutines.
This decorator enables legacy generator-based coroutines to be
compatible with async/await code::
@asyncio.coroutine
def old_style_coroutine():
yield from asyncio.sleep(1)
async def main():
await old_style_coroutine()
This decorator should not be used for :keyword:`async def`
coroutines.
.. deprecated-removed:: 3.8 3.11
Use :keyword:`async def` instead.
.. function:: iscoroutine(obj) Return ``True`` if *obj* is a :ref:`coroutine object <coroutine>`. This method is different from :func:`inspect.iscoroutine` because it returns ``True`` for generator-based coroutines.
.. function:: iscoroutinefunction(func) Return ``True`` if *func* is a :ref:`coroutine function <coroutine>`. This method is different from :func:`inspect.iscoroutinefunction` because it returns ``True`` for generator-based coroutine functions decorated with :func:`@coroutine <coroutine>`.