当两个异步任务访问同一个可爱的对象时,是否安全?

时间:2021-09-10 10:12:28

Simply speaking, thread-safe means that it is safe when more than one thread access the same resource and I know Asyncio use a single thread fundamentally.

简单地说,线程安全意味着当多个线程访问相同的资源时是安全的,我知道Asyncio基本上只使用一个线程。

However, more than one Asyncio Task could access a resource multiple time at a time like multi-threading.

但是,多个Asyncio任务可以在多线程的时候访问资源。

For example DB connection(if the object is not thread-safe and supports Asyncio operation).

例如,DB连接(如果对象不是线程安全的,并且支持Asyncio操作)。

  1. Schedule Task A and Task B accessing the same DB object.
  2. 安排任务A和任务B访问相同的DB对象。
  3. IO Loop executes Task A.
  4. IO循环执行任务A。
  5. Task A await IO operation on the DB object.(it will take long time enough)
  6. 任务A等待DB对象上的IO操作。(这需要很长时间)
  7. IO Loop executes Task B
  8. IO循环执行任务B。
  9. Step3's IO operation is still in progress(not done).
  10. Step3的IO操作仍在进行中(未完成)。
  11. Task B await IO operation on the same DB object.
  12. 任务B等待同一个DB对象上的IO操作。
  13. Now Task B is trying to access the same object at a time.
  14. 现在任务B试图一次访问相同的对象。

Is it completely safe in Asyncio and if so, what does it make safe?

在Asyncio中它是完全安全的吗?如果是,什么是安全的呢?

2 个解决方案

#1


2  

Using the same asyncio object from multiple tasks is safe in general. As an example, aiohttp has a session object, and it is expected for multiple tasks to access the same session "in parallel".

从多个任务中使用相同的asyncio对象通常是安全的。例如,aiohttp有一个会话对象,多个任务可以“并行”访问同一个会话。

if so, what does it make safe?

如果是的话,什么是安全的呢?

The basic architecture of asyncio allows for multiple coroutines to await a single future result - they will simply all subscribe to the future's completion, and all will be scheduled to run once the result is ready. And this applies not only to coroutines, but also to synchronous code that subscribes to the future using add_done_callback.

asyncio的基本架构允许多个coroutines等待一个未来的结果——它们只会订阅未来的完成,并且所有的都将在结果就绪后运行。这不仅适用于coroutines,也适用于使用add_done_callback订阅未来的同步代码。

That is how asyncio will handle your scenario: tasks A and B will ultimately subscribe to some future awaited by the DB object and. Once the result is available, it will be delivered to both of them, in turn.

这就是asyncio处理场景的方式:任务A和任务B最终会订阅DB对象和。一旦结果可用,它将依次传递给两者。

Pitfalls typically associated with multi-threaded programming do not apply to asyncio because:

通常与多线程编程相关的陷阱不适用于asyncio,因为:

  • Unlike with threads, it is very predictable where a context switch can occur - just look at await statements in the code (and also async with and async for - but those are still very visible keywords). Anything between them is, for all intents and purposes, atomic. This eliminates the need for synchronization primitives to protect objects, as well as the mistakes that result from mishandling such tools.

    与线程不同的是,上下文切换可以在什么地方发生是非常可预测的——只需查看代码中的wait语句(也可以使用异步和异步),但这些都是非常可见的关键字。它们之间的任何东西,无论出于什么目的,都是原子的。这消除了需要同步原语来保护对象,以及由于错误处理此类工具而导致的错误。

  • All access to data happens from the thread that runs the event loop. This eliminates the possibility of a data race, reading of shared memory that is being concurrently written to.

    所有对数据的访问都发生在运行事件循环的线程中。这消除了数据竞争的可能性,即读取并发写入的共享内存。

One scenario in which multi-tasking could fail is multiple consumers attaching to the same stream-like resource. For example, if several tasks try to await reader.read(n) on the same reader stream, exactly one of them will get the new data1, and the others will keep waiting until new data arrives. The same applies to any shared streaming resource, including file descriptors or generators shared by multiple objects. And even then, one of the tasks is guaranteed to obtain the data, and the integrity of the stream object will not be compromised in any way.

多任务处理可能失败的一种情况是多个使用者附加到类似于流的资源。例如,如果有几个任务试图等待reader.read(n)在同一个阅读器流上,其中一个任务将获得新的data1,其他任务将继续等待新的数据到达。这同样适用于任何共享流资源,包括由多个对象共享的文件描述符或生成器。而且,即使这样,其中一个任务仍然保证能够获得数据,并且流对象的完整性不会受到任何影响。


1 One task receiving the data only applies if the tasks share the reader and each task separately calls data = await reader.read(n). If one were to extract a future with fut = reader.read(n) ( without using await), share the future among multiple tasks, and await it in each task with data = await fut, all tasks would be notified of the particular chunk of data that ends up returned by that future.

#2


0  

No, asyncio is not thread safe. Generally only one thread should have control over an event loop and/or a resource associated to the event loop. If some other thread wants to access it, it should do it via special methods, like call_soon_threadsafe.

不,asyncio不是线程安全的。通常只有一个线程应该拥有对事件循环和/或与事件循环关联的资源的控制。如果其他线程想要访问它,它应该通过特殊的方法来访问,比如call_soon_threadsafe。

#1


2  

Using the same asyncio object from multiple tasks is safe in general. As an example, aiohttp has a session object, and it is expected for multiple tasks to access the same session "in parallel".

从多个任务中使用相同的asyncio对象通常是安全的。例如,aiohttp有一个会话对象,多个任务可以“并行”访问同一个会话。

if so, what does it make safe?

如果是的话,什么是安全的呢?

The basic architecture of asyncio allows for multiple coroutines to await a single future result - they will simply all subscribe to the future's completion, and all will be scheduled to run once the result is ready. And this applies not only to coroutines, but also to synchronous code that subscribes to the future using add_done_callback.

asyncio的基本架构允许多个coroutines等待一个未来的结果——它们只会订阅未来的完成,并且所有的都将在结果就绪后运行。这不仅适用于coroutines,也适用于使用add_done_callback订阅未来的同步代码。

That is how asyncio will handle your scenario: tasks A and B will ultimately subscribe to some future awaited by the DB object and. Once the result is available, it will be delivered to both of them, in turn.

这就是asyncio处理场景的方式:任务A和任务B最终会订阅DB对象和。一旦结果可用,它将依次传递给两者。

Pitfalls typically associated with multi-threaded programming do not apply to asyncio because:

通常与多线程编程相关的陷阱不适用于asyncio,因为:

  • Unlike with threads, it is very predictable where a context switch can occur - just look at await statements in the code (and also async with and async for - but those are still very visible keywords). Anything between them is, for all intents and purposes, atomic. This eliminates the need for synchronization primitives to protect objects, as well as the mistakes that result from mishandling such tools.

    与线程不同的是,上下文切换可以在什么地方发生是非常可预测的——只需查看代码中的wait语句(也可以使用异步和异步),但这些都是非常可见的关键字。它们之间的任何东西,无论出于什么目的,都是原子的。这消除了需要同步原语来保护对象,以及由于错误处理此类工具而导致的错误。

  • All access to data happens from the thread that runs the event loop. This eliminates the possibility of a data race, reading of shared memory that is being concurrently written to.

    所有对数据的访问都发生在运行事件循环的线程中。这消除了数据竞争的可能性,即读取并发写入的共享内存。

One scenario in which multi-tasking could fail is multiple consumers attaching to the same stream-like resource. For example, if several tasks try to await reader.read(n) on the same reader stream, exactly one of them will get the new data1, and the others will keep waiting until new data arrives. The same applies to any shared streaming resource, including file descriptors or generators shared by multiple objects. And even then, one of the tasks is guaranteed to obtain the data, and the integrity of the stream object will not be compromised in any way.

多任务处理可能失败的一种情况是多个使用者附加到类似于流的资源。例如,如果有几个任务试图等待reader.read(n)在同一个阅读器流上,其中一个任务将获得新的data1,其他任务将继续等待新的数据到达。这同样适用于任何共享流资源,包括由多个对象共享的文件描述符或生成器。而且,即使这样,其中一个任务仍然保证能够获得数据,并且流对象的完整性不会受到任何影响。


1 One task receiving the data only applies if the tasks share the reader and each task separately calls data = await reader.read(n). If one were to extract a future with fut = reader.read(n) ( without using await), share the future among multiple tasks, and await it in each task with data = await fut, all tasks would be notified of the particular chunk of data that ends up returned by that future.

#2


0  

No, asyncio is not thread safe. Generally only one thread should have control over an event loop and/or a resource associated to the event loop. If some other thread wants to access it, it should do it via special methods, like call_soon_threadsafe.

不,asyncio不是线程安全的。通常只有一个线程应该拥有对事件循环和/或与事件循环关联的资源的控制。如果其他线程想要访问它,它应该通过特殊的方法来访问,比如call_soon_threadsafe。