在Python 3.5中协程和未来/任务之间的区别？

Let's say we have a dummy function:

假设我们有一个虚函数：

async def foo(arg):
    result = await some_remote_call(arg)
    return result.upper()

What's the difference between:

有什么区别：

coros = []
for i in range(5):
    coros.append(foo(i))

loop = get_event_loop()
loop.run_until_complete(wait(coros))

And:

和：

from asyncio import ensure_future

futures = []
for i in range(5):
    futures.append(ensure_future(foo(i)))

loop = get_event_loop()
loop.run_until_complete(wait(futures))

Note: The example returns a result, but this isn't the focus of the question. When return value matters, use gather() instead of wait().

注意：该示例返回结果，但这不是问题的焦点。当返回值很重要时，使用gather（）而不是wait（）。

Regardless of return value, I'm looking for clarity on ensure_future(). wait(coros) and wait(futures) both run the coroutines, so when and why should a coroutine be wrapped in ensure_future?

无论返回值如何，我都在寻找Ensure_future（）的清晰度。等待（coros）和等待（期货）都运行协同程序，所以何时以及为什么一个协程应该包含在ensure_future中？

Basically, what's the Right Way (tm) to run a bunch of non-blocking operations using Python 3.5's async?

基本上，使用Python 3.5的异步运行一堆非阻塞操作的正确方法（tm）是什么？

For extra credit, what if I want to batch the calls? For example, I need to call some_remote_call(...) 1000 times, but I don't want to crush the web server/database/etc with 1000 simultaneous connections. This is doable with a thread or process pool, but is there a way to do this with asyncio?

如果我需要批量拨打电话，还需要额外的信用额度？例如，我需要调用some_remote_call（...）1000次，但我不想粉碎web服务器/数据库/等1000个同时连接。这对于线程或进程池是可行的，但有没有办法用asyncio执行此操作？

2 个解决方案

#1

A coroutine is a generator function that can both yield values and accept values from the outside. The benefit of using a coroutine is that we can pause the execution of a function and resume it later. In case of a network operation, it makes sense to pause the execution of a function while we're waiting for the response. We can use the time to run some other functions.

协程是一个生成器函数，它既可以产生值，也可以接受来自外部的值。使用协程的好处是我们可以暂停函数的执行并在以后恢复它。在网络操作的情况下，在我们等待响应时暂停执行功能是有意义的。我们可以利用时间来运行其他一些功能。

A future is like the Promise objects from Javascript. It is like a place holder for a value that will be materialized in the future. In the above mentioned case, while waiting on network I/O, a function can give us a container, a promise that it will fill the container with the value when the operation completes. We hold on to the future object and when it's fulfilled, we can call a method on it to retrieve the actual result.

未来就像来自Javascript的Promise对象。它就像一个价值的占位符，将来会实现。在上面提到的情况下，在等待网络I / O时，一个函数可以给我们一个容器，一个承诺，它将在操作完成时用容器填充容器。我们坚持未来的对象，当它实现时，我们可以调用一个方法来检索实际的结果。

Direct Answer: You don't need ensure_future if you don't need the results. They are good if you need the results or retrieve exceptions occured.

直接答案：如果您不需要结果，则不需要ensure_future。如果您需要结果或检索发生的异常，它们都很好。

Extra Credits: I would choose run_in_executor and pass an Executor instance to control the number of max workers.

额外积分：我会选择run_in_executor并传递Executor实例来控制最大工作人员的数量。

Explanations and Sample codes

In the first example, you are using coroutines. The wait function takes a bunch of coroutines and combines them together. So wait() finishes when all the coroutines are exhausted (completed/finished returning all the values).

在第一个示例中，您使用的是协同程序。 wait函数需要一堆协程并将它们组合在一起。所以当所有协程都用尽时（完成/完成返回所有值），wait（）结束。

loop = get_event_loop() # 
loop.run_until_complete(wait(coros))

The run_until_complete method would make sure that the loop is alive until the execution is finished. Please notice how you are not getting the results of the async execution in this case.

run_until_complete方法将确保循环处于活动状态，直到执行完成。请注意在这种情况下您是如何得不到异步执行的结果的。

In the second example, you are using the ensure_future function to wrap a coroutine and return a Task object which is a kind of Future. The coroutine is scheduled to be executed in the main event loop when you call ensure_future. The returned future/task object doesn't yet have a value but over time, when the network operations finish, the future object will hold the result of the operation.

在第二个示例中，您使用ensure_future函数来包装协程并返回一个Task对象，这是一种Future。当您调用ensure_future时，协程计划在主事件循环中执行。返回的future / task对象还没有值，但随着时间的推移，当网络操作完成时，future对象将保存操作的结果。

from asyncio import ensure_future

futures = []
for i in range(5):
    futures.append(ensure_future(foo(i)))

loop = get_event_loop()
loop.run_until_complete(wait(futures))

So in this example, we're doing the same thing except we're using futures instead of just using coroutines.

所以在这个例子中，我们做同样的事情，除了我们使用期货而不仅仅是使用协同程序。

Let's look at an example on how to use asyncio/coroutines/futures:

我们来看一个如何使用asyncio / coroutines / futures的例子：

import asyncio


async def slow_operation():
    await asyncio.sleep(1)
    return 'Future is done!'


def got_result(future):
    print(future.result())

    # We have result, so let's stop
    loop.stop()


loop = asyncio.get_event_loop()
task = loop.create_task(slow_operation())
task.add_done_callback(got_result)

# We run forever
loop.run_forever()

Here, we have used the create_task method on the loop object. ensure_future would schedule the task in the main event loop. This method enables us to schedule a coroutine on a loop we choose.

在这里，我们在循环对象上使用了create_task方法。 ensure_future会在主事件循环中安排任务。这种方法使我们能够在我们选择的循环上安排一个协同程序。

We also see the concept of adding a callback using the add_done_callback method on the task object.

我们还看到了在任务对象上使用add_done_callback方法添加回调的概念。

A Task is done when the coroutine returns a value, raises an exception or gets cancelled. There are methods to check these incidents.

当协同程序返回值，引发异常或被取消时，将执行任务。有方法可以检查这些事件。

I have written some blog posts on these topics which might help:

我写了一些关于这些主题的博客文章可能有所帮助：

• http://masnun.com/2015/11/13/python-generators-coroutines-native-coroutines-and-async-await.html
• http://masnun.com/2015/11/13/python-generators-coroutines-native-coroutines-and-async-await.html
• http://masnun.com/2015/11/20/python-asyncio-future-task-and-the-event-loop.html
• http://masnun.com/2015/11/20/python-asyncio-future-task-and-the-event-loop.html
• http://masnun.com/2015/12/07/python-3-using-blocking-functions-or-codes-with-asyncio.html
• http://masnun.com/2015/12/07/python-3-using-blocking-functions-or-codes-with-asyncio.html

Of course, you can find more details on the official manual: https://docs.python.org/3/library/asyncio.html

当然，您可以在官方手册中找到更多详细信息：https：//docs.python.org/3/library/asyncio.html

#2

A comment by Vincent linked to https://github.com/python/asyncio/blob/master/asyncio/tasks.py#L346, which shows that wait() wraps the coroutines in ensure_future() for you!

文森特的评论链接到https://github.com/python/asyncio/blob/master/asyncio/tasks.py#L346，这表明wait（）为你包装了ensure_future（）中的协同程序！

In other words, we do need a future, and coroutines will be silently transformed into them.

换句话说，我们确实需要一个未来，协同作用将被默默地转化为它们。

I'll update this answer when I find a definitive explanation of how to batch coroutines/futures.

当我找到如何批处理协程/期货的确切解释时，我会更新这个答案。