如何从Python异步运行外部命令?

I need to run a shell command asynchronously from a Python script. By this I mean that I want my Python script to continue running while the external command goes off and does whatever it needs to do.

我需要从Python脚本异步运行shell命令。我的意思是，我希望我的Python脚本可以在外部命令关闭时继续运行，并执行它需要做的任何事情。

I read this post:

我读了这篇文章:

Calling an external command in Python

在Python中调用外部命令。

I then went off and did some testing, and it looks like os.system() will do the job provided that I use & at the end of the command so that I don't have to wait for it to return. What I am wondering is if this is the proper way to accomplish such a thing? I tried commands.call() but it will not work for me because it blocks on the external command.

然后我去做了一些测试，看起来os.system()将完成我在命令末尾使用&的工作，这样我就不必等待它返回。我想知道的是，这是否是完成这样一件事的合适方式?我尝试过commands.call()，但它对我不起作用，因为它会阻塞外部命令。

Please let me know if using os.system() for this is advisable or if I should try some other route.

请告诉我使用os.system()是否明智，或者我是否应该尝试其他路线。

7 个解决方案

#1

subprocess.Popen does exactly what you want.

子流程。Popen做的正是你想要的。

from subprocess import Popen
p = Popen(['watch', 'ls']) # something long running
# ... do other stuff while subprocess is running
p.terminate()

(Edit to complete the answer from comments)

(编辑完成注释中的答案)

The Popen instance can do various other things like you can poll() it to see if it is still running, and you can communicate() with it to send it data on stdin, and wait for it to terminate.

Popen实例可以执行各种其他操作，比如轮询()它以查看它是否仍在运行，还可以与它通信()在stdin上发送数据，并等待它终止。

#2

If you want to run many processes in parallel and then handle them when they yield results, you can use polling like in the following:

如果您希望并行运行多个进程，并在它们产生结果时处理它们，您可以使用如下的轮询:

from subprocess import Popen, PIPE
import time

running_procs = [
    Popen(['/usr/bin/my_cmd', '-i %s' % path], stdout=PIPE, stderr=PIPE)
    for path in '/tmp/file0 /tmp/file1 /tmp/file2'.split()]

while running_procs:
    for proc in running_procs:
        retcode = proc.poll()
        if retcode is not None: # Process finished.
            running_procs.remove(proc)
            break
        else: # No process is done, wait a bit and check again.
            time.sleep(.1)
            continue

    # Here, `proc` has finished with return code `retcode`
    if retcode != 0:
        """Error handling."""
    handle_results(proc.stdout)

The control flow there is a little bit convoluted because I'm trying to make it small -- you can refactor to your taste. :-)

这里的控制流程有点复杂，因为我想让它更小，你可以根据自己的喜好进行重构。:-)

This has the advantage of servicing the early-finishing requests first. If you call communicate on the first running process and that turns out to run the longest, the other running processes will have been sitting there idle when you could have been handling their results.

这样做的好处是先为提前完成的请求提供服务。如果您在第一个正在运行的进程上调用通信，结果发现它运行的时间最长，那么当您可以处理它们的结果时，其他正在运行的进程将一直处于空闲状态。

#3

What I am wondering is if this [os.system()] is the proper way to accomplish such a thing?

我想知道的是，这个[os.system()]是完成这样一件事的合适方法吗?

No. os.system() is not the proper way. That's why everyone says to use subprocess.

不。os.system()不是正确的方法。这就是为什么每个人都说要使用子过程。

For more information, read http://docs.python.org/library/os.html#os.system

要了解更多信息，请阅读http://docs.python.org/library/os.html#os.system

The subprocess module provides more powerful facilities for spawning new processes and retrieving their results; using that module is preferable to using this function. Use the subprocess module. Check especially the Replacing Older Functions with the subprocess Module section.

子过程模块提供了更强大的工具来生成新进程和检索结果;使用该模块比使用此函数更可取。使用子流程模块。特别是检查用子过程模块部分替换旧函数。

#4

Using pexpect [ http://www.noah.org/wiki/Pexpect ] with non-blocking readlines is another way to do this. Pexpect solves the deadlock problems, allows you to easily run the processes in the background, and gives easy ways to have callbacks when your process spits out predefined strings, and generally makes interacting with the process much easier.

使用pexpect [http://www.noah.org/wiki/Pexpect]和非阻塞readline是另一种方法。Pexpect解决了死锁问题，允许您轻松地在后台运行进程，并提供了在进程输出预定义字符串时进行回调的简单方法，并且通常使与进程的交互更容易。

#5

I've had good success with the asyncproc module, which deals nicely with the output from the processes. For example:

我已经成功地使用了asyncproc模块，它可以很好地处理来自进程的输出。例如:

import os
from asynproc import Process
myProc = Process("myprogram.app")

while True:
    # check to see if process has ended
    poll = myProc.wait(os.WNOHANG)
    if poll is not None:
        break
    # print any new output
    out = myProc.read()
    if out != "":
        print out

#6

I have the same problem trying to connect to an 3270 terminal using the s3270 scripting software in Python. Now I'm solving the problem with an subclass of Process that I found here:

我在尝试使用Python中的s3270脚本软件连接到3270终端时遇到了同样的问题。现在我用我在这里找到的一个过程的子类来解决这个问题:

http://code.activestate.com/recipes/440554/

And here is the sample taken from file:

这是来自文件的样本:

def recv_some(p, t=.1, e=1, tr=5, stderr=0):
    if tr < 1:
        tr = 1
    x = time.time()+t
    y = []
    r = ''
    pr = p.recv
    if stderr:
        pr = p.recv_err
    while time.time() < x or r:
        r = pr()
        if r is None:
            if e:
                raise Exception(message)
            else:
                break
        elif r:
            y.append(r)
        else:
            time.sleep(max((x-time.time())/tr, 0))
    return ''.join(y)

def send_all(p, data):
    while len(data):
        sent = p.send(data)
        if sent is None:
            raise Exception(message)
        data = buffer(data, sent)

if __name__ == '__main__':
    if sys.platform == 'win32':
        shell, commands, tail = ('cmd', ('dir /w', 'echo HELLO WORLD'), '\r\n')
    else:
        shell, commands, tail = ('sh', ('ls', 'echo HELLO WORLD'), '\n')

    a = Popen(shell, stdin=PIPE, stdout=PIPE)
    print recv_some(a),
    for cmd in commands:
        send_all(a, cmd + tail)
        print recv_some(a),
    send_all(a, 'exit' + tail)
    print recv_some(a, e=0)
    a.wait()

#7

Considering "I don't have to wait for it to return", one of the easiest solutions will be this:

考虑到“我不必等它回来”，最简单的解决方案之一将是:

subprocess.Popen( \
    [path_to_executable, arg1, arg2, ... argN],
    creationflags = subprocess.CREATE_NEW_CONSOLE,
).pid

But... From what I read this is not "the proper way to accomplish such a thing" because of security risks created by subprocess.CREATE_NEW_CONSOLE flag.

但是…从我读到的内容来看，这并不是“完成这样一件事的正确方式”，因为子进程会产生安全风险。CREATE_NEW_CONSOLE国旗。

The key things that happen here is use of subprocess.CREATE_NEW_CONSOLE to create new console and .pid (returns process ID so that you could check program later on if you want to) so that not to wait for program to finish its job.

这里发生的关键事情是使用子过程。CREATE_NEW_CONSOLE创建新的控制台和.pid(返回进程ID，以便您可以在以后检查程序(如果您愿意))，这样就不会等待程序完成它的工作。

#1