A python script is controlling an external application on Linux, passing in input via a pipe to the external applications stdin, and reading output via a pipe from the external applications stdout.
python脚本控制Linux上的一个外部应用程序，通过管道将输入传递给外部应用程序stdin，并通过管道从外部应用程序stdout读取输出。
The problem is that writes to pipes are buffered by block, and not by line, and therefore delays occur before the controlling script receives data output by, for example, printf in the external application.
问题是对管道的写入是按块而不是按行进行缓冲的，因此在控制脚本接收数据输出(例如，外部应用程序中的printf)之前会发生延迟。
The external application cannot be altered to add explicit fflush(0) calls.
不能更改外部应用程序以添加显式的fflush(0)调用。
How can the pty module of the python standard library be used with the subprocess module to achieve this?
python标准库的pty模块如何与子流程模块一起使用以实现这一点?
7 个解决方案
#1
6
You can use PTYs to solve this by:
你可以用PTYs来解决这个问题:
- Creating a pty master/slave pair;
- 创建一个pty主/从对;
- Connecting the child process's stdin, stdout and stderr to the pty slave device;
- 将子进程的stdin、stdout和stderr连接到pty从设备;
- Reading from and writing to the pty master in the parent.
- 阅读并给家长的家长写信。
#2
5
Doing this is possible, but the only solution I can think of is fairly convoluted, non-portable, and probably fraught with problematic details. You can use LD_PRELOAD to cause the external application to load a dynamic library which contains a constructor that invokes setvbuf to unbuffer stdout. You will probably also want to wrap setvbuf in the library to prevent the application from explicitly buffering its own stdout. And you'll want to wrap fwrite and printf so that they flush on each call. Writing the .so to be preloaded will take you outside of python.
这样做是有可能的，但我能想到的唯一解决方案是相当复杂的、不可移植的，而且可能充满了有问题的细节。您可以使用LD_PRELOAD使外部应用程序加载一个动态库，该动态库包含一个调用setvbuf来解压stdout的构造函数。您可能还希望在库中包装setvbuf，以防止应用程序显式地缓冲其自身的stdout。你需要将fwrite和printf包装起来，这样每次调用都可以刷新。编写.so预加载将使您脱离python。
#3
3
I don't think it's possible. If the source application doesn't flush its outgoing buffer, the data will not reach outside that process until the buffer overflows and a flush is forced.
我认为这是不可能的。如果源应用程序没有刷新它的传出缓冲区，那么在缓冲区溢出并强制刷新之前，数据不会到达进程外部。
Notice how a well-established command such as file has an option (-n) that causes it to flush its output explicitly. This is required when using file in the mode where it reads input file names from a pipe, and prints the detected type. Since in this mode, the file program doesn't quit when done, the output would otherwise not appear.
请注意，像file这样的已完善的命令有一个选项(-n)，使它显式地刷新其输出。在从管道读取输入文件名并打印检测到的类型的模式中使用文件时，这是必需的。因为在这种模式下，文件程序在完成时不会退出，否则输出将不会出现。
Consider this at a lower level: the output buffering simply means that doing write() on a buffered stream copies the data into an in-memory buffer, until the buffer fills up or (typically) until a linefeed is found. Then, the part of the buffer up to the overflow or linefeed is written write()n to the underlying system-level file descriptor (which could be a file, a pipe, a socket, ...).
在较低的层次上考虑这一点:输出缓冲仅仅意味着在缓冲流上执行write()将数据复制到内存中的缓冲区中，直到缓冲区被填满，或者(通常)直到找到linefeed。然后，直到溢出或linefeed的缓冲区部分被写入到底层系统级文件描述符(可以是文件、管道、套接字…)的write()n。
I don't understand how you're going to convince that program to flush its buffer, from the outside.
我不明白你怎么说服那个程序从外部刷新它的缓冲区。
#4
2
Its worth noting that some programs only buffer their output when they think it's not going to a "real user" (ie, a tty). When they detect that their output is being read by another program, they buffer.
值得注意的是，有些程序只在认为输出不是“真正的用户”(即tty)时才缓冲输出。当它们检测到它们的输出正在被另一个程序读取时，它们会进行缓冲。
The emulation of a tty is one of the things that Expect does in automating other processes.
模拟tty是在自动化其他过程中需要做的事情之一。
There is a pure Python implementation of Expect, but I don't know how well it handles tty emulation.
Expect有一个纯粹的Python实现，但是我不知道它如何很好地处理模拟。
#5
2
The answer to this question might help:
这个问题的答案可能会有帮助:
Python Run a daemon sub-process & read stdout
Python运行一个守护进程子进程并读取stdout
Seems to be addressing the same issue.
似乎也在解决同样的问题。
#6
1
This question is a bit old, but I think your problem could now be solved by using subprocess to call stdbuf with the command you want to exec.
这个问题有点老，但是我认为您的问题现在可以通过使用subprocess来调用stdbuf，并使用您想要执行的命令来调用stdbuf。
#7
-1
Try running the Python interpreter with the -u argument:
尝试运行带有-u参数的Python解释器:
python -u myscript.py
This forces Python to use unbuffered stdin/stdout which may help you.
这迫使Python使用未缓冲的stdin/stdout，这可能对您有所帮助。
#1
6
You can use PTYs to solve this by:
你可以用PTYs来解决这个问题:
- Creating a pty master/slave pair;
- 创建一个pty主/从对;
- Connecting the child process's stdin, stdout and stderr to the pty slave device;
- 将子进程的stdin、stdout和stderr连接到pty从设备;
- Reading from and writing to the pty master in the parent.
- 阅读并给家长的家长写信。
#2
5
Doing this is possible, but the only solution I can think of is fairly convoluted, non-portable, and probably fraught with problematic details. You can use LD_PRELOAD to cause the external application to load a dynamic library which contains a constructor that invokes setvbuf to unbuffer stdout. You will probably also want to wrap setvbuf in the library to prevent the application from explicitly buffering its own stdout. And you'll want to wrap fwrite and printf so that they flush on each call. Writing the .so to be preloaded will take you outside of python.
这样做是有可能的，但我能想到的唯一解决方案是相当复杂的、不可移植的，而且可能充满了有问题的细节。您可以使用LD_PRELOAD使外部应用程序加载一个动态库，该动态库包含一个调用setvbuf来解压stdout的构造函数。您可能还希望在库中包装setvbuf，以防止应用程序显式地缓冲其自身的stdout。你需要将fwrite和printf包装起来，这样每次调用都可以刷新。编写.so预加载将使您脱离python。
#3
3
I don't think it's possible. If the source application doesn't flush its outgoing buffer, the data will not reach outside that process until the buffer overflows and a flush is forced.
我认为这是不可能的。如果源应用程序没有刷新它的传出缓冲区，那么在缓冲区溢出并强制刷新之前，数据不会到达进程外部。
Notice how a well-established command such as file has an option (-n) that causes it to flush its output explicitly. This is required when using file in the mode where it reads input file names from a pipe, and prints the detected type. Since in this mode, the file program doesn't quit when done, the output would otherwise not appear.
请注意，像file这样的已完善的命令有一个选项(-n)，使它显式地刷新其输出。在从管道读取输入文件名并打印检测到的类型的模式中使用文件时，这是必需的。因为在这种模式下，文件程序在完成时不会退出，否则输出将不会出现。
Consider this at a lower level: the output buffering simply means that doing write() on a buffered stream copies the data into an in-memory buffer, until the buffer fills up or (typically) until a linefeed is found. Then, the part of the buffer up to the overflow or linefeed is written write()n to the underlying system-level file descriptor (which could be a file, a pipe, a socket, ...).
在较低的层次上考虑这一点:输出缓冲仅仅意味着在缓冲流上执行write()将数据复制到内存中的缓冲区中，直到缓冲区被填满，或者(通常)直到找到linefeed。然后，直到溢出或linefeed的缓冲区部分被写入到底层系统级文件描述符(可以是文件、管道、套接字…)的write()n。
I don't understand how you're going to convince that program to flush its buffer, from the outside.
我不明白你怎么说服那个程序从外部刷新它的缓冲区。
#4
2
Its worth noting that some programs only buffer their output when they think it's not going to a "real user" (ie, a tty). When they detect that their output is being read by another program, they buffer.
值得注意的是，有些程序只在认为输出不是“真正的用户”(即tty)时才缓冲输出。当它们检测到它们的输出正在被另一个程序读取时，它们会进行缓冲。
The emulation of a tty is one of the things that Expect does in automating other processes.
模拟tty是在自动化其他过程中需要做的事情之一。
There is a pure Python implementation of Expect, but I don't know how well it handles tty emulation.
Expect有一个纯粹的Python实现，但是我不知道它如何很好地处理模拟。
#5
2
The answer to this question might help:
这个问题的答案可能会有帮助:
Python Run a daemon sub-process & read stdout
Python运行一个守护进程子进程并读取stdout
Seems to be addressing the same issue.
似乎也在解决同样的问题。
#6
1
This question is a bit old, but I think your problem could now be solved by using subprocess to call stdbuf with the command you want to exec.
这个问题有点老，但是我认为您的问题现在可以通过使用subprocess来调用stdbuf，并使用您想要执行的命令来调用stdbuf。
#7
-1
Try running the Python interpreter with the -u argument:
尝试运行带有-u参数的Python解释器:
python -u myscript.py
This forces Python to use unbuffered stdin/stdout which may help you.
这迫使Python使用未缓冲的stdin/stdout，这可能对您有所帮助。