I'm writing a custom ftp client to act as a gatekeeper for incoming multimedia content from subcontractors hired by one of our partners. I chose twisted because it allows me to parse the file contents before writing the files to disk locally, and I've been looking for occasion to explore twisted anyway. I'm using 'twisted.protocols.ftp.FTPClient.retrieveFile' to get the file, passing the escaped path to the file, and a protocol to the 'retrieveFile' method. I want to be absolutely sure that the entire file has been retrieved because the event handler in the call back is going to write the file to disk locally, then delete the remote file from the ftp server alla '-E' switch behavior in the lftp client. My question is, do I really need to worry about this, or can I assume that an err back will happen if the file is not fully retrieved?
我正在编写一个自定义ftp客户端,作为我们合作伙伴雇用的分包商传入的多媒体内容的守门人。我之所以选择twisted,是因为它允许我在将文件写入磁盘之前解析文件内容,而且我一直在寻找探索扭曲的机会。我使用'twisted.protocols.ftp.FTPClient.retrieveFile'来获取文件,将转义路径传递给文件,并将协议传递给'retrieveFile'方法。我想绝对确定整个文件已被检索,因为回调中的事件处理程序将把文件写入本地磁盘,然后从lftp中的ftp服务器alla'-E'切换行为中删除远程文件客户。我的问题是,我真的需要担心这个问题,还是我可以假设如果没有完全检索到文件会发生错误的回复?
1 个解决方案
#1
4
There are a couple unit tests for behavior in this area.
在这个领域有几个单元测试行为。
twisted.test.test_ftp.FTPClientTestCase.test_failedRETR is the most directly relevant one. It covers the case where the control and data connections are lost while a file transfer is in progress.
twisted.test.test_ftp.FTPClientTestCase.test_failedRETR是最直接相关的。它涵盖了在文件传输过程中丢失控制和数据连接的情况。
It seems to me that test coverage in this area could be significantly improved. There are no tests covering the case where just the data connection is lost while a transfer is in progress, for example. One thing that makes this tricky, though, is that FTP is not a very robust protocol. The end of a file transfer is signaled by the data connection closing. To be safe, you have to check to see if you received as many bytes as you expected to receive. The only way to perform this check is to know the file size in advance or ask the server for it using LIST (FTPClient.list).
在我看来,该领域的测试覆盖率可以得到显着改善。例如,没有测试覆盖仅在传输正在进行时数据连接丢失的情况。然而,有一件事使得这个棘手的是,FTP不是一个非常强大的协议。文件传输的结束由数据连接关闭发出信号。为了安全起见,您必须检查是否收到了预期接收的字节数。执行此检查的唯一方法是事先知道文件大小或使用LIST(FTPClient.list)向服务器询问。
Given all this, I'd suggest that when a file transfer completes, you always ask the server how many bytes you should have gotten and make sure it agrees with the number of bytes delivered to your protocol. You may sometimes get an errback on the Deferred returned from retrieveFile, but this will keep you safe even in the cases where you don't.
考虑到这一切,我建议当文件传输完成时,你总是要求服务器你应该得到多少字节,并确保它与传递给你的协议的字节数一致。有时你可能会在retrieveFile返回的Deferred上得到一个错误的错误,但即使你不这样做,这也会让你安全。
#1
4
There are a couple unit tests for behavior in this area.
在这个领域有几个单元测试行为。
twisted.test.test_ftp.FTPClientTestCase.test_failedRETR is the most directly relevant one. It covers the case where the control and data connections are lost while a file transfer is in progress.
twisted.test.test_ftp.FTPClientTestCase.test_failedRETR是最直接相关的。它涵盖了在文件传输过程中丢失控制和数据连接的情况。
It seems to me that test coverage in this area could be significantly improved. There are no tests covering the case where just the data connection is lost while a transfer is in progress, for example. One thing that makes this tricky, though, is that FTP is not a very robust protocol. The end of a file transfer is signaled by the data connection closing. To be safe, you have to check to see if you received as many bytes as you expected to receive. The only way to perform this check is to know the file size in advance or ask the server for it using LIST (FTPClient.list).
在我看来,该领域的测试覆盖率可以得到显着改善。例如,没有测试覆盖仅在传输正在进行时数据连接丢失的情况。然而,有一件事使得这个棘手的是,FTP不是一个非常强大的协议。文件传输的结束由数据连接关闭发出信号。为了安全起见,您必须检查是否收到了预期接收的字节数。执行此检查的唯一方法是事先知道文件大小或使用LIST(FTPClient.list)向服务器询问。
Given all this, I'd suggest that when a file transfer completes, you always ask the server how many bytes you should have gotten and make sure it agrees with the number of bytes delivered to your protocol. You may sometimes get an errback on the Deferred returned from retrieveFile, but this will keep you safe even in the cases where you don't.
考虑到这一切,我建议当文件传输完成时,你总是要求服务器你应该得到多少字节,并确保它与传递给你的协议的字节数一致。有时你可能会在retrieveFile返回的Deferred上得到一个错误的错误,但即使你不这样做,这也会让你安全。