Python-FTP下载目录中的所有文件

Python newb here so please bear with me. I m putting together a script to download all the files from a directory via FTP. So far I have managed to connect and fetch one file, but I cannot seem to make to work in batch (get all the files from the directory) Here is what I have so far:

Python newb在这里请耐心等待。我正在整理一个脚本,通过FTP下载目录中的所有文件。到目前为止,我已经设法连接并获取一个文件,但我似乎无法批量工作(从目录中获取所有文件)这是我到目前为止:

from ftplib import FTP
import os, sys, os.path

def handleDownload(block):
    file.write(block)
    print ".",

ddir='C:\\Data\\test\\'
os.chdir(ddir)
ftp = FTP('test1/server/')

print 'Logging in.'
ftp.login('user1\\anon', 'pswrd20')
directory = '\\data\\test\\'

print 'Changing to ' + directory
ftp.cwd(directory)
ftp.retrlines('LIST')

print 'Accessing files'

for subdir, dirs, files in os.walk(directory):
    for file in files: 
        full_fname = os.path.join(root, fname);  
        print 'Opening local file ' 
        ftp.retrbinary('RETR C:\\Data\\test\\' + fname,
                       handleDownload,
                       open(full_fname, 'wb'));
        print 'Closing file ' + filename
        file.close();
ftp.close()

I bet you can tell that it does not do much when I run it, so any suggestions for improvements would be greatly appreciated

我打赌你可以告诉我,当我运行它时它没有做太多,所以任何改进建议都将非常感激

PS This is not homework :DD

PS这不是作业:DD

Thanks!!

5 个解决方案

#1

I've managed to crack this, so now posting the relevant bit of code for future visitors:

我已经设法破解了这个,所以现在为未来的访问者发布相关的代码:

filenames = ftp.nlst() # get filenames within the directory
print filenames

for filename in filenames:
    local_filename = os.path.join('C:\\test\\', filename)
    file = open(local_filename, 'wb')
    ftp.retrbinary('RETR '+ filename, file.write)

    file.close()

ftp.quit() # This is the “polite” way to close a connection

This worked for me on Python 2.5, Windows XP.

这适用于Python 2.5,Windows XP。

#2

If this is just a problem you'd like to solve, I might suggest the wget command:

如果这只是您想要解决的问题,我可能会建议使用wget命令:

cd c:\destination
wget --mirror --continue --no-host-directories --user=username --password=s3cr3t ftp://hostname/source/path/

The --continue option could be very dangerous if files change on the server. If files are only ever added, then it is very friendly.

如果服务器上的文件发生更改,则--continue选项可能非常危险。如果只添加文件,那么它非常友好。

However, if this is a learning exercise for you and you'd like to make your program work, I think you should start by looking at this line:

但是,如果这是一个学习练习,你想让你的程序工作,我想你应该从这一行开始:

for subdir, dirs, files in os.walk(directory):

directory has been the remote source directory in most of your program, but the os.walk() function cannot walk a remote directory. You need to iterate over the returned files yourself, using a callback supplied to the retrlines function.

目录一直是你的大多数程序中的远程源目录,但是os.walk()函数不能遍历远程目录。您需要使用提供给retrlines函数的回调自己迭代返回的文件。

Take a look at the MLSD or NLST options instead of LIST, they will probably be easier to parse. (Note that FTP doesn't actually specify how lists should look; it was always intended to be driven by a human at a console, or a specific filename transferred. So programs that do clever things with FTP listings like present them to the user in a GUI probably have to have huge piles of special case code, for odd or obscure servers. And they probably all do something stupid when faced with malicious file names.)

看看MLSD或NLST选项而不是LIST,它们可能更容易解析。 (请注意,FTP实际上并没有指定列表的外观;它总是打算由控制台上的人驱动,或者传输特定的文件名。所以使用FTP列表做聪明事情的程序会将它们呈现给用户对于奇怪或模糊的服务器,GUI可能必须拥有大量的特殊情况代码。当面对恶意文件名时,他们可能都会做一些愚蠢的事情。)

Can you use sftp instead? sftp does have a specification for how file listings are supposed to be parsed, doesn't transmit username/password in the clear, and doesn't have the giant annoyance of passive vs active connections -- it simply uses the single connection, which means it works across more firewalls than FTP does.

你能用sftp吗? sftp确实有一个关于如何解析文件列表的规范,不清楚地传输用户名/密码,并且没有被动与活动连接的巨大烦恼 - 它只是使用单一连接,这意味着它比FTP更适用于防火墙。

Edit: You need to pass a 'callable' object to the retrlines function. A callable object is either an instance of a class that defined a __call__ method, or a function. While the function might be easier to describe, an instance of a class may be more useful. (You could use the instance to collect the filenames, but the function would have to write to a global variable. Bad.)

编辑:您需要将'callable'对象传递给retrlines函数。可调用对象是定义__call__方法的类的实例,或者是函数。虽然函数可能更容易描述,但类的实例可能更有用。 (您可以使用该实例来收集文件名,但该函数必须写入全局变量。不好。)

Here's one of the simplest callable object:

这是最简单的可调用对象之一:

>>> class c:
...  def __call__(self, *args):
...   print(args)
...
>>> f = c()
>>> f('hello')
('hello',)
>>> f('hello', 'world')
('hello', 'world')

This creates a new class, c, that defines an instance method __call__. This just prints its arguments in a fairly stupid manner, but it shows how minimal we're talking. :)

这将创建一个新类c,它定义一个实例方法__call__。这只是以相当愚蠢的方式打印它的参数,但它显示了我们谈论的最小化。 :)

If you wanted something smarter, it could do something like this:

如果你想要更聪明的东西,它可以做这样的事情:

class handle_lines:
  def __init__(self):
    self.lines = []
  def __call__(self, *args):
    self.lines << args[0]

Call iterlines with an object of this class, then look in the object's lines member for details.

使用此类的对象调用iterlines,然后查看对象的lines成员以获取详细信息。

#3

this code is a little bit of overkill I think.

我认为这段代码有点过分。

(from the python example https://docs.python.org/2/library/ftplib.html) After ftp.login() and setting ftp.cwd() you can just use:

(来自python示例https://docs.python.org/2/library/ftplib.html)在ftp.login()和设置ftp.cwd()之后你可以使用:

os.chdir(ddir)
ls = ftp.nlst()
count = len(ls)
curr = 0
print "found {} files".format(count)
for fn in ls:
    curr += 1
    print 'Processing file {} ... {} of {} ...'.format(fn, curr, count)
    ftp.retrbinary('RETR ' + fn, open(fn, 'wb').write)

ftp.quit()
print "download complete."

to download all the files.

下载所有文件。

#4

I am a beginner so I have not made the code efficiently but I made it and tested it is working. This is what I did to download files and folders from ftp site but only limited depth in file structure.

我是初学者,所以我没有有效地编写代码,但我做了它并测试它是否正常工作。这是我从ftp站点下载文件和文件夹所做的,但文件结构只有有限的深度。

try:
   a = input("Enter hostname : ")
   b = input("Enter username : ")
   c = input("Enter password : ")
   from ftplib import FTP
   import os
   os.makedirs("C:\\Users\\PREM\\Desktop\\pyftp download\\ftp")
   os.chdir("C:\\Users\\PREM\\Desktop\\pyftp download\\ftp")
   ftp = FTP(host = a, user= b, passwd = c)
   D = ftp.nlst()
   for d in D:
      l = len(d)
      char = False
      for i in range(0,l):
          char = char or d[i]=="."
      if not char:
         ftp.cwd("..")
         ftp.cwd("..")
         E = ftp.nlst("%s"%(d))
         ftp.cwd("%s"%(d))
         try:
             os.makedirs("C:\\Users\\PREM\\Desktop\\pyftp download\\ftp\\%s"%(d))
         except:
             print("you can debug if you try some more")
         finally:
             os.chdir("C:\\Users\\PREM\\Desktop\\pyftp download\\ftp\\%s"%(d))
             for e in E:
                l1 = len(e)
                char1 = False
                for i in range(0,l1):
                   char1 = char1 or e[i]=="."
                if not char1:
                   ftp.cwd("..")
                   ftp.cwd("..")
                   F = ftp.nlst("%s/%s"%(d,e))
                   ftp.cwd("%s/%s"%(d,e))
                   try:
                       os.makedirs("C:\\Users\\PREM\\Desktop\\pyftp download\\ftp\\%s\\%s"%(d,e))
                   except:
                       print("you can debug if you try some more")
                   finally:
                       os.chdir("C:\\Users\\PREM\\Desktop\\pyftp download\\ftp\\%s\\%s"%(d,e))
                       for f in F:
                           if "." in f[2:]:
                               with open(f,'wb') as filef:
                                   ftp.retrbinary('RETR %s' %(f), filef.write)
                           elif not "." in f:
                               try:
                                  os.makedirs("C:\\Users\\PREM\\Desktop\\pyftp download\\ftp\\%s\\%s\\%s"%(d,e,f))
                               except:
                                  print("you can debug if you try some more")
                elif "." in e[2:]:
                   os.chdir("C:\\Users\\PREM\\Desktop\\pyftp download\\ftp\\%s"%(d))
                   ftp.cwd("..")
                   ftp.cwd("..")
                   ftp.cwd("..")
                   ftp.cwd("%s"%(d))
                   with open(e,'wb') as filee:
                      ftp.retrbinary('RETR %s' %(e), filee.write)
      elif "." in d[2:]:
          ftp.cwd("..")
          ftp.cwd("..")
          os.chdir("C:\\Users\\PREM\\Desktop\\pyftp download\\ftp")
          with open(d,'wb') as filed:
             ftp.retrbinary('RETR %s'%(d), filed.write)
   ftp.close()
   print("Your files has been successfully downloaded and saved. Bye")
except:
    print("try again you can do it")
finally:
    print("code ran")

#5

Instead of using Python lib to ftp download a directory, we can call a dos script from python program. In the dos script we would use the native ftp protocol which can download all file from the folder using mget *.*.

我们可以从python程序调用dos脚本,而不是使用Python lib来ftp下载目录。在dos脚本中,我们将使用本机ftp协议,该协议可以使用mget *。*从文件夹下载所有文件。

fetch.bat
ftp -s:fetch.txt

fetch.txt
open <ipaddress>
<userid>
<password>
bin (set the mnode to binary)
cd </desired directory>
mget *.*
bye

fetch.py
import os
os.system("fetch.bat")