How do I download all files in a directory and all subdirectories on an HTTP server?
如何下载HTTP服务器上的目录和所有子目录中的所有文件?
Thanks!
2 个解决方案
#1
2
If directory browsing is enabled on the server then you can crawl the directory listings, i.e. Use HttpWebRequest to get the listing page, parse the response to find the file links, download each file (also with HttpWebRequest), navigate to each subfolder, rinse and repeat.
如果在服务器上启用了目录浏览,那么您可以抓取目录列表,即使用HttpWebRequest获取列表页面,解析响应以查找文件链接,下载每个文件(也使用HttpWebRequest),导航到每个子文件夹,冲洗和重复。
If directory browsing isn't enabled then you can't really download ALL files in ALL subdirectories because you can't know they exist.
如果未启用目录浏览,则无法真正下载所有子目录中的所有文件,因为您无法知道它们是否存在。
However, you could still use HttpWebRequest to crawl the exposed web pages and download any linked files that are of interest.
但是,您仍然可以使用HttpWebRequest来抓取公开的网页并下载任何感兴趣的链接文件。
#2
5
By using a command-line tool like wget rather than reinventing the wheel.
通过使用像wget这样的命令行工具而不是重新发明*。
#1
2
If directory browsing is enabled on the server then you can crawl the directory listings, i.e. Use HttpWebRequest to get the listing page, parse the response to find the file links, download each file (also with HttpWebRequest), navigate to each subfolder, rinse and repeat.
如果在服务器上启用了目录浏览,那么您可以抓取目录列表,即使用HttpWebRequest获取列表页面,解析响应以查找文件链接,下载每个文件(也使用HttpWebRequest),导航到每个子文件夹,冲洗和重复。
If directory browsing isn't enabled then you can't really download ALL files in ALL subdirectories because you can't know they exist.
如果未启用目录浏览,则无法真正下载所有子目录中的所有文件,因为您无法知道它们是否存在。
However, you could still use HttpWebRequest to crawl the exposed web pages and download any linked files that are of interest.
但是,您仍然可以使用HttpWebRequest来抓取公开的网页并下载任何感兴趣的链接文件。
#2
5
By using a command-line tool like wget rather than reinventing the wheel.
通过使用像wget这样的命令行工具而不是重新发明*。