I'm trying to mirror files on FTP server. Those files can be very large so downloads might be interrupted. I'd like to keep the original files while downloading partial files to a temporary folder and once completed override local older versions.
我正在尝试镜像FTP服务器上的文件。这些文件可能非常大,因此下载可能会中断。我想在将部分文件下载到临时文件夹时保留原始文件,并在完成后覆盖本地旧版本。
- Can I do this? how?
- Is there another easy to use (command line) tool that I can use?
我可以这样做吗?怎么样?
是否有其他易于使用(命令行)的工具,我可以使用?
2 个解决方案
#1
Looking at the wget manual I can't see this functionality, however you could write a bash script to do what you want, which would essentially run an individual wget for each file then move it using normal mv command.
看一下wget手册我看不到这个功能,但是你可以编写一个bash脚本来做你想要的,这本质上会为每个文件运行一个单独的wget,然后使用普通的mv命令移动它。
Alternativly have a look at rsync according to the manual there is a paramater that sets a temp dir
另外根据手册查看rsync有一个设置临时目录的参数
-T --temp-dir=DIR create temporary files in directory DIR
I am not 100% sure wheather this is where it puts the files during downloads as not had chance to try it out.
我不是百分之百确定这是在下载期间放置文件的地方因为没有机会尝试它。
#2
First, download the files to a temp directory. Use -c so you can resume.
首先,将文件下载到临时目录。使用-c这样你就可以恢复了。
After the download has completed, use copy, rename or rsync to copy the files to the final place.
下载完成后,使用copy,rename或rsync将文件复制到最终位置。
Note: Consider using rsync for the whole process because it was designed for just this use case and it will cause much less strain on the server and the Internet. Most site admins are happy if you ask them for an rsync access just for this reason.
注意:考虑在整个过程中使用rsync,因为它仅针对此用例而设计,它将减少服务器和Internet上的压力。如果您因为这个原因要求他们进行rsync访问,大多数站点管理员都很高兴。
#1
Looking at the wget manual I can't see this functionality, however you could write a bash script to do what you want, which would essentially run an individual wget for each file then move it using normal mv command.
看一下wget手册我看不到这个功能,但是你可以编写一个bash脚本来做你想要的,这本质上会为每个文件运行一个单独的wget,然后使用普通的mv命令移动它。
Alternativly have a look at rsync according to the manual there is a paramater that sets a temp dir
另外根据手册查看rsync有一个设置临时目录的参数
-T --temp-dir=DIR create temporary files in directory DIR
I am not 100% sure wheather this is where it puts the files during downloads as not had chance to try it out.
我不是百分之百确定这是在下载期间放置文件的地方因为没有机会尝试它。
#2
First, download the files to a temp directory. Use -c so you can resume.
首先,将文件下载到临时目录。使用-c这样你就可以恢复了。
After the download has completed, use copy, rename or rsync to copy the files to the final place.
下载完成后,使用copy,rename或rsync将文件复制到最终位置。
Note: Consider using rsync for the whole process because it was designed for just this use case and it will cause much less strain on the server and the Internet. Most site admins are happy if you ask them for an rsync access just for this reason.
注意:考虑在整个过程中使用rsync,因为它仅针对此用例而设计,它将减少服务器和Internet上的压力。如果您因为这个原因要求他们进行rsync访问,大多数站点管理员都很高兴。