在Amazon s3中下载批量对象

时间:2022-08-11 17:15:26

In Amazon S3, I have created 1 bucket under that bucket multiple subfolders like <bucket_name>/<year>/<month>/<day>/files (i.e.objects).

在Amazon S3中,我在该存储桶下创建了1个存储桶,其中包含多个子文件夹,如 / / / / files(ieobjects)。

I want functionality where on request I can download bulk objects by year / month / day, all files in zip.

我希望功能可以根据要求我按年/月/日下载批量对象,所有文件都是zip。

Is there any way I can do this by Amazon Java SDK?

我有什么办法可以通过Amazon Java SDK实现这一目标吗?

2 个解决方案

#1


1  

There is a MultipleFileDownload method provided with the Transfer Manager library that provides Multiple file download of an entire virtual directory. The contents, however, is not zipped.

Transfer Manager库提供了一个MultipleFileDownload方法,该方法提供整个虚拟目录的多个文件下载。但是,内容不是压缩的。

See: MultipleFileDownload javadoc

请参阅:MultipleFileDownload javadoc

Since your objects are in directories by year/month/day, you could use this method to download all files in a specific path. However, the files would not be zipped.

由于您的对象按年/月/日显示在目录中,因此您可以使用此方法下载特定路径中的所有文件。但是,文件不会被压缩。

#2


0  

You can achieve it using mc tool: mc cp -r https://bucketName.s3.amazonaws.com/<year>/<month>/<day> backupdir it copies from S3 to the local directory backup

您可以使用mc工具实现它:mc cp -r https://bucketName.s3.amazonaws.com/ / / backupdir它从S3复制到本地目录备份

and then you can zip the backup directory by: zip backupdir.zip -r backupdir

然后你可以通过以下方式压缩备份目录:zip backupdir.zip -r backupdir

#1


1  

There is a MultipleFileDownload method provided with the Transfer Manager library that provides Multiple file download of an entire virtual directory. The contents, however, is not zipped.

Transfer Manager库提供了一个MultipleFileDownload方法,该方法提供整个虚拟目录的多个文件下载。但是,内容不是压缩的。

See: MultipleFileDownload javadoc

请参阅:MultipleFileDownload javadoc

Since your objects are in directories by year/month/day, you could use this method to download all files in a specific path. However, the files would not be zipped.

由于您的对象按年/月/日显示在目录中,因此您可以使用此方法下载特定路径中的所有文件。但是,文件不会被压缩。

#2


0  

You can achieve it using mc tool: mc cp -r https://bucketName.s3.amazonaws.com/<year>/<month>/<day> backupdir it copies from S3 to the local directory backup

您可以使用mc工具实现它:mc cp -r https://bucketName.s3.amazonaws.com/ / / backupdir它从S3复制到本地目录备份

and then you can zip the backup directory by: zip backupdir.zip -r backupdir

然后你可以通过以下方式压缩备份目录:zip backupdir.zip -r backupdir