I need to list all files contained in a certain folder contained in my S3 bucket.
我需要列出我的S3存储桶中包含的某个文件夹中包含的所有文件。
The folder structure is the following
文件夹结构如下
/my-bucket/users/<user-id>/contacts/<contact-id>
I have files related to users and files related to a certain user's contact. I need to list both.
我有与用户相关的文件和与某个用户的联系人相关的文件。我需要列出两者。
To list files I'm using this code:
要列出我正在使用此代码的文件：
ListObjectsRequest listObjectsRequest = new ListObjectsRequest().withBucketName("my-bucket")
.withPrefix("some-prefix").withDelimiter("/");
ObjectListing objects = transferManager.getAmazonS3Client().listObjects(listObjectsRequest);
To list a certain user's files I'm using this prefix:
要列出某个用户的文件我正在使用此前缀：
users/<user-id>/
用户/ <用户id> /
and I'm correctly getting all files in the directory excluding contacts subdirectory, for example:
我正确地获取目录中除contacts子目录之外的所有文件，例如：
users/<user-id>/file1.txt
users/<user-id>/file2.txt
users/<user-id>/file3.txt
To list a certain user contact's files instead I'm using this prefix:
要列出某个用户联系人的文件而不是我使用此前缀：
users/<user-id>/contacts/<contact-id>/
用户/ <用户id> /联系人/ <接触-id> /
but in this case I'm getting also the directory itself as a returned object:
但在这种情况下，我也将目录本身作为返回的对象：
users/<user-id>/contacts/<contact-id>/file1.txt
users/<user-id>/contacts/<contact-id>/file2.txt
users/<user-id>/contacts/<contact-id>/
Why am I getting this behaviour? What's different beetween the two listing requests? I need to list only files in the directory, excluding sub-directories.
为什么我会出现这种行为？这两个上市要求之间有什么不同？我只需要列出目录中的文件，不包括子目录。
4 个解决方案
#1
12
Everthing in S3 is an object. To you, it may be files and folders. But to S3, they're just objects.
S3中的Everthing是一个对象。对你来说，它可能是文件和文件夹。但对于S3来说，它们只是对象。
Objects that end with the delimiter (/ in most cases) are usually perceived as a folder, but it's not always the case. It depends on the application. Again, in your case, you're interpretting it as a folder. S3 is not. It's just another object.
以分隔符结尾的对象（在大多数情况下）通常被视为文件夹，但情况并非总是如此。这取决于应用程序。同样，在您的情况下，您将其解释为文件夹。 S3不是。这只是另一个对象。
In your case above, the object users/<user-id>/contacts/<contact-id>/ exists in S3 as a distinct object, but the object users/<user-id>/ does not. That's the difference in your responses. Why they're like that, we cannot tell you, but someone made the object in one case, and didn't in the other. You don't see it in the AWS Management Console because the console is interpreting it as a folder and hiding it from you.
在上面的例子中，对象users /
Since S3 just sees these things as objects, it won't "exclude" certain things for you. It's up to the client to deal with the objects as they should be dealt with.
由于S3只是将这些东西视为对象，因此它不会为您“排除”某些东西。由客户来处理应该处理的对象。
Your Solution
你的解决方案
Since you're the one that doesn't want the folder objects, you can exclude it yourself by checking the last character for a /. If it is, then ignore the object from the response.
由于您是不想要文件夹对象的人，因此可以通过检查/的最后一个字符来自行排除它。如果是，则忽略响应中的对象。
#2
15
While everybody say that there are no directories and files in s3, but only objects (and buckets), which is absolutely true, I would suggest to take advantage of CommonPrefixes, described in this answer. So, you can do following to get list of "folders" (commonPrefixes) and "files" (objectSummaries):
虽然每个人都说s3中没有目录和文件，但只有对象（和存储桶），这是绝对正确的，我建议利用此答案中描述的CommonPrefixes。因此，您可以执行以下操作以获取“文件夹”（commonPrefixes）和“文件”（objectSummaries）的列表：
ListObjectsV2Request req = new ListObjectsV2Request().withBucketName(bucket.getName()).withPrefix(prefix).withDelimiter(DELIMITER);
ListObjectsV2Result listing = s3Client.listObjectsV2(req);
for (String commonPrefix : listing.getCommonPrefixes()) {
System.out.println(commonPrefix);
}
for (S3ObjectSummary summary: listing.getObjectSummaries()) {
System.out.println(summary.getKey());
}
In your case, for objectSummaries (files) it should return (in case of correct prefix):
users/user-id/contacts/contact-id/file1.txt
users/user-id/contacts/contact-id/file2.txt
for commonPrefixes:
users/user-id/contacts/contact-id/
在你的情况下，对于objectSummaries（文件）它应该返回（如果前缀正确）：users / user-id / contacts / contact-id / file1.txt users / user-id / contacts / contact-id / file2.txt for commonPrefixes：users / user-id / contacts / contact-id /
#3
0
S3 does not have directories, while you can list files in a pseudo directory manner like you demonstrated, there is no directory "file" per-se.
You may of inadvertently created a data file called users/<user-id>/contacts/<contact-id>/.
S3没有目录，虽然你可以像你演示的那样以伪目录方式列出文件，但本身没有目录“文件”。您可能无意中创建了名为users /
#4
0
you can check the type. s3 has a special application/x-directory
你可以查看类型。 s3有一个特殊的应用程序/ x目录
bucket.objects({:delimiter=>"/", :prefix=>"f1/"}).each { |obj| p obj.object.content_type }
#1
12
Everthing in S3 is an object. To you, it may be files and folders. But to S3, they're just objects.
S3中的Everthing是一个对象。对你来说，它可能是文件和文件夹。但对于S3来说，它们只是对象。
Objects that end with the delimiter (/ in most cases) are usually perceived as a folder, but it's not always the case. It depends on the application. Again, in your case, you're interpretting it as a folder. S3 is not. It's just another object.
以分隔符结尾的对象（在大多数情况下）通常被视为文件夹，但情况并非总是如此。这取决于应用程序。同样，在您的情况下，您将其解释为文件夹。 S3不是。这只是另一个对象。
In your case above, the object users/<user-id>/contacts/<contact-id>/ exists in S3 as a distinct object, but the object users/<user-id>/ does not. That's the difference in your responses. Why they're like that, we cannot tell you, but someone made the object in one case, and didn't in the other. You don't see it in the AWS Management Console because the console is interpreting it as a folder and hiding it from you.
在上面的例子中，对象users /
Since S3 just sees these things as objects, it won't "exclude" certain things for you. It's up to the client to deal with the objects as they should be dealt with.
由于S3只是将这些东西视为对象，因此它不会为您“排除”某些东西。由客户来处理应该处理的对象。
Your Solution
你的解决方案
Since you're the one that doesn't want the folder objects, you can exclude it yourself by checking the last character for a /. If it is, then ignore the object from the response.
由于您是不想要文件夹对象的人，因此可以通过检查/的最后一个字符来自行排除它。如果是，则忽略响应中的对象。
#2
15
While everybody say that there are no directories and files in s3, but only objects (and buckets), which is absolutely true, I would suggest to take advantage of CommonPrefixes, described in this answer. So, you can do following to get list of "folders" (commonPrefixes) and "files" (objectSummaries):
虽然每个人都说s3中没有目录和文件，但只有对象（和存储桶），这是绝对正确的，我建议利用此答案中描述的CommonPrefixes。因此，您可以执行以下操作以获取“文件夹”（commonPrefixes）和“文件”（objectSummaries）的列表：
ListObjectsV2Request req = new ListObjectsV2Request().withBucketName(bucket.getName()).withPrefix(prefix).withDelimiter(DELIMITER);
ListObjectsV2Result listing = s3Client.listObjectsV2(req);
for (String commonPrefix : listing.getCommonPrefixes()) {
System.out.println(commonPrefix);
}
for (S3ObjectSummary summary: listing.getObjectSummaries()) {
System.out.println(summary.getKey());
}
In your case, for objectSummaries (files) it should return (in case of correct prefix):
users/user-id/contacts/contact-id/file1.txt
users/user-id/contacts/contact-id/file2.txt
for commonPrefixes:
users/user-id/contacts/contact-id/
在你的情况下，对于objectSummaries（文件）它应该返回（如果前缀正确）：users / user-id / contacts / contact-id / file1.txt users / user-id / contacts / contact-id / file2.txt for commonPrefixes：users / user-id / contacts / contact-id /
#3
0
S3 does not have directories, while you can list files in a pseudo directory manner like you demonstrated, there is no directory "file" per-se.
You may of inadvertently created a data file called users/<user-id>/contacts/<contact-id>/.
S3没有目录，虽然你可以像你演示的那样以伪目录方式列出文件，但本身没有目录“文件”。您可能无意中创建了名为users /
#4
0
you can check the type. s3 has a special application/x-directory
你可以查看类型。 s3有一个特殊的应用程序/ x目录
bucket.objects({:delimiter=>"/", :prefix=>"f1/"}).each { |obj| p obj.object.content_type }