我怎样才能列出S3中前缀下的所有对象

There are more than 3k objects under the prefix. I use the following code to list all objects to get their names, but the API only retrieve 1000 objects. s3_client = boto3.client('s3')

前缀下有超过3k个对象。我使用以下代码列出所有对象以获取其名称，但API仅检索1000个对象。 s3_client = boto3.client（'s3'）

response = s3_client.list_objects(
    Bucket = "my-bucket",
    Prefix = "my-prefix",
    MaxKeys=50000
)


s3 = boto3.resource('s3')
bucket = s3.Bucket(S3)

print(len(response['Contents'])) # only retrieve 1000

1 个解决方案

#1

Use paginators to loop through multiple pages. See: Creating Paginators

使用分页器循环遍历多个页面。请参阅：创建Paginators

import boto3

client = boto3.client('s3')
paginator = client.get_paginator('list_objects')
operation_parameters = {'Bucket': 'my-bucket',
                        'Prefix': 'my-prefix'}
page_iterator = paginator.paginate(**operation_parameters)
for page in page_iterator:
    print(page['Contents'])