AWS PHP SDK:在预签名URL中限制S3文件上传大小

I'm working on a project involving generating S3 URLs that someone else can use to upload files to my S3 bucket. Here's a minimal working example:

我正在开发一个项目，其中涉及生成S3 url，其他人可以用它将文件上载到我的S3桶中。这里有一个最小的工作示例:

<?php

    require('aws.phar');
    use Aws\S3\S3Client;

    $s3client = new S3Client(...); // credentials go here

    $id = uniqid(); // generate some kind of key

    $command = $s3client->getCommand('PutObject', [
        'ACL' => 'private',
        'Body' => '',
        'Bucket' => 'mybucket',
        'Key' => 'tmp/' . $id]);

    echo (string) $s3client->createPresignedRequest($command, '+5 minutes')->getURI();

?>

Now, if I put that file at a location accessible by the internet, my web server can be used to fetch new signed upload URLs:

现在，如果我把这个文件放在一个互联网可访问的位置，我的web服务器可以用来获取新的签名上传url:

$ curl http://my.domain.com/some/page.php
https://s3.amazonaws.com/mybucket/tmp/someID?x-amz-acl=private&lots-of-aws-params...
$ curl -X PUT -d "@someFile" https://s3.amazonaws.com/mybucket/tmp/someID?x-amz-acl=private&lots-of-aws-params...
$

This successfully uploads a local file to my bucket, so I can play with it in S3.

这成功地将一个本地文件上载到我的bucket，因此我可以在S3中使用它。

Let's suppose that I'm not too worried about people generating many URLs and uploading many files to my bucket in a short period of time, but I would like to limit the size of uploaded files. Many resources suggest attaching a policy to the signed URL:

假设我不太担心人们在短时间内生成许多url并将许多文件上载到我的bucket中，但是我希望限制上传文件的大小。许多资源建议将策略附加到已签名的URL:

<?php

    require('aws.phar');
    use Aws\S3\S3Client;

    $s3client = new S3Client(...); // credentials go here

    $id = uniqid(); // generate some kind of key

    $policy = [
        'conditions' => [
            ['acl' => 'private'],
            ['bucket' => 'mybucket'],
            ['content-length-range', 0, 8*1024], // 8 KiB
            ['starts-with', '$key', 'tmp/']
        ], 'expiration' =>
            (new DateTime())->modify('+5 minutes')->format(DateTime::ATOM)];

    $command = $s3client->getCommand('PutObject', [
        'ACL' => 'private',
        'Body' => '',
        'Bucket' => 'mybucket',
        'Key' => 'tmp/' . $id,
        'Policy' => $policy]);

    echo (string) $s3client->createPresignedRequest($command, '+5 minutes')->getURI();

?>

This version generates URLS (without any indication of errors) that can be used in the same way. I'm not sure if I need some of those conditions in the policy (acl, bucket, starts-with), but I don't think that including them would break the policy.

这个版本生成的url(没有任何错误提示)可以以相同的方式使用。我不确定在策略中是否需要这些条件(acl、bucket、starts-with)，但我不认为包含这些条件会违反策略。

In theory, attempting to use this signed URL to upload a file larger than 8 KiB should cause S3 to abort the upload. However, testing this with a larger file shows that curl still happily uploads the file:

理论上，尝试使用这个签名URL上传大于8 KiB的文件会导致S3中止上传。但是，用一个较大的文件来测试这个文件显示curl仍然很高兴地上传文件:

$ ls -lh file.txt
-rw-rw-r-- 1 millinon millinon 210K Jan  2 00:41 file.txt
$ curl http://my.domain.com/some/page.php
https://s3.amazonaws.com/mybucket/tmp/someOtherID?x-amz-acl=private&lots-of-aws-params...
$ curl -X PUT -d "@file.txt" https://s3.amazonaws.com/mybucket/tmp/someOtherID?x-amz-acl=private&lots-of-aws-params...
$

Checking the bucket shows that, indeed, the large file was uploaded, and the file's size is larger than the policy supposedly indicates.

检查桶表明，确实是上传了大文件，文件的大小比策略所显示的要大。

Since various pages show different ways of attaching the policy, I have also tried the following versions:

由于不同的页面显示了附加策略的不同方式，我也尝试了以下版本:

'Policy' => json_encode($policy)
'Policy' => base64_encode(json_encode($policy))

However, URLs generated with any of these versions allow files larger than the specified size to be uploaded.

但是，任何这些版本生成的url都允许文件大于指定的大小。

Am I attaching the policy incorrectly, or is there a fundamental limitation to restricting uploads to S3 in this manner?

我是否错误地附加了策略，或者是否存在以这种方式限制上传至S3的基本限制?

For my web server, I'm using HHVM 3.11.1 with version 3.14.1 of the AWS SDK for PHP.

对于我的web服务器，我使用HHVM 3.11.1和版本3.14.1的AWS SDK用于PHP。

2 个解决方案

#1

An S3 upload policy cannot be used with pre-signed URLs.

S3上传策略不能与预签名的url一起使用。

A policy document can be used with browser uploads to S3 using HTML POST Forms.

策略文档可以与使用HTML POST表单的浏览器上传至S3一起使用。

Pre-signed URLs and HTML POST forms are two different methods of uploading to S3. The former is arguably simpler, but less flexible, than the latter.

预签名url和HTML POST表单是向S3上传的两种不同方法。前者可能比后者更简单，但不那么灵活。

UPDATE

更新

If you must upload the files without the use of a browser, the HTML POST Form's request can be reproduced using PHP's curl functions, a library such as Guzzle, or using the command line as follows:

如果您必须在不使用浏览器的情况下上传文件，那么可以使用PHP的curl函数、Guzzle之类的库或以下命令行复制HTML POST表单的请求:

curl 'https://s3-bucket.s3.amazonaws.com/' \
    -F 'key=uploads/${filename}' \
    -F 'AWSAccessKeyId=YOUR_AWS_ACCESS_KEY' \
    -F 'acl=private' \
    -F 'success_action_redirect=http://localhost/' \
    -F 'policy=YOUR_POLICY_DOCUMENT_BASE64_ENCODED' \
    -F 'signature=YOUR_CALCULATED_SIGNATURE' \
    -F 'Content-Type=image/jpeg' \
    -F 'file=@file.jpg'

#2

can you try to add the specific policy to your bucket

您可以尝试将特定的策略添加到您的bucket吗

$s3client->putBucketPolicy(array(
    'Bucket' => $bucket,
    'Policy' => json_encode(array(
        'conditions' => array(
            array(
                'content-length-range', 0, 8*1024,
                'starts-with', '$key', 'tmp/'
                ...

and after this, just use the normal putObject command

然后，使用普通的putObject命令

$command = $s3client->getCommand('PutObject', [
    'ACL' => 'private',
    'Body' => '',
    'Bucket' => 'mybucket',
    'Key' => 'tmp/' . $id]);