Background
I'm doing fairly big file uploads on Django. File size is generally 10MB-100MB.
我在Django上做了相当大的文件上传。文件大小一般为10MB-100MB。
I'm on Heroku and I've been hitting the request timeout of 30 seconds.
我在Heroku上，我已经达到30秒的请求超时。
The Beginning
In order to get around the limit, Heroku's recommendation is to upload from the browser DIRECTLY to S3.
为了达到极限，Heroku的建议是从浏览器直接上传到S3。
Amazon documents this by showing you how to write an HTML form to perform the upload.
亚马逊通过向您展示如何编写HTML表单来执行上传来记录此文档。
Since I'm on Django, rather than write the HTML by hand, I'm using django-uploadify-s3 (example). This provides me with an SWF object, wrapped in JS, that performs the actual upload.
由于我在使用Django，而不是手工编写HTML，我使用的是django-uploadify-s3（示例）。这为我提供了一个SWF对象，用JS包装，执行实际的上传。
This part is working fine! Hooray!
这部分工作正常！万岁！
The Problem
The problem is in tying that data back to my Django model in a sane way. Right now the data comes back as a simple URL string, pointing to the file's location.
问题是以理智的方式将数据绑定回我的Django模型。现在，数据以简单的URL字符串形式返回，指向文件的位置。
However, I was previously using S3 Boto from django-storages to manage all of my files as FileFields, backed by the delightful S3BotoStorageFile.
但是，我以前使用django-storages的S3 Boto来管理我的所有文件作为FileFields，由令人愉快的S3BotoStorageFile支持。
To reiterate, S3 Boto is working great in isolation, Uploadify is working great in isolation, the problem is in putting the two together.
重申一下，S3 Boto在隔离方面工作得很好，上传它是孤立地工作，问题在于将两者结合在一起。
My understanding is that the only way to populate the FileField is by providing both the filename AND the file content. When you're uploading files from the browser to Django, this is no problem, as Django has the file content in a buffer and can do whatever it likes with it. However, when doing direct-to-S3 uploads like me, Django only receives the file name and URL, not the binary data, so I can't properly populate the FieldFile.
我的理解是填充FileField的唯一方法是提供文件名和文件内容。当您将文件从浏览器上传到Django时，这没有问题，因为Django将文件内容放在缓冲区中并且可以随意使用它。但是，当像我这样直接进行S3上传时，Django只接收文件名和URL，而不是二进制数据，所以我无法正确填充FieldFile。
Cry For Help
Anyone know a graceful way to use S3Boto's FileField in conjunction with direct-to-S3 uploading?
任何人都知道将S3Boto的FileField与直接到S3上传结合使用的优雅方式？
Else, what's the best way to manage an S3 file just based on its URL? Including setting expiration, key id, etc.
另外，基于URL管理S3文件的最佳方法是什么？包括设置到期，密钥ID等。
Many thanks!
非常感谢！
6 个解决方案
#2
1
I had a similar issue where i want to store file to s3 either directly using FileField or i have an option for the user to input the url directly. So to circumvent that, i used 2 fields in my model, one for FileField and one for URLField. And in the template i could use 'or' to see which one exists and to use that like {{ instance.filefield or instance.url }}.
我有一个类似的问题，我想直接使用FileField将文件存储到s3，或者我有一个选项让用户直接输入url。因此，为了规避这一点，我在模型中使用了2个字段，一个用于FileField，另一个用于URLField。在模板中，我可以使用'或'来查看哪个存在并使用{{instance.filefield或instance.url}}。
#3
1
This is untested, but you should be able to use:
这是未经测试的，但您应该可以使用：
from django.core.files.storage import default_storage
f = default_storage.open('name_you_expect_in_s3', 'r')
#f is an instance of S3BotoStorageFile, and can be assigned to a field
obj, created = YourObject.objects.get_or_create(**stuff_you_know)
obj.s3file_field = f
obj.save()
I think this should set up the local pointer to s3 and save it, without over writing the content.
我认为这应该设置指向s3的本地指针并保存它，而不是过度写入内容。
ETA: You should do this only after the upload completes on S3 and you know the key in s3.
ETA：只有在S3上完成并且您知道s3中的密钥后才能执行此操作。
#4
0
Checkout django-filetransfers. Looks like it plays nice with django-storages.
结帐django-filetransfers。看起来它与django-storage一起玩得很好。
#5
-1
I've never used django, so ymmv :) but why not just write a single byte to populate the content? That way, you can still use FieldFile.
我从来没有使用过django，所以ymmv :)但为什么不写一个字节来填充内容呢？这样，您仍然可以使用FieldFile。
#6
-1
I'm thinking that writing actual SQL may be the easiest solution here. Alternatively you could subclass S3BotoStorage, override the _save method and allow for an optional kwarg of filepath which sidesteps all the other saving stuff and just returns the cleaned_name.
我认为编写实际的SQL可能是最简单的解决方案。或者你可以继承S3BotoStorage，覆盖_save方法并允许一个可选的文件路径kwarg，它可以回避所有其他保存内容并返回clean_name。
#1
#2
1
I had a similar issue where i want to store file to s3 either directly using FileField or i have an option for the user to input the url directly. So to circumvent that, i used 2 fields in my model, one for FileField and one for URLField. And in the template i could use 'or' to see which one exists and to use that like {{ instance.filefield or instance.url }}.
我有一个类似的问题，我想直接使用FileField将文件存储到s3，或者我有一个选项让用户直接输入url。因此，为了规避这一点，我在模型中使用了2个字段，一个用于FileField，另一个用于URLField。在模板中，我可以使用'或'来查看哪个存在并使用{{instance.filefield或instance.url}}。
#3
1
This is untested, but you should be able to use:
这是未经测试的，但您应该可以使用：
from django.core.files.storage import default_storage
f = default_storage.open('name_you_expect_in_s3', 'r')
#f is an instance of S3BotoStorageFile, and can be assigned to a field
obj, created = YourObject.objects.get_or_create(**stuff_you_know)
obj.s3file_field = f
obj.save()
I think this should set up the local pointer to s3 and save it, without over writing the content.
我认为这应该设置指向s3的本地指针并保存它，而不是过度写入内容。
ETA: You should do this only after the upload completes on S3 and you know the key in s3.
ETA：只有在S3上完成并且您知道s3中的密钥后才能执行此操作。
#4
0
Checkout django-filetransfers. Looks like it plays nice with django-storages.
结帐django-filetransfers。看起来它与django-storage一起玩得很好。
#5
-1
I've never used django, so ymmv :) but why not just write a single byte to populate the content? That way, you can still use FieldFile.
我从来没有使用过django，所以ymmv :)但为什么不写一个字节来填充内容呢？这样，您仍然可以使用FieldFile。
#6
-1
I'm thinking that writing actual SQL may be the easiest solution here. Alternatively you could subclass S3BotoStorage, override the _save method and allow for an optional kwarg of filepath which sidesteps all the other saving stuff and just returns the cleaned_name.
我认为编写实际的SQL可能是最简单的解决方案。或者你可以继承S3BotoStorage，覆盖_save方法并允许一个可选的文件路径kwarg，它可以回避所有其他保存内容并返回clean_name。