I'm configuring a Django project that were using the server filesystem for storing the apps static files (STATIC_ROOT
) and user uploaded files (MEDIA_ROOT
).
我正在配置一个Django项目,该项目使用服务器文件系统存储应用程序静态文件(STATIC_ROOT)和用户上传文件(MEDIA_ROOT)。
I need now to host all that content on Amazon's S3, so I have created a bucket for this. Using django-storages
with the boto
storage backend, I managed to upload collected statics to the S3 bucket:
我现在需要在Amazon的S3上托管所有这些内容,因此我为此创建了一个桶。使用django存储和boto存储后端,我成功地将收集到的静态数据上载到S3桶中:
MEDIA_ROOT = '/media/'
STATIC_ROOT = '/static/'
DEFAULT_FILE_STORAGE = 'storages.backends.s3boto.S3BotoStorage'
AWS_ACCESS_KEY_ID = 'KEY_ID...'
AWS_SECRET_ACCESS_KEY = 'ACCESS_KEY...'
AWS_STORAGE_BUCKET_NAME = 'bucket-name'
STATICFILES_STORAGE = 'storages.backends.s3boto.S3BotoStorage'
Then, I got a problem: the MEDIA_ROOT
and STATIC_ROOT
are not used within the bucket, so the bucket root contains both the static files and user uploaded paths.
然后,我遇到了一个问题:在bucket中不使用MEDIA_ROOT和STATIC_ROOT,所以bucket根包含静态文件和用户上传的路径。
So then I could set:
然后我可以设置
S3_URL = 'http://s3.amazonaws.com/%s' % AWS_STORAGE_BUCKET_NAME
STATIC_URL = S3_URL + STATIC_ROOT
MEDIA_URL = 'S3_URL + MEDIA_ROOT
And use those settings in the templates, but there is no distinction of static/media files when storing in S3 with django-storages
.
并且在模板中使用这些设置,但是在使用django存储存储在S3时,没有任何静态/媒体文件的区别。
How this can be done?
这是怎么做到的呢?
Thanks!
谢谢!
4 个解决方案
#1
118
I think the following should work, and be simpler than Mandx's method, although it's very similar:
我认为下面的方法应该是可行的,并且比Mandx的方法更简单,尽管它非常相似:
Create a s3utils.py
file:
创建一个s3utils。py文件:
from storages.backends.s3boto import S3BotoStorage
StaticRootS3BotoStorage = lambda: S3BotoStorage(location='static')
MediaRootS3BotoStorage = lambda: S3BotoStorage(location='media')
Then in your settings.py
:
然后在你的settings.py:
DEFAULT_FILE_STORAGE = 'myproject.s3utils.MediaRootS3BotoStorage'
STATICFILES_STORAGE = 'myproject.s3utils.StaticRootS3BotoStorage'
A different but related example (that I've actually tested) can be seen in the two example_
files here.
在这里的两个示例文件中可以看到一个不同但相关的示例(我实际上测试过)。
#2
7
I'm currently using this code in a separated s3utils
module:
我目前在一个单独的s3utils模块中使用这个代码:
from django.core.exceptions import SuspiciousOperation
from django.utils.encoding import force_unicode
from storages.backends.s3boto import S3BotoStorage
def safe_join(base, *paths):
"""
A version of django.utils._os.safe_join for S3 paths.
Joins one or more path components to the base path component intelligently.
Returns a normalized version of the final path.
The final path must be located inside of the base path component (otherwise
a ValueError is raised).
Paths outside the base path indicate a possible security sensitive operation.
"""
from urlparse import urljoin
base_path = force_unicode(base)
paths = map(lambda p: force_unicode(p), paths)
final_path = urljoin(base_path + ("/" if not base_path.endswith("/") else ""), *paths)
# Ensure final_path starts with base_path and that the next character after
# the final path is '/' (or nothing, in which case final_path must be
# equal to base_path).
base_path_len = len(base_path) - 1
if not final_path.startswith(base_path) \
or final_path[base_path_len:base_path_len + 1] not in ('', '/'):
raise ValueError('the joined path is located outside of the base path'
' component')
return final_path
class StaticRootS3BotoStorage(S3BotoStorage):
def __init__(self, *args, **kwargs):
super(StaticRootS3BotoStorage, self).__init__(*args, **kwargs)
self.location = kwargs.get('location', '')
self.location = 'static/' + self.location.lstrip('/')
def _normalize_name(self, name):
try:
return safe_join(self.location, name).lstrip('/')
except ValueError:
raise SuspiciousOperation("Attempted access to '%s' denied." % name)
class MediaRootS3BotoStorage(S3BotoStorage):
def __init__(self, *args, **kwargs):
super(MediaRootS3BotoStorage, self).__init__(*args, **kwargs)
self.location = kwargs.get('location', '')
self.location = 'media/' + self.location.lstrip('/')
def _normalize_name(self, name):
try:
return safe_join(self.location, name).lstrip('/')
except ValueError:
raise SuspiciousOperation("Attempted access to '%s' denied." % name)
Then, in my settings module:
然后,在我的设置模块:
DEFAULT_FILE_STORAGE = 'myproyect.s3utils.MediaRootS3BotoStorage'
STATICFILES_STORAGE = 'myproyect.s3utils.StaticRootS3BotoStorage'
I got to redefine the _normalize_name()
private method to use a "fixed" version of the safe_join()
function, since the original code is giving me SuspiciousOperation
exceptions for legal paths.
我重新定义了_normalize_name()私有方法,以使用safe_join()函数的“固定”版本,因为原始代码为合法路径提供了可疑的操作异常。
I'm posting this for consideration, if anyone can give a better answer or improve this one, it will be very welcome.
我发这篇文章是为了考虑,如果有人能给出更好的答案或改进这篇文章,我将非常欢迎。
#3
5
File: PROJECT_NAME/custom_storages.py
文件:PROJECT_NAME / custom_storages.py
from django.conf import settings
from storages.backends.s3boto import S3BotoStorage
class StaticStorage(S3BotoStorage):
location = settings.STATICFILES_LOCATION
class MediaStorage(S3BotoStorage):
location = settings.MEDIAFILES_LOCATION
File: PROJECT_NAME/settings.py
文件:PROJECT_NAME / settings.py
STATICFILES_LOCATION = 'static'
MEDIAFILES_LOCATION = 'media'
if not DEBUG:
STATICFILES_STORAGE = 'PROJECT_NAME.custom_storages.StaticStorage'
DEFAULT_FILE_STORAGE = 'PROJECT_NAME.custom_storages.MediaStorage'
AWS_ACCESS_KEY_ID = 'KEY_XXXXXXX'
AWS_SECRET_ACCESS_KEY = 'SECRET_XXXXXXXXX'
AWS_STORAGE_BUCKET_NAME = 'BUCKET_NAME'
AWS_HEADERS = {'Cache-Control': 'max-age=86400',}
AWS_QUERYSTRING_AUTH = False
And run: python manage.py collectstatic
和运行:python管理。py collectstatic
#4
2
I think the answer is pretty simple and done by default. This is working for me on AWS Elastic Beanstalk with Django 1.6.5 and Boto 2.28.0:
我认为答案很简单,而且是默认的。这是为我做的AWS弹性豆茎,Django 1.6.5和Boto 2.28.0:
STATICFILES_FINDERS = (
'django.contrib.staticfiles.finders.FileSystemFinder',
'django.contrib.staticfiles.finders.AppDirectoriesFinder',
)
TEMPLATE_LOADERS = (
'django.template.loaders.filesystem.Loader',
'django.template.loaders.app_directories.Loader',
)
DEFAULT_FILE_STORAGE = 'storages.backends.s3boto.S3BotoStorage'
STATICFILES_STORAGE = 'storages.backends.s3boto.S3BotoStorage'
AWS_ACCESS_KEY_ID = os.environ['AWS_ACCESS_KEY_ID']
AWS_SECRET_ACCESS_KEY = os.environ['AWS_SECRET_KEY']
The AWS keys are passed in from the container config file and I have no STATIC_ROOT
or STATIC_URL
set at all. Also, no need for the s3utils.py
file. These details are handled by the storage system automatically. The trick here is that I needed to reference this unknown path in my templates correctly and dynamically. For example:
AWS键是从容器配置文件传入的,我根本没有设置STATIC_ROOT或STATIC_URL。同样,不需要s3utils。py文件。这些细节是由存储系统自动处理的。这里的诀窍是,我需要在模板中正确和动态地引用这个未知路径。例如:
<link rel="icon" href="{% static "img/favicon.ico" %}">
That is how I address my favicon which lives locally (pre-deployment) in ~/Projects/my_app/project/my_app/static/img/favicon.ico
.
这就是我如何在~/Projects/my_app/project/my_app/ my_app/static/img/favicon.ico中使用的favicon。
Of course I have a separate local_settings.py
file for accessing this stuff locally in dev environment and it does have STATIC and MEDIA settings. I had to do a lot of experimenting and reading to find this solution and it works consistently with no errors.
当然,我有一个单独的local_settings。在dev环境中访问这些东西的py文件,它有静态和媒体设置。我必须做大量的实验和阅读来找到这个解决方案,它始终没有错误。
I understand that you need the static and root separation and considering that you can only provide one bucket I would point out that this method takes all the folders in my local environment under ~/Projects/my_app/project/my_app/static/
and creates a folder in the bucket root (ie: S3bucket/img/ as in the example above). So you do get separation of files. For example you could have a media
folder in the static
folder and access it via templating with this:
我理解您需要静态和根分离,并且考虑到您只能提供一个bucket,我想指出的是,这个方法取我本地环境下~/Projects/my_app/project/my_app/static/,并在bucket根中创建一个文件夹(例如:S3bucket/img/上面的示例)。这样就得到了文件的分离。例如,您可以在静态文件夹中有一个媒体文件夹,并通过以下模板访问:
{% static "media/" %}
I hope this helps. I came here looking for the answer and pushed a bit harder to find a simpler solution than to extend the storage system. Instead, I read the documentation about the intended use of Boto and I found that a lot of what I needed was built-in by default. Cheers!
我希望这可以帮助。我来这里是为了寻找答案,并努力寻找比扩展存储系统更简单的解决方案。相反,我阅读了关于Boto预期用途的文档,发现我需要的很多东西都是默认内置的。干杯!
#1
118
I think the following should work, and be simpler than Mandx's method, although it's very similar:
我认为下面的方法应该是可行的,并且比Mandx的方法更简单,尽管它非常相似:
Create a s3utils.py
file:
创建一个s3utils。py文件:
from storages.backends.s3boto import S3BotoStorage
StaticRootS3BotoStorage = lambda: S3BotoStorage(location='static')
MediaRootS3BotoStorage = lambda: S3BotoStorage(location='media')
Then in your settings.py
:
然后在你的settings.py:
DEFAULT_FILE_STORAGE = 'myproject.s3utils.MediaRootS3BotoStorage'
STATICFILES_STORAGE = 'myproject.s3utils.StaticRootS3BotoStorage'
A different but related example (that I've actually tested) can be seen in the two example_
files here.
在这里的两个示例文件中可以看到一个不同但相关的示例(我实际上测试过)。
#2
7
I'm currently using this code in a separated s3utils
module:
我目前在一个单独的s3utils模块中使用这个代码:
from django.core.exceptions import SuspiciousOperation
from django.utils.encoding import force_unicode
from storages.backends.s3boto import S3BotoStorage
def safe_join(base, *paths):
"""
A version of django.utils._os.safe_join for S3 paths.
Joins one or more path components to the base path component intelligently.
Returns a normalized version of the final path.
The final path must be located inside of the base path component (otherwise
a ValueError is raised).
Paths outside the base path indicate a possible security sensitive operation.
"""
from urlparse import urljoin
base_path = force_unicode(base)
paths = map(lambda p: force_unicode(p), paths)
final_path = urljoin(base_path + ("/" if not base_path.endswith("/") else ""), *paths)
# Ensure final_path starts with base_path and that the next character after
# the final path is '/' (or nothing, in which case final_path must be
# equal to base_path).
base_path_len = len(base_path) - 1
if not final_path.startswith(base_path) \
or final_path[base_path_len:base_path_len + 1] not in ('', '/'):
raise ValueError('the joined path is located outside of the base path'
' component')
return final_path
class StaticRootS3BotoStorage(S3BotoStorage):
def __init__(self, *args, **kwargs):
super(StaticRootS3BotoStorage, self).__init__(*args, **kwargs)
self.location = kwargs.get('location', '')
self.location = 'static/' + self.location.lstrip('/')
def _normalize_name(self, name):
try:
return safe_join(self.location, name).lstrip('/')
except ValueError:
raise SuspiciousOperation("Attempted access to '%s' denied." % name)
class MediaRootS3BotoStorage(S3BotoStorage):
def __init__(self, *args, **kwargs):
super(MediaRootS3BotoStorage, self).__init__(*args, **kwargs)
self.location = kwargs.get('location', '')
self.location = 'media/' + self.location.lstrip('/')
def _normalize_name(self, name):
try:
return safe_join(self.location, name).lstrip('/')
except ValueError:
raise SuspiciousOperation("Attempted access to '%s' denied." % name)
Then, in my settings module:
然后,在我的设置模块:
DEFAULT_FILE_STORAGE = 'myproyect.s3utils.MediaRootS3BotoStorage'
STATICFILES_STORAGE = 'myproyect.s3utils.StaticRootS3BotoStorage'
I got to redefine the _normalize_name()
private method to use a "fixed" version of the safe_join()
function, since the original code is giving me SuspiciousOperation
exceptions for legal paths.
我重新定义了_normalize_name()私有方法,以使用safe_join()函数的“固定”版本,因为原始代码为合法路径提供了可疑的操作异常。
I'm posting this for consideration, if anyone can give a better answer or improve this one, it will be very welcome.
我发这篇文章是为了考虑,如果有人能给出更好的答案或改进这篇文章,我将非常欢迎。
#3
5
File: PROJECT_NAME/custom_storages.py
文件:PROJECT_NAME / custom_storages.py
from django.conf import settings
from storages.backends.s3boto import S3BotoStorage
class StaticStorage(S3BotoStorage):
location = settings.STATICFILES_LOCATION
class MediaStorage(S3BotoStorage):
location = settings.MEDIAFILES_LOCATION
File: PROJECT_NAME/settings.py
文件:PROJECT_NAME / settings.py
STATICFILES_LOCATION = 'static'
MEDIAFILES_LOCATION = 'media'
if not DEBUG:
STATICFILES_STORAGE = 'PROJECT_NAME.custom_storages.StaticStorage'
DEFAULT_FILE_STORAGE = 'PROJECT_NAME.custom_storages.MediaStorage'
AWS_ACCESS_KEY_ID = 'KEY_XXXXXXX'
AWS_SECRET_ACCESS_KEY = 'SECRET_XXXXXXXXX'
AWS_STORAGE_BUCKET_NAME = 'BUCKET_NAME'
AWS_HEADERS = {'Cache-Control': 'max-age=86400',}
AWS_QUERYSTRING_AUTH = False
And run: python manage.py collectstatic
和运行:python管理。py collectstatic
#4
2
I think the answer is pretty simple and done by default. This is working for me on AWS Elastic Beanstalk with Django 1.6.5 and Boto 2.28.0:
我认为答案很简单,而且是默认的。这是为我做的AWS弹性豆茎,Django 1.6.5和Boto 2.28.0:
STATICFILES_FINDERS = (
'django.contrib.staticfiles.finders.FileSystemFinder',
'django.contrib.staticfiles.finders.AppDirectoriesFinder',
)
TEMPLATE_LOADERS = (
'django.template.loaders.filesystem.Loader',
'django.template.loaders.app_directories.Loader',
)
DEFAULT_FILE_STORAGE = 'storages.backends.s3boto.S3BotoStorage'
STATICFILES_STORAGE = 'storages.backends.s3boto.S3BotoStorage'
AWS_ACCESS_KEY_ID = os.environ['AWS_ACCESS_KEY_ID']
AWS_SECRET_ACCESS_KEY = os.environ['AWS_SECRET_KEY']
The AWS keys are passed in from the container config file and I have no STATIC_ROOT
or STATIC_URL
set at all. Also, no need for the s3utils.py
file. These details are handled by the storage system automatically. The trick here is that I needed to reference this unknown path in my templates correctly and dynamically. For example:
AWS键是从容器配置文件传入的,我根本没有设置STATIC_ROOT或STATIC_URL。同样,不需要s3utils。py文件。这些细节是由存储系统自动处理的。这里的诀窍是,我需要在模板中正确和动态地引用这个未知路径。例如:
<link rel="icon" href="{% static "img/favicon.ico" %}">
That is how I address my favicon which lives locally (pre-deployment) in ~/Projects/my_app/project/my_app/static/img/favicon.ico
.
这就是我如何在~/Projects/my_app/project/my_app/ my_app/static/img/favicon.ico中使用的favicon。
Of course I have a separate local_settings.py
file for accessing this stuff locally in dev environment and it does have STATIC and MEDIA settings. I had to do a lot of experimenting and reading to find this solution and it works consistently with no errors.
当然,我有一个单独的local_settings。在dev环境中访问这些东西的py文件,它有静态和媒体设置。我必须做大量的实验和阅读来找到这个解决方案,它始终没有错误。
I understand that you need the static and root separation and considering that you can only provide one bucket I would point out that this method takes all the folders in my local environment under ~/Projects/my_app/project/my_app/static/
and creates a folder in the bucket root (ie: S3bucket/img/ as in the example above). So you do get separation of files. For example you could have a media
folder in the static
folder and access it via templating with this:
我理解您需要静态和根分离,并且考虑到您只能提供一个bucket,我想指出的是,这个方法取我本地环境下~/Projects/my_app/project/my_app/static/,并在bucket根中创建一个文件夹(例如:S3bucket/img/上面的示例)。这样就得到了文件的分离。例如,您可以在静态文件夹中有一个媒体文件夹,并通过以下模板访问:
{% static "media/" %}
I hope this helps. I came here looking for the answer and pushed a bit harder to find a simpler solution than to extend the storage system. Instead, I read the documentation about the intended use of Boto and I found that a lot of what I needed was built-in by default. Cheers!
我希望这可以帮助。我来这里是为了寻找答案,并努力寻找比扩展存储系统更简单的解决方案。相反,我阅读了关于Boto预期用途的文档,发现我需要的很多东西都是默认内置的。干杯!