Django保存对统计数据的整个请求,有什么可用的?

时间:2021-09-19 16:57:32

I want to save everything that can be used for statistics, such as referrer, os, browser etc. What is available and what's the best way to store it?

我想保存所有可以用于统计的东西,比如referrer, os,浏览器等等。什么是可用的,最好的存储方式是什么?

This is only important for 1 application (1 page) in the project, the other pages some standard analytics product will be used such as google analytics.

这对于项目中的1个应用程序(1页)来说是非常重要的,其他的页面,一些标准的分析产品将会被使用,比如谷歌分析。

I had a look at django-tracking, but it seems this is overkill as I only want to use it on 1 view. The ideal situation would be, passing the whole request object to a TaskQue and do the processing later. So the user is redirected first and the analytics processing will be done behind the scenes.

我看了一下django-tracking,但是这似乎有点过头了,因为我只想在一个视图中使用它。理想的情况是,将整个请求对象传递给TaskQue并稍后进行处理。因此用户首先被重定向,分析处理将在幕后完成。

3 个解决方案

#1


12  

We use some simple middleware.. below is an excerpt. You can modify it to use directly within a view.

我们使用一些简单的中间件。下面是一段。您可以修改它以直接在视图中使用。

class WebRequest(models.Model):
    time = models.DateTimeField(auto_now_add=True)
    host = models.CharField(max_length=1000)
    path = models.CharField(max_length=1000)
    method = models.CharField(max_length=50)
    uri = models.CharField(max_length=2000)
    status_code = models.IntegerField()
    user_agent = models.CharField(max_length=1000,blank=True,null=True)
    remote_addr = models.IPAddressField()
    remote_addr_fwd = models.IPAddressField(blank=True,null=True)
    meta = models.TextField()
    cookies = models.TextField(blank=True,null=True)
    get = models.TextField(blank=True,null=True)
    post = models.TextField(blank=True,null=True)
    raw_post = models.TextField(blank=True,null=True)
    is_secure = models.BooleanField()
    is_ajax = models.BooleanField()
    user = models.ForeignKey(User,blank=True,null=True)

def dumps(value):
    return json.dumps(value,default=lambda o:None)

class WebRequestMiddleware(object):
    def process_view(self, request, view_func, view_args, view_kwargs):
        setattr(request,'hide_post',view_kwargs.pop('hide_post',False))


    def process_response(self, request, response):

        if request.path.endswith('/favicon.ico'):
            return response

        if type(response) == HttpResponsePermanentRedirect and settings.APPEND_SLASH:
            new_location = response.get('location',None)
            content_length = response.get('content-length',None)

            if new_location and content_length is '0':
                new_parsed = urlparse(new_location)

                old = (('http','https')[request.is_secure()], request.get_host(), '{0}/'.format(request.path), request.META['QUERY_STRING'])
                new = (new_parsed.scheme, new_parsed.netloc, new_parsed.path, new_parsed.query)

                if old == new:
                    #dont log - it's just adding a /
                    return response
        try:
            self.save(request, response)
        except Exception as e:
            print >> sys.stderr, "Error saving request log", e

        return response

    def save(self, request, response):
        if hasattr(request, 'user'):
            user = request.user if type(request.user) == User else None
        else:
            user = None

        meta = request.META.copy()
        meta.pop('QUERY_STRING',None)
        meta.pop('HTTP_COOKIE',None)
        remote_addr_fwd = None

        if 'HTTP_X_FORWARDED_FOR' in meta:
            remote_addr_fwd = meta['HTTP_X_FORWARDED_FOR'].split(",")[0].strip()
            if remote_addr_fwd == meta['HTTP_X_FORWARDED_FOR']:
                meta.pop('HTTP_X_FORWARDED_FOR')

        post = None
        uri = request.build_absolute_uri()
        if request.POST and uri != '/login/':
            post = dumps(request.POST)

        models.WebRequest(
            host = request.get_host(),
            path = request.path,
            method = request.method,
            uri = request.build_absolute_uri(),
            status_code = response.status_code,
            user_agent = meta.pop('HTTP_USER_AGENT',None),
            remote_addr = meta.pop('REMOTE_ADDR',None),
            remote_addr_fwd = remote_addr_fwd,
            meta = None if not meta else dumps(meta),
            cookies = None if not request.COOKIES else dumps(request.COOKIES),
            get = None if not request.GET else dumps(request.GET),
            post = None if (not request.POST or getattr(request,'hide_post') == True) else dumps(request.POST),
            raw_post = None if getattr(request,'hide_post') else request.raw_post_data,
            is_secure = request.is_secure(),
            is_ajax = request.is_ajax(),
            user = user
        ).save()

#2


1  

Just manually pull it from the request.

只需手动从请求中提取它。

The docs outlines a lot of the info that can be pulled from the request object.

文档列出了许多可以从请求对象中提取的信息。

For example, headers are stored in request.META, GET params in request.GET, etc.
http://docs.djangoproject.com/en/dev/ref/request-response/#django.http.HttpRequest.META

例如,header被存储在request中。元,请求获得参数。得到等http://docs.djangoproject.com/en/dev/ref/request-response/ django.http.HttpRequest.META

What's the best way to store it? Depends what you're doing. Log it, store it in a DB, send it somewhere else... You say for statistics, so a database sounds like a good place to put this as it's easy to query.

最好的储存方式是什么?取决于你在做什么。记录它,存储在DB中,发送到其他地方……对于统计数据,数据库听起来是个不错的地方,因为它很容易查询。

#3


0  

Extension to Josh answer, you could use JSONField for post data if you are using postgres as your backend. It will help in dealing with json directly rather than loading it manually.

扩展到Josh answer,如果使用postgres作为后端,您可以使用JSONField来发布数据。它将有助于直接处理json,而不是手工加载它。

read more: https://docs.djangoproject.com/en/2.0/ref/contrib/postgres/fields/#jsonfield

阅读更多:https://docs.djangoproject.com/en/2.0/ref/contrib/postgres/fields/ # jsonfield

you could do something like this

你可以这样做

from django.contrib.postgres.fields import JSONField

class WebRequest(models.Model):
    post = JSONField(default=dict)

#1


12  

We use some simple middleware.. below is an excerpt. You can modify it to use directly within a view.

我们使用一些简单的中间件。下面是一段。您可以修改它以直接在视图中使用。

class WebRequest(models.Model):
    time = models.DateTimeField(auto_now_add=True)
    host = models.CharField(max_length=1000)
    path = models.CharField(max_length=1000)
    method = models.CharField(max_length=50)
    uri = models.CharField(max_length=2000)
    status_code = models.IntegerField()
    user_agent = models.CharField(max_length=1000,blank=True,null=True)
    remote_addr = models.IPAddressField()
    remote_addr_fwd = models.IPAddressField(blank=True,null=True)
    meta = models.TextField()
    cookies = models.TextField(blank=True,null=True)
    get = models.TextField(blank=True,null=True)
    post = models.TextField(blank=True,null=True)
    raw_post = models.TextField(blank=True,null=True)
    is_secure = models.BooleanField()
    is_ajax = models.BooleanField()
    user = models.ForeignKey(User,blank=True,null=True)

def dumps(value):
    return json.dumps(value,default=lambda o:None)

class WebRequestMiddleware(object):
    def process_view(self, request, view_func, view_args, view_kwargs):
        setattr(request,'hide_post',view_kwargs.pop('hide_post',False))


    def process_response(self, request, response):

        if request.path.endswith('/favicon.ico'):
            return response

        if type(response) == HttpResponsePermanentRedirect and settings.APPEND_SLASH:
            new_location = response.get('location',None)
            content_length = response.get('content-length',None)

            if new_location and content_length is '0':
                new_parsed = urlparse(new_location)

                old = (('http','https')[request.is_secure()], request.get_host(), '{0}/'.format(request.path), request.META['QUERY_STRING'])
                new = (new_parsed.scheme, new_parsed.netloc, new_parsed.path, new_parsed.query)

                if old == new:
                    #dont log - it's just adding a /
                    return response
        try:
            self.save(request, response)
        except Exception as e:
            print >> sys.stderr, "Error saving request log", e

        return response

    def save(self, request, response):
        if hasattr(request, 'user'):
            user = request.user if type(request.user) == User else None
        else:
            user = None

        meta = request.META.copy()
        meta.pop('QUERY_STRING',None)
        meta.pop('HTTP_COOKIE',None)
        remote_addr_fwd = None

        if 'HTTP_X_FORWARDED_FOR' in meta:
            remote_addr_fwd = meta['HTTP_X_FORWARDED_FOR'].split(",")[0].strip()
            if remote_addr_fwd == meta['HTTP_X_FORWARDED_FOR']:
                meta.pop('HTTP_X_FORWARDED_FOR')

        post = None
        uri = request.build_absolute_uri()
        if request.POST and uri != '/login/':
            post = dumps(request.POST)

        models.WebRequest(
            host = request.get_host(),
            path = request.path,
            method = request.method,
            uri = request.build_absolute_uri(),
            status_code = response.status_code,
            user_agent = meta.pop('HTTP_USER_AGENT',None),
            remote_addr = meta.pop('REMOTE_ADDR',None),
            remote_addr_fwd = remote_addr_fwd,
            meta = None if not meta else dumps(meta),
            cookies = None if not request.COOKIES else dumps(request.COOKIES),
            get = None if not request.GET else dumps(request.GET),
            post = None if (not request.POST or getattr(request,'hide_post') == True) else dumps(request.POST),
            raw_post = None if getattr(request,'hide_post') else request.raw_post_data,
            is_secure = request.is_secure(),
            is_ajax = request.is_ajax(),
            user = user
        ).save()

#2


1  

Just manually pull it from the request.

只需手动从请求中提取它。

The docs outlines a lot of the info that can be pulled from the request object.

文档列出了许多可以从请求对象中提取的信息。

For example, headers are stored in request.META, GET params in request.GET, etc.
http://docs.djangoproject.com/en/dev/ref/request-response/#django.http.HttpRequest.META

例如,header被存储在request中。元,请求获得参数。得到等http://docs.djangoproject.com/en/dev/ref/request-response/ django.http.HttpRequest.META

What's the best way to store it? Depends what you're doing. Log it, store it in a DB, send it somewhere else... You say for statistics, so a database sounds like a good place to put this as it's easy to query.

最好的储存方式是什么?取决于你在做什么。记录它,存储在DB中,发送到其他地方……对于统计数据,数据库听起来是个不错的地方,因为它很容易查询。

#3


0  

Extension to Josh answer, you could use JSONField for post data if you are using postgres as your backend. It will help in dealing with json directly rather than loading it manually.

扩展到Josh answer,如果使用postgres作为后端,您可以使用JSONField来发布数据。它将有助于直接处理json,而不是手工加载它。

read more: https://docs.djangoproject.com/en/2.0/ref/contrib/postgres/fields/#jsonfield

阅读更多:https://docs.djangoproject.com/en/2.0/ref/contrib/postgres/fields/ # jsonfield

you could do something like this

你可以这样做

from django.contrib.postgres.fields import JSONField

class WebRequest(models.Model):
    post = JSONField(default=dict)