使用rails，nginx和send_file在Chrome中流式传输mp4

I can't for the life of me stream a mp4 to Chrome with a html5 <video> tag. If I drop the file in public then everything is gravy and works as expected. But if I try to serve it using send_file, pretty much everything imaginable goes wrong. I am using a rails app that is proxied by nginx, with a Video model that has a location attribute that is an absolute path on disk.

我不能为我的生活使用html5

At first I tried:

起初我尝试过:

def show
  send_file Video.find(params[:id]).location
end

And I was sure I would be basking in the glory that is modern web development. Ha. This plays in both Chrome and Firefox, but neither seek and neither have any idea how long the video is. I poked at the response headers and realized that Content-Type is being sent as application/octet-stream and there is no Content-Length set. Umm... wth?

而且我确信我会沉浸在现代网络发展的荣耀之中。哈。这可以在Chrome和Firefox中播放,但既不会寻求,也不会知道视频有多长。我在响应标头上戳了一下,发现Content-Type是作为application / octet-stream发送的,并且没有Content-Length集。嗯......好吗?

Okay, I guess I can set those in rails:

好吧,我想我可以在rails中设置它们:

def show
  video = Video.find(params[:id])
  response.headers['Content-Length'] = File.stat(video.location).size
  send_file(video.location, type: 'video/mp4')
end

At this point everything works pretty much as expected in Firefox. It knows how long the video is and seeking works as expected. Chrome appears to know how long the video is (doesn't show timestamps, but seek bar looks appropriate) but seeking doesn't work.

在这一点上,一切都与Firefox中预期的一样。它知道视频有多长,并且寻求按预期工作。 Chrome似乎知道视频的持续时间(不显示时间戳,但搜索条看起来合适),但搜索不起作用。

Apparently Chrome is pickier than Firefox. It requires that the server respond with a Accept-Ranges header with value bytes and respond to subsequent requests (that happen when the users seeks) with 206 and the appropriate portion of the file.

显然Chrome比Firefox更挑剔。它要求服务器使用带有值字节的Accept-Ranges标头进行响应,并使用206和文件的相应部分响应后续请求(在用户搜索时发生)。

Okay, so I borrowed some code from here and then I had this:

好的,所以我从这里借了一些代码然后我有了这个:

video = Video.find(params[:id])

file_begin = 0
file_size = File.stat(video.location).size
file_end = file_size - 1

if !request.headers["Range"]
  status_code = :ok
else
  status_code = :partial_content
  match = request.headers['Range'].match(/bytes=(\d+)-(\d*)/)
  if match
    file_begin = match[1]
    file_end = match[2] if match[2] && !match[2].empty?
  end
  response.header["Content-Range"] = "bytes " + file_begin.to_s + "-" + file_end.to_s + "/" + file_size.to_s
end
response.header["Content-Length"] = (file_end.to_i - file_begin.to_i + 1).to_s
response.header["Accept-Ranges"]=  "bytes"
response.header["Content-Transfer-Encoding"] = "binary"
send_file(video.location,
:filename => File.basename(video.location),
:type => 'video/mp4',
:disposition => "inline",
:status => status_code,
:stream =>  'true',
:buffer_size  =>  4096)

Now Chrome attempts to seek, but when you do the video stops playing and never works again until the page reloads. Argh. So I decided to play around with curl to see what was happening and I discovered this:

现在Chrome会尝试搜索,但是当您执行此操作时,视频会停止播放,并且在页面重新加载之前不会再次运行。哎呀。所以我决定玩弄curl来看看发生了什么,我发现了这个:

$ curl --header "Range: bytes=200-400" http://localhost:8080/videos/1/001.mp4 ftypisomisomiso2avc1mp41 �moovlmvhd��@��trak\tkh��

$ curl --header“范围:字节= 200-400”http:// localhost:8080 / videos / 1 / 001.mp4ftypisomisomiso2avc1mp41 moovlmvhd @ trak\ttkh

$ curl --header "Range: bytes=1200-1400" http://localhost:8080/videos/1/001.mp4 ftypisomisomiso2avc1mp41 �moovlmvhd��@��trak\tkh��

$ curl --header“范围:字节= 1200-1400”http:// localhost:8080 / videos / 1 / 001.mp4ftypisomisomiso2avc1mp41 moovlmvhd @ trak\ttkh

No matter the byte range request, the data always starts from the beginning of the file. The appropriate amount of bytes is returned (201 bytes in this case), but it's always from the beginning of the file. Apparently nginx respects the Content-Length header but ignores the Content-Range header.

无论字节范围请求如何,数据始终从文件的开头开始。返回适当的字节数(在这种情况下为201字节),但它总是从文件的开头。显然,nginx尊重Content-Length标头但忽略了Content-Range标头。

My nginx.conf is untouched default:

我的nginx.conf默认是不变的:

user www-data;
worker_processes 4;
pid /run/nginx.pid;

events {
        worker_connections 768;
}

http {
        sendfile on;
        tcp_nopush on;
        tcp_nodelay on;
        keepalive_timeout 65;
        types_hash_max_size 2048;

        include /etc/nginx/mime.types;
        default_type application/octet-stream;

        ssl_protocols TLSv1 TLSv1.1 TLSv1.2; # Dropping SSLv3, ref: POODLE
        ssl_prefer_server_ciphers on;

        access_log /var/log/nginx/access.log;
        error_log /var/log/nginx/error.log;

        gzip on;
        gzip_disable "msie6";

        include /etc/nginx/conf.d/*.conf;
        include /etc/nginx/sites-enabled/*;
}

and my app.conf is pretty basic:

我的app.conf非常基本:

upstream unicorn {
server unix:/tmp/unicorn.app.sock fail_timeout=0;
}

server {
listen 80 default deferred;
root /vagrant/public;
try_files $uri/index.html $uri @unicorn;
location @unicorn {
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header HOST $http_host;
    proxy_redirect off;
    proxy_pass http://unicorn;
}

error_page 500 502 503 504 /500.html;
client_max_body_size 4G;
keepalive_timeout 5;
}

First I tried the nginx 1.4.x that comes with Ubuntu 14.04, then tried 1.7.x from a ppa - same results. I even tried apache2 and had exactly the same results.

首先,我尝试了Ubuntu 14.04附带的nginx 1.4.x,然后从ppa尝试了1.7.x - 结果相同。我甚至尝试过apache2并得到完全相同的结果。

I would like to reiterate that the video file is not the problem. If I drop it in public then nginx serves it with the appropriate mime types, headers and everything needed for Chrome to work properly.

我想重申视频文件不是问题。如果我将其公之于众,那么nginx会使用适当的mime类型,标题以及Chrome正常运行所需的一切来为其提供服务。

So my question is a two-parter:

所以我的问题是两个部分:

Why doesn't nginx/apache handle all this stuff automagically with send_file (X-Accel-Redirect/X-Sendfile) like it does when the file is served statically from public? Handling this stuff in rails is so backwards.

为什么nginx / apache不能像send_file(X-Accel-Redirect / X-Sendfile)一样自动处理所有这些东西,就像从公共场所静态提供文件一样?在轨道中处理这些东西是如此倒退。
How the heck can I actually use send_file with nginx (or apache) so that Chrome will be happy and allow seeking?

我怎么能真正使用带有nginx(或apache)的send_file,这样Chrome会很高兴并且允许搜索?

Update 1

Okay, so I thought I'd try to take the complication of rails out of the picture and just see if I could get nginx to proxy the file correctly. So I spun up a dead-simple nodjs server:

好吧,所以我想我会尝试将rails的复杂性从图片中删除,然后看看我是否可以让nginx正确地代理文件。所以我创建了一个简单的nodjs服务器:

var http = require('http');
http.createServer(function (req, res) {
res.writeHead(200, {
    'X-Accel-Redirect': '/path/to/file.mp4'
});
res.end();
}).listen(3000, '127.0.0.1');
console.log('Server running at http://127.0.0.1:3000/');

And chrome is happy as a clam. =/ curl -I even shows that Accept-Ranges: bytes and Content-Type: video/mp4 is being inserted by nginx automagically - as it should be. What could rails be doing that's preventing nginx from doing this?

而铬则是一种快乐的蛤蜊。 = / curl - 我甚至显示了Accept-Ranges:bytes和Content-Type:video / mp4正在被nginx自动插入 - 应该是这样。有什么可以阻止nginx这样做?

Update 2

I might be getting closer...

我可能会越来越近了......

If I have:

如果我有:

def show
  video = Video.find(params[:id])
  send_file video.location
end

Then I get:

然后我得到:

$ curl -I localhost:8080/videos/1/001.mp4
HTTP/1.1 200 OK
Server: nginx/1.7.9
Date: Sun, 18 Jan 2015 12:06:38 GMT
Content-Type: application/octet-stream
Connection: keep-alive
Status: 200 OK
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
X-Content-Type-Options: nosniff
Content-Disposition: attachment; filename="001.mp4"
Content-Transfer-Encoding: binary
Cache-Control: private
Set-Cookie: request_method=HEAD; path=/
X-Meta-Request-Version: 0.3.4
X-Request-Id: cd80b6e8-2eaa-4575-8241-d86067527094
X-Runtime: 0.041953

And I have all the problems described above.

我有上述所有问题。

But if I have:

但如果我有:

def show
  video = Video.find(params[:id])
  response.headers['X-Accel-Redirect'] = video.location
  head :ok
end

Then I get:

然后我得到:

$ curl -I localhost:8080/videos/1/001.mp4
HTTP/1.1 200 OK
Server: nginx/1.7.9
Date: Sun, 18 Jan 2015 12:06:02 GMT                                                                                                                                                                            
Content-Type: text/html                                                                                                                                                                                        
Content-Length: 186884698                                                                                                                                                                                      
Last-Modified: Sun, 18 Jan 2015 03:49:30 GMT                                                                                                                                                                   
Connection: keep-alive                                                                                                                                                                                         
Cache-Control: max-age=0, private, must-revalidate
Set-Cookie: request_method=HEAD; path=/
ETag: "54bb2d4a-b23a25a"
Accept-Ranges: bytes

And everything works perfectly.

一切都很完美。

But why? Those should do exactly the same thing. And why doesn't nginx set Content-Type automagically here like it does for the simple nodejs example? I have config.action_dispatch.x_sendfile_header = 'X-Accel-Redirect' set. I have moved it back and forth between application.rb and development.rb with the same results. I guess I never mentioned... this is rails 4.2.0.

但为什么?那些应该做同样的事情。为什么nginx在这里没有像在简单的nodejs示例中那样自动设置Content-Type?我有config.action_dispatch.x_sendfile_header ='X-Accel-Redirect'设置。我在application.rb和development.rb之间来回移动它,结果相同。我想我从来没有提到......这是rails 4.2.0。

Update 3

Now I've changed my unicorn server to listen on port 3000 (since I already changed nginx to listen on 3000 for the nodejs example). Now I can make requests directly to unicorn (since it's listening on a port and not a socket) so I have found that curl -I directly to unicorn shows that no X-Accel-Redirect header is sent and just curling unicorn directly actually sends the file. It's like send_file isn't doing what it's supposed to.

现在我已经将我的独角兽服务器改为侦听端口3000(因为我已经将nginx更改为在3000上侦听nodejs示例)。现在我可以直接向独角兽发出请求(因为它正在侦听端口而不是套接字)所以我发现curl -I直接向unicorn显示没有发送X-Accel-Redirect标头而只是直接卷曲独角兽实际发送文件。这就像send_file没有做到它应该做的事情。

1 个解决方案

#1

I finally have the answers to my original questions. I didn't think I'd ever get here. All my research had lead to dead-ends, hacky non-solutions and "it just works out of the box" (well, not for me).

我终于得到了原始问题的答案。我不认为我会来这里。我所有的研究都导致死胡同,hacky非解决方案和“它只是开箱即用”(好吧,不适合我)。

Why doesn't nginx/apache handle all this stuff automagically with send_file (X-Accel-Redirect/X-Sendfile) like it does when the file is served statically from public? Handling this stuff in rails is so backwards.

为什么nginx / apache不能像send_file(X-Accel-Redirect / X-Sendfile)一样自动处理所有这些东西,就像从公共场所静态提供文件一样?在轨道中处理这些东西是如此倒退。

They do, but they have to be configured properly to please Rack::Sendfile (see below). Trying to handle this in rails is a hacky non-solution.

他们这样做,但必须正确配置以取悦Rack :: Sendfile(见下文)。试图在rails中处理这个问题是一个非常糟糕的解决方案。

How the heck can I actually use send_file with nginx (or apache) so that Chrome will be happy and allow seeking?

我怎么能真正使用带有nginx(或apache)的send_file,这样Chrome会很高兴并且允许搜索?

I got desperate enough to start poking around rack source code and that's where I found my answer, in the comments of Rack::Sendfile. They are structured as documentation that you can find at rubydoc.

我非常绝望地开始讨论机架源代码,这就是我在Rack :: Sendfile的评论中找到答案的地方。它们的结构是您可以在rubydoc找到的文档。

For whatever reason, Rack::Sendfile requires the front end proxy to send a X-Sendfile-Type header. In the case of nginx it also requires a X-Accel-Mapping header. The documentation also has examples for apache and lighttpd as well.

无论出于何种原因,Rack :: Sendfile都需要前端代理发送X-Sendfile-Type标头。在nginx的情况下,它还需要X-Accel-Mapping头。该文档还提供了apache和lighttpd的示例。

One would think the rails documentation could link to the Rack::Sendfile documentation since send_file does not work out of the box without additional configuration. Perhaps I'll submit a pull request.

有人会认为rails文档可以链接到Rack :: Sendfile文档,因为如果没有额外的配置,send_file就无法开箱即用。也许我会提交拉取请求。

In the end I only needed to add a couple lines to my app.conf:

最后我只需要在app.conf中添加几行:

upstream unicorn {
  server unix:/tmp/unicorn.app.sock fail_timeout=0;
}

server {
  listen 80 default deferred;
  root /vagrant/public;
  try_files $uri/index.html $uri @unicorn;
  location @unicorn {
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header HOST $http_host;
    proxy_set_header X-Sendfile-Type X-Accel-Redirect; # ADDITION
    proxy_set_header X-Accel-Mapping /=/; # ADDITION
    proxy_redirect off;
    proxy_pass http://localhost:3000;
  }

  error_page 500 502 503 504 /500.html;
  client_max_body_size 4G;
  keepalive_timeout 5;
}

Now my original code works as expected:

现在我的原始代码按预期工作:

def show
  send_file(Video.find(params[:id]).location)
end

Edit:

Although this worked initially, it stopped working after I restarted my vagrant box and I had to make further changes:

虽然这最初起作用,但是在我重新启动我的流浪盒之后它停止了工作,我不得不做出进一步的改变:

upstream unicorn {
  server unix:/tmp/unicorn.app.sock fail_timeout=0;
}

server {
  listen 80 default deferred;
  root /vagrant/public;
  try_files $uri/index.html $uri @unicorn;

  location ~ /files(.*) { # NEW
    internal;             # NEW
    alias $1;             # NEW
  }                       # NEW

  location @unicorn {
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header HOST $http_host;
    proxy_set_header X-Sendfile-Type X-Accel-Redirect;
    proxy_set_header X-Accel-Mapping /=/files/; # CHANGED
    proxy_redirect off;
    proxy_pass http://localhost:3000;
  }

  error_page 500 502 503 504 /500.html;
  client_max_body_size 4G;
  keepalive_timeout 5;
}

I find this whole thing of mapping one URI to another and then mapping that URI to a location on disk to be totally unnecessary. It's useless for my use case and I'm just mapping one to another and back again. Apache and lighttpd don't require it. But at least it works.

我发现将一个URI映射到另一个URI然后将该URI映射到磁盘上的位置完全没必要。它对我的用例没用,我只是将一个映射到另一个并再次映射。 Apache和lighttpd不需要它。但至少它有效。

I also added Mime::Type.register('video/mp4', :mp4) to config/initializers/mime_types.rb so the file is served with the correct mime type.

我还将Mime :: Type.register('video / mp4',:mp4)添加到config / initializers / mime_types.rb,以便使用正确的mime类型提供文件。

#1