大型视频上传和处理超时

I am working on a rails application where users upload some videos. Apache with passenger is my server. The videos can be as big as 500MB or 1 GB. After the video is uploaded it is converted to mp4 and played.

我正在开发一个rails应用程序，用户可以在其中上传一些视频。带乘客的Apache是我的服务器。视频大小可达500MB或1gb。视频上传后被转换为mp4并播放。

I am using https://github.com/valums/file-uploader to upload the files. With this i can upload as large as a 500MB video file easily. I want to display the mp4 preview of the video as soon as the upload is complete. I am using ffmpeg to process convert the video to mp4. This conversion may take as long as 10+ minutes some times and my request times out after 5 mins due to the default apache timeout.

我正在使用https://github.com/valums/fileuploader上传文件。有了这个，我可以轻松地上传一个500MB的视频文件。我想在上传完成后立即显示视频的mp4预览。我正在使用ffmpeg处理将视频转换为mp4。由于默认的apache超时，这种转换可能需要10分钟以上的时间，而我的请求在5分钟后就会超时。

So, the problem here is the ajax upload keeps the request alive only till 5 minutes after the upload to complete. I need the request to stay alive for about 10-20 minutes after the file upload is complete. It would be great to know in case any other javascipt based solutions available to keep this request alive till the video processing completes and the preview can be shown to the user.

因此，这里的问题是ajax上传使请求在上传完成后5分钟内保持活跃。我需要请求在文件上传完成后保持10-20分钟的生命。如果有任何其他基于javascript的解决方案可以在视频处理完成并向用户显示预览之前让这个请求保持活动状态，那么最好知道这一点。

3 个解决方案

#1

Keeping a blocking request until the video is transcoded can burden your servers. Most servers can handle 40-80 concurrent requests concurrently, depending on the memory footprint of your Rails stack, gems installed, and Apache server setting. If you allow 10 minutes requests, this means that 40 users who are uploading simultaneously could take down your service.

在传输视频之前保持阻塞请求可能会增加服务器的负担。大多数服务器可以同时处理40-80个并发请求，这取决于您的Rails堆栈、gem安装和Apache服务器设置的内存占用情况。如果你允许10分钟的请求，这意味着同时上传的40个用户可能会占用你的服务。

My suggestion is using Ajax to poll your Rails server every few seconds if the job is done transcoding. If it does, you can refresh the page or add some more javascript to initalize the video playback.

我的建议是，如果任务完成了代码转换，那么每隔几秒钟就使用Ajax轮询Rails服务器。如果有的话，你可以刷新页面或者添加更多的javascript来进行视频回放。

window.setInterval( function() {
  $.get('video_path').done(function(data, code, xhr) {
    location.reload(); // or some code for playing back the video
  });
}, 2000);

Another similar solution could use a third party service like Pusher or PubNub which allow browsers to maintain persistent connection, and then allow your Rails server to notify the clients once an event is fired. Push services like these usually use Node.Js for better asynchronous handling and maintaing many open connections concurrently.

另一个类似的解决方案可以使用第三方服务，比如Pusher或PubNub，它允许浏览器维护持久连接，然后允许Rails服务器在事件触发时通知客户端。像这样的Push服务通常使用Node。为了更好地异步处理和同时维护许多打开的连接。

Third option would be to switch Apache to Puma and use Rails new streaming APIs. Puma is better suited to handling concurrency.

第三种选择是将Apache切换到Puma，并使用Rails新的流api。Puma更适合处理并发。

#2

This sounds like an Apache config directive (KeepAliveTimeout and TimeOut).

这听起来像是Apache配置指令(KeepAliveTimeout和TimeOut)。

KeepAliveTimeout 1200 # 60sec * 20 min

持续超时1200 # 60sec * 20分钟

More info & syntax from Apache here: http://httpd.apache.org/docs/2.2/mod/core.html#keepalivetimeout

Apache的更多信息和语法:http://httpd.apache.org/docs/2.2/mod/core.html#keepalivetimeout

An alternative: can you modify your JS/AJAX to re-request every 2 mins., and check if a preview exists yet or not?

另一种选择:能否修改JS/AJAX，每两分钟重新请求一次。，并检查预览是否存在?

#3

Two tomcat parameters (ATTENTION! Only available after Tomcat 7.0) allow you to keep the session alive while running your request. Also, only after the request is finished, the session timeout will start counting again:

两个tomcat参数(注意!只有在Tomcat 7.0之后才可用，允许您在运行请求时保持会话存活。此外，只有在请求完成后，会话超时才会重新开始计数:

org.apache.catalina.session.StandardSession.ACTIVITY_CHECK=true
org.apache.catalina.session.StandardSession.LAST_ACCESS_AT_START=false

You can set both in server.xml or pass them through JAVA_OPTS:

您可以在服务器中设置这两个。xml或通过JAVA_OPTS传递:

-XX:MaxPermSize=252m -Dorg.apache.catalina.session.StandardSession.ACTIVITY_CHECK=true -Dorg.apache.catalina.session.StandardSession.LAST_ACCESS_AT_START=false

#1

window.setInterval( function() {
  $.get('video_path').done(function(data, code, xhr) {
    location.reload(); // or some code for playing back the video
  });
}, 2000);