Rails send_file not working with files larger than 400mb

Question

I used Helicon Zoo to set up a rails application on a Windows Server 2008 machine.

My problem is downloading files above 400MB.

In my rails app I use the following to send files to a client:

app/controllers/hosted_files_controller.rb

class HostedFilesController < ApplicationController
  before_filter :authenticate_user!
  around_filter :catch_not_foun

  def download
    @footprint = UserAbility.new(current_user).footprints.find(params[:id])
    send_file path
  end

  private

    def path
      if @footprint.subpath?
        @path = "#{HOSTED_FILES_PATH}\\#{@footprint.subpath}\\#{@footprint.filename}"
      else
        @path = "#{HOSTED_FILES_PATH}\\#{@footprint.filename}"
      end
    end

    def catch_not_found
      yield
    rescue ActiveRecord::RecordNotFound
      recover_and_log "We couldn't find that record.", "No record found using the id (#{params[:id]})"
    rescue ActionController::MissingFile
      recover_and_log "We couldn't find the file on our server.", "The file was not found at the following path: #{@path}"
    end

    def recover_and_log (displayed, logged)
      logger.info "!!! Error: #{logged}"
      redirect_to root_url, alert: displayed
    end
end

I have config.action_dispatch.x_sendfile_header commented out in the production.rb file since I am not using Apache or Nginx.

This works great for all the files on the server that are below ~400MB. After I get above it, I get a 500 internal server error from Helicon Zoo that says the following:

Helicon Zoo module has caught up an error. Please see the details below.
Worker Status
The process was created
Windows error
The pipe has been ended. (ERROR CODE: 109)
Internal module error
message: ZooApplication backend read Error. 
type: ZooException
file: Jobs\JobBase.cpp
line: 566 
version: 3.1.98.508 
STDERR
Empty stderr

Does anyone have any idea what is going on? I'm at a loss.

I've tried:

• increasing the buffer_size on send_file (didn't work)
• play around with memory settings in IIS for the application pool (didn't work)
• change x_sendfile_header to X-Sendfile and X-Accel-Redirect (didn't work)

I'm considering trying to install Apache on the Windows Server and using the x_sendfile_header to offload sending the file to Apache, but I'm afraid of messing up the already (almost) working application.

Does anyone have any ideas of how to fix this?

Answer 1

By default with current version of Helicon Zoo Ruby applications are installed as FastCGI Ruby Rack connector. Since FastCGI protocol is a blocking protocol it may have some limitations on request timeout or request max size. If you need to send large files I suggest you to go "Ruby 1.9 Rack over HTTP with Thin" route instead. I suppose you've been following Ruby on Rails (2.3.x and 3.x.x) instruction. Now just follow additional steps from Ruby Rack over HTTP with Thin instruction, like running "gem install thin" command and editing web.config as follows:

In the < handlers> section comment out two lines that follows

< !-- Ruby 1.9 over FastCGI -->

Uncomment two lines that follows 
< !-- Ruby 1.9 over HTTP, using Thin as a back-end application server -->
In the <environmentVariables> section uncomment line 
< !-- Use this APP_WORKER with HTTP Ruby engine and Thin. Thin need to be installed.
< add name="APP_WORKER" value="GEM_HOME\bin\thin start" />

Another solution, since you already using Helicon products, would be installing Helicon Ape that provides support for X-Sendfile HTTP header (see documentation) header as in Apache and is free for several sites per server. This solution would be even better since low level WinHTTP code will be used to send data, which will decrease load to the server and improve response speed.