Ruby 2.0 Open Uri always uses proxy, even with proxy = nil flag

Question

The following call is made in our code base:

require 'open-uri'

open(url, :proxy => nil)

However, when the call is made, Open uri uses the http_proxy environment variable to make the call, which is effectively blocked by our firewall. According to the docs when passing nil or false to the proxy option, the environment variables should be ignored as well, but this doesn't seem to be the case.

If we either switch to Ruby version 1.8.7 or unset the environment variable, the call is made successfully without the proxy. The problem is that we need to use a proxy for calls to the internet, but we cannot use them for internal calls (which is the case here) due to our firewall configuration.

Any help would be greatly appreciated!

Answer 1

I can reproduce what you described

http_proxy=http://127.0.0.1 ruby -e "require 'open-uri'; open('http://google.com', proxy: nil)"
~/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/net/http.rb:878:in `initialize': Connection refused - connect(2) (Errno::ECONNREFUSED)
    from ~/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/net/http.rb:878:in `open'
    from ~/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/net/http.rb:878:in `block in connect'
    from ~/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/timeout.rb:52:in `timeout'
    from ~/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/net/http.rb:877:in `connect'
    from ~/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/net/http.rb:862:in `do_start'
    from ~/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/net/http.rb:851:in `start'
    from ~/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/open-uri.rb:313:in `open_http'
    from ~/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/open-uri.rb:708:in `buffer_open'
    from ~/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/open-uri.rb:210:in `block in open_loop'
    from ~/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/open-uri.rb:208:in `catch'
    from ~/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/open-uri.rb:208:in `open_loop'
    from ~/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/open-uri.rb:149:in `open_uri'
    from ~/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/open-uri.rb:688:in `open'
    from ~/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/open-uri.rb:34:in `open'
    from -e:1:in `<main>'

I've digged the code, and I think I've found the problem.

So far open-uri manage correctly the proxy. For instance, in OpenURI.open_loop, it correctly considers that the proxy is set to nil (see open-uri.rb#195

case opt_proxy
when true
  find_proxy = lambda {|u| pxy = u.find_proxy; pxy ? [pxy, nil, nil] : nil}
when nil, false
  find_proxy = lambda {|u| nil} # <-- THIS LINE

But when it creates a Net::HTTP instance into OpenURI.open_http

http = klass.new(target_host, target_port)

it doesn't pass p_addr = nil. According the source code and doc for HTTP.new

def HTTP.new(address, port = nil, p_addr = :ENV, p_port = nil, p_user = nil, p_pass = nil)
  http = super address, port

  if proxy_class? then # from Net::HTTP::Proxy()
    http.proxy_from_env = @proxy_from_env
    http.proxy_address = @proxy_address
    http.proxy_port = @proxy_port
    http.proxy_user = @proxy_user
    http.proxy_pass = @proxy_pass
  elsif p_addr == :ENV then # <---- THIS LINE
    http.proxy_from_env = true
  else
    http.proxy_address = p_addr
    http.proxy_port = p_port || default_port
    http.proxy_user = p_user
    http.proxy_pass = p_pass
  end

  http
end

Above the doc says

To disable proxy detection set +p_addr+ to nil.

I've patched open-uri.rb by replacing the line 291 by

http = proxy ? klass.new(target_host, target_port) : klass.new(target_host, target_port, nil)

Now, it behaves as expected.

---

just filled an issue in Ruby bug tracker about that.