We have a gem which as development dependencies relies on sass-rails (specifically in version 5.0.4) which in turn depends on railties >4.0.0 and <5.0.
Now we don't check our Gemfile.lock into the repo to keep it as flexible as possible (it's our 'universal' gem after all), and during the travis build, installing sass-rails resolves its dependency railties to version 5.0.0.beta1, which I'd say intuitively isn't <5.0 and now causes problems when running the tests on ruby < 2.2.2 (due to rack).
Now, am I misunderstanding something or is this a bug, that bundler installs 5.0.0.beta1 for <5.0?
This is not a bug but expected behaviour. Prerelease versions are expected to be released before the actual release, hence they are considered smaller.
See http://ruby-doc.org/stdlib-2.0.0/libdoc/rubygems/rdoc/Gem/Version.html for more detail.
You can add a runtime dependency to
railties '~> 4.0'to your gemspec to solve this problem.