Ruby on Rails test coverage with simplecov

Question

I want to analyse the test coverage of our code , and therefore, installed the simplecov gem.

Our testing environement has 2 seperate project: REST API test (Java+Rest-Assured) and Web UI testing (Java-Selenium).
As you can see, we dont have unit testing inside the rails app, and we are testing using external projects.

I configured the simplecov gem as descriped in the tutorial and put this, in the rails script:

require 'simplecov'
SimpleCov.start 'rails'
puts "require simplecov"

When loading the app, I see the string I printed.

I ran both automation test projects, saw their printouts in the rails log, but I don't see any coverage of controllers/models, I see only small precentage of initializtion files of some gems and rails.

I searched the net, and tried putting the code phrase in boot.rb or even puma.rb and it returned the same results.

Any ideas?

EDIT

Nothing helped with all the comments, but I figured out something very interesting, in all cases, I only see the name of methods marked as tested, not the content (in controllers).
I tried to put the simplecov start phrase in both bin/rails, puma.rb, config.ru, environment.rb, all not given the desired results of code coverage.

Answer 1

I'm not sure simplecov can measure the whole rails app coverage... But I googled something that you can attach as a rack middleware:

https://github.com/danmayer/coverband

And it's output is compatible with simplecov. So it looks like it could be useful in your case.

As you mentioned in your question you're using puma. I suspect that, since it's multi-threaded, it spawns a few rails apps and their simplecov output overwrites each other's results. I'd try with the single threaded server like webrick - but this may make your tests slower (depending on how the tests are fired up really) or try the coverband gem.

Also - even if the server is single threaded - I'm not sure if each request would not overwrite simplecov's output.

Answer 2

Maybe you have to specify the paths

require 'simplecov'
SimpleCov.start do

  # add_filter '/admin/'

  add_group "Models", "app/models"
  add_group "Controllers", "app/controllers"
  add_group "Lib", "lib/"
  add_group "Helpers", "app/helpers"
end

Answer 3

You need to to start SimpleCov before loading any of your files, so put these lines as early as possible in your ruby entrypoint:

require 'simplecov'
SimpleCov.start

You could see an example in one of my repos here: https://github.com/tareksamni/DockUp/blob/master/spec/spec_helper.rb

I do autoload my ruby code after starting SimpleCov. You need to the same as well:

require 'simplecov'
SimpleCov.start

require './autoload'

Answer 4

I had the same problem trying to get coverage for the e2e playwright tests. Meanwhile subprocess handling was added to SimpleCov in 2020, so I created unicorn conf for the CI

timeout          60
worker_processes 1
preload_app      true
listen           3000

app_path =  File.expand_path('../', __dir__)    
stderr_path "#{app_path}/log/unicorn.stderr.log"

if ENV['CI']
  require File.expand_path('../lib/tutuf/simplecov', __dir__)
  SimpleCov.command_name 'End-to-End Tests'
  SimpleCov.enable_for_subprocesses true
  SimpleCov.at_fork do |pid|
    # This needs a unique name so it won't be overwritten
    SimpleCov.command_name "#{SimpleCov.command_name} (subprocess: #{pid})"
    SimpleCov.formatter SimpleCov::Formatter::LcovFormatter
    SimpleCov.minimum_coverage 0
    SimpleCov.start
  end
end

Then started the unicorn server with

bundle exec unicorn_rails --config-file config/unicorn_test.rb --env test --daemonize

and ran the tests with

npx playwright test e2e --workers=1

The coverage is written to the disk on shutting down the server. To be able to post it to codacy, I added to the CI configuration

kill -QUIT `cat tmp/pids/unicorn.pid`
while [ -s tmp/pids/unicorn.pid ]; do
    sleep 2
done

to give some time to SimpleCov to flush the report to the disk before attempting to send it. Initially I started with wait time of 0.2s but it was insufficient and I increased it to 2s. Depending on the machine you run and the size of the coverage report you may need to increase it further.