Database Cleaner Not Cleaning One Cucumber Scenario

Question

So my team is driving out our rails app (a survey and lite electronic record system) using Cucumber, Rspec and all the gems necessary to use these testing frameworks. I am in the process of setting up a Jenkins CI server and wanted to standardize our databases across our testing, development, and staging environments (we ended up choosing MySQL).

Upon switching the testing environment from SQlite to MySQL we discovered a couple of caching-related testing bugs that we resolved by using relative ids instead of hard-coded ones. example:

describe "#instance_method" do
  before(:each) do
  @survey = FactoryGirl.create(:survey)
  @question_1 = FactoryGirl.create(:question)
  @question_2 = FactoryGirl.create(:question)
  ....
  end
  context "question has no requirements" do
    it "should match" do
      # below breaks under MySQL (and postgres), but not SQlite
      expect { @survey.present_question?(2) }.to be_true
      # always works
      expect { @survey.present_question?(@question_2.id) }.to be_true
    end
  end
end

After resolving this one failing spec, I addressed some unrelated ajax issues that have forever plaguing our test suite. Thinking that I finally masted the art of testing, I confidently ran rake. I was met with this unfriendly sight:

Now of course cucumber features/presenting_default_questions.feature rains green when ran in isolation.

Failing Scenario:

@javascript
Feature: Dynamic Presentation of Questions
  In order to only answer questions that are relevant considering previously answered      questions
  As a patient
  I want to not be presented questions that illogical given my previously answered question on the survey

  Scenario: Answering a question that does not depend on any other questions
    Given I have the following questions:
      | prompt             | datatype | options | parent_id | requirement |
      | Do you like cars?  | bool     |         |           |             |
      | Do you like fruit? | bool     |         |           |             |
    When I visit the patient sign in page
    And I fill out the form with the name "Jim Dog", date of birth "1978-03-30", and gender "male"
    And I accept the waiver
    Then I should see the question "Do you like cars"
    When I respond to the boolean question with "Yes"
    Then I should see the question "Do you like fruit?"
    When I respond to the boolean question with "No"
    Given I wait for the ajax request to finish
    Then I should be on the results page

Relevant step:

Then(/^I should be on the results page$/) do
  # fails under ``rake`` passes when run in isolation
  current_path.should == results_survey_path(1)
end

Then(/^I should be on the results page$/) do
  # passes in isolation and under ``rake``
  current_path.should == results_survey_path(Survey.last.id)
end

Besides from using Capybara.javascript_driver = :webkit, the cucumber / database cleaner config is unmodified from rails g cucumber:install.

It seems like both the failing rspec and cucumber tests suffer from the same sort of indexing problem. While the solution proposed above works, it's super janky and begs the question as to why a simple absolute index doesn't work (after all the database is cleaned between each scenario and feature). Is there something up with my tests? With database_cleaner?

Please let me know if more code would be helpful!

Relavent gem versions:

• activemodel (3.2.14)
• cucumber (1.3.6)
• cucumber-rails (1.3.1)
• database_cleaner (1.0.1)
• capybara (2.1.0)
• capybara-webkit (1.0.0)
• mysql2 (0.3.13)
• rspec (2.13.0)

Answer 1

The problem seems to be that you are missing the database_cleaner code for your Cucumber scenarios.

You mentioned that you have the database_cleaner code for your RSpec scenarios, but it seems that you're missing something like this for Cucumber's env.rb:

begin
  require 'database_cleaner'
  require 'database_cleaner/cucumber'

  DatabaseCleaner.strategy = :truncation
rescue NameError
  raise "You need to add database_cleaner to your Gemfile (in the :test group) if you wish to use it."
end

Around do |scenario, block|
  DatabaseCleaner.cleaning(&block)
end

Do you have this code in there?