Why isn't behave's documented manual integration with Django working?

Question

I have a Django (1.10.2) project ("theproject") and some behave (0.4.0) features. I've been using behave-django. python manage.py behave works. However, PyCharm (which uses the behave executable rather than a Django management command) doesn't know how to run my features, so I'm attempting to use behave's documented "manual integration" with Django.

My entire features/environment.py:

import os
import django
from django.test.runner import DiscoverRunner
from django.test.testcases import LiveServerTestCase
from splinter.browser import Browser

os.environ["DJANGO_SETTINGS_MODULE"] = "theproject.settings"

def before_all(context):
    django.setup()
    context.test_runner = DiscoverRunner()
    context.test_runner.setup_test_environment()
    context.old_db_config = context.test_runner.setup_databases()
    context.browser = Browser('phantomjs')

    # When we're running with PhantomJS we need to specify the window size.
    # This is a workaround for an issue where PhantomJS cannot find elements
    # by text - see: https://github.com/angular/protractor/issues/585
    if context.browser.driver_name == 'PhantomJS':
        context.browser.driver.set_window_size(1280, 1024)

def before_scenario(context, _):
    context.test_case = LiveServerTestCase
    context.test_case.setUpClass()

def after_scenario(context, _):
    context.test_case.tearDownClass()
    del context.test_case

def after_all(context):
    context.test_runner.teardown_databases(context.old_db_config)
    context.test_runner.teardown_test_environment()

    context.browser.quit()
    del context.browser

Here's INSTALLED_APPS from theproject/setting.py in case it's helpful (I removed 'behave-django' for this experiment):

INSTALLED_APPS = [
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'django_extensions',
    'oauth2_provider',
    'push_notifications',
    'raven.contrib.django.raven_compat',
    'rest_framework',
    'app1.apps.App1Config',
    'app2',
    'django.contrib.admin'  # Must follow apps for apps' models to appear in admin UI
]

When I run behave I get

Exception AppRegistryNotReady: Apps aren't loaded yet.
Traceback (most recent call last):
  File "/usr/local/bin/behave", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python2.7/site-packages/behave/__main__.py", line 109, in main
    failed = runner.run()
  File "/usr/local/lib/python2.7/site-packages/behave/runner.py", line 672, in run
    return self.run_with_paths()
  File "/usr/local/lib/python2.7/site-packages/behave/runner.py", line 678, in run_with_paths
    self.load_step_definitions()
  File "/usr/local/lib/python2.7/site-packages/behave/runner.py", line 658, in load_step_definitions
    exec_file(os.path.join(path, name), step_module_globals)
  File "/usr/local/lib/python2.7/site-packages/behave/runner.py", line 304, in exec_file
    exec(code, globals, locals)
  File "features/steps/common.py", line 5, in <module>
    from django.contrib.auth.models import User
  File "/usr/local/lib/python2.7/site-packages/django/contrib/auth/models.py", line 4, in <module>
    from django.contrib.auth.base_user import AbstractBaseUser, BaseUserManager
  File "/usr/local/lib/python2.7/site-packages/django/contrib/auth/base_user.py", line 52, in <module>
    class AbstractBaseUser(models.Model):
  File "/usr/local/lib/python2.7/site-packages/django/db/models/base.py", line 105, in __new__
    app_config = apps.get_containing_app_config(module)
  File "/usr/local/lib/python2.7/site-packages/django/apps/registry.py", line 237, in get_containing_app_config
    self.check_apps_ready()
  File "/usr/local/lib/python2.7/site-packages/django/apps/registry.py", line 124, in check_apps_ready
    raise AppRegistryNotReady("Apps aren't loaded yet.")
django.core.exceptions.AppRegistryNotReady: Apps aren't loaded yet.

How can this way of integrating django and behave be made to work?

Something I tried that didn't work (or not completely): I moved django.setup() to the top level of environment.py, right after setting DJANGO_SETTINGS_MODULE. That fixes AppRegistryNotReady, but many scenarios fail with

IntegrityError: duplicate key value violates unique constraint "auth_user_username_key"
DETAIL:  Key (username)=(username) already exists.

Under behave-django a transaction was started before each scenario and rolled back afterwards; that seems not to be happening now. LiveServerTestCase extends TransactionTestCase, so I'm puzzled.

Answer 1

Your database changes won't be rolled back between scenarios (hence the IntegrityError). The database is only torn down in after_all(). Try moving all code into the before_scenario and after_scenario functions (see related docs).

The execution time of your tests will increase, but the tests will be isolated. And as a quick fix this makes them pass for now at least. Hints for cleaner solutions in the comments and alternative answers.

Answer 2

I am pretty sure that the error is actually at

  File "features/steps/common.py", line 5, in <module>

Which tries to load the User before loading the apps. Try to move import inside step_impl.

Answer 3

You seem to have misunderstood what TransactionTestCase (and LiveServerTestCase) does with relation to transactions. It's not a test case that uses transactions to run your tests and roll back changes at the end (this is what Django's "regular" TestCase does). It's a test case that does not use transactions to you can test your own transactions:

Django’s TestCase class is a more commonly used subclass of TransactionTestCase that makes use of database transaction facilities to speed up the process of resetting the database to a known state at the beginning of each test. A consequence of this, however, is that some database behaviors cannot be tested within a Django TestCase class. For instance, you cannot test that a block of code is executing within a transaction, as is required when using select_for_update(). In those cases, you should use TransactionTestCase.

(source)

TransactionTestCase still resets the database by flushing all tables, firing the post-migrate signal and reinstalling fixtures, but it runs under the assumption that __call__ is called on the test case. It seems that behave is calling run() directly, so it skips the setup/teardown of database data for each test.

You can probably fix this by creating your own subclass that calls _pre_setup() and _post_teardown() in setUp() and tearDown(). The only caveat is that every subclass that overrides setUp() or tearDown() must call super(), or _pre_setup()/_post_teardown() won't be called.

class BehaveLiveServerTestCase(LiveServerTestCase):
    def setUp(self):
        self._pre_setup()

    def tearDown(self):
        self._post_teardown()

Answer 4

Here's what I ended up with. It allows my features to run under the behave command without behave-django, and thus to run in PyCharm run configurations, without changing the features or steps. BehaviorDrivenTestCase and calling context.test() in before_scenario are copied from behave-django internals. I'm omitting browser setup and teardown and some other bits not relevant to this question.

import os
import django
from django.contrib.staticfiles.testing import StaticLiveServerTestCase
from django.core import management
from django.shortcuts import resolve_url
from django.test.runner import DiscoverRunner

os.environ["DJANGO_SETTINGS_MODULE"] = "api.settings"
django.setup()

def before_all(context):
    context.test_runner = DiscoverRunner()
    context.test_runner.setup_test_environment()
    context.old_db_config = context.test_runner.setup_databases()

def before_scenario(context, _):
    context.test = BehaviorDrivenTestCase()
    context.test.setUpClass()
    context.test() # this starts a transaction

    context.base_url = context.test.live_server_url
    def get_url(to=None, *args, **kwargs):
        return context.base_url + (
            resolve_url(to, *args, **kwargs) if to else '')
    context.get_url = get_url

class BehaviorDrivenTestCase(StaticLiveServerTestCase):
    """
    Test case attached to the context during behave execution

    This test case prevents the regular tests from running.
    """

    def runTest(*args, **kwargs):
        pass

def after_scenario(context, _):
    context.test.tearDownClass()
    del context.test

def after_all(context):
    context.test_runner.teardown_databases(context.old_db_config)
    context.test_runner.teardown_test_environment()

Isolating scenarios in transactions won't work for scenarios that exercise a UI that uses async Javascript. Removing context.test() and adding management.call_command('flush', verbosity=0, interactive=False) at the beginning of before_scenario also works, takes about the same amount of time, and should work better with async Javascript.