Unable to test Python project due to import errors

Question

I have the following directory structure in my current project:

Project/
    project/
        __init__.py
        classifier.py
        util.py
        lib/
            __init__.py
           scraper.py
        tests/
           test_classifier.py
           test_util.py

I'm trying to run tests in the tests/ directory from within the Project/ directory, but am currently unable to do so. The reason is that the first line of each of my testing Python files is the following:

from project import Event, EventClassifier

and so I'm unable to directory call the testing files with something like the following:

python project/tests/test_util.py

Does anyone have a solution for this? I would imagine this is a pretty common problem, in that the Python convention is to include your testing directory inside of your main package.

Answer 1

I use a setuptools-based setup.py for my project, and thanks to python setup.py develop I can "install" e.g. inside a virtualenv without a copy of the sources, but an egg-link to my project. Thus imports work, and changes while developing take effect immediately.

I would challenge your assertion though that testing is inside your main package. I place it outside, at project-level. This allows for simpler packaging, because my deployed package shouldn't contain the tests.

The following setup.py illustrates the concept:

  import os
  import sys
  from setuptools import setup, find_packages

  test_requirements = ["nose", "coverage"]

  # THIS IS JUST FOR ILLUSTRATING THE IDEA
  base = os.path.dirname(__file__)

  project = os.path.join(base, "project")
  tests = os.path.join(base, "tests")

  if not os.path.exists(project):
      os.mkdir(project)
      with open(os.path.join(project, "__init__.py"), "wb") as outf:
          outf.write("""
  def magic():
      print "It's magic!"
  """)
      os.mkdir(tests)
      with open(os.path.join(tests, "project_tests.py"), "wb") as outf:
          outf.write("""
  import unittest
  import project

  class ProjectTests(unittest.TestCase):

       def test_magic(self):
           project.magic()
  """)

  # END: THIS IS JUST FOR ILLUSTRATING THE IDEA



  setup(
      name="test.project",
      version="1.0",
      description="Simple test-project",
      author="Diez B. Roggisch",
      author_email="[email protected]",
      license="MIT",
      install_requires=[
          "requests",
          ],
      extras_require=dict(
          test=test_requirements,
          ),
      packages=find_packages(exclude=['ez_setup', 'tests']),
      zip_safe=True,
      entry_points={
          "console_scripts" : [
              "test-commandline = project:magic",
              ],
          },
  )

Create a virtualenv and activate it, then inside the project, run python setup.py develop. This should set up a small sample-project (obviously you need to remove that part), create a console-script and a test-case you can run using nosetestes tests -s.