Run Python integration tests inside pBuilder environment

Question

I have build a Debian package out of a Python project which depends on a bunch of Debian packages to work. The Python project also contains unit and integration tests (using py.test).

Since my project depends on some Debian packages, I need to have these packages installed inside the test environment and they should verify that my module works correctly together with these packages (I want to make sure that my module does not break on updates of these libs).

I want to put the configuration and execution of these test environments into a Makefile. These tests should be runnable both on a developer workstation and on an Jenkins-powered Integration Server.

My current approach for a complete integrationtest of my package is to:

1. Create a pBuilder environment
2. Install dependencies from Debian-controlfile into pBuilder env
3. Inside this pBuilder-environment launch py.test, also using virtualenv / tox to test different Python interpreter versions
4. collect test-results from test environment (py.test can generate nice JUnit-like test output which Jenkins can publish easily)

Unfortunately I could not find any hints how to realize this stuff - especially how to do 3). All documentation I could find was just about building the package, not about running tests.

Can anyone provide me some hints how I can build up such a test setup? Also, I want to know if it makes sense at all to do what I aim to do. How do others test integration of (Python) modules with other Debian packages?

Answer 1

I assume that you want to run the test-suite while building the packaging.

1) Add a check target to your package's Makefile, which runs your actual testsuite including all the virtualenv/tox magic; e.g.:

.PHONY: check
check:
    ./runtests.sh

with the runtests.sh script running the interpreter checks and then launching py.test

2) Configure your debian/rules to actually run the tests

2a) If you use CDBS, add the following line at the beginning of your debian/rules

DEB_MAKE_CHECK_TARGET = check

2b) If you use debhelper, then the dh_auto_test script should automagically run the tests for you (provided there is a check target in your Makefile)

2c) If you have a custom debian/rules, run the "check" target after a successfull build

Answer 2

How do others test integration of (Python) modules with other Debian packages?

One option is to provision a full new system to run your tests in, i.e. on a virtual machine or a cheap cloud server.

If you are deadset on pBuilder, you can invoke an arbitrary script with pbuilder execute

Some other helpful tips at http://packaging.ubuntu.com/html/chroots.html