Automake, generated source files and VPATH builds

Question

I'm doing VPATH builds with automake. I'm now also using generated source, with SWIG. I've got rules in Makefile.am like:

dist_noinst_DATA = whatever.swig

whatever.cpp: whatever.swig
    swig -c++ -php $^

Then the file gets used later:

myprogram_SOURCES = ... whatever.cpp

It works fine when $builddir == $srcdir. But when doing VPATH builds (e.g. mkdir build; cd build; ../configure; make), I get error messages about missing whatever.cpp.

Should generated source files go to $builddir or $srcdir? (I reckon probably $builddir.)

How should dependencies and rules be specified to put generated files in the right place?

Answer 1

Simple answer

You should assume that $srcdir is a read-only, so you must not write anything there. So, your generated source-code will end up in $(builddir).

By default, autotool-generated Makefiles will only look for source-files in $srcdir, so you have to tell it to check $builddir as well. Adding the following to your Makefile.am should help:

VPATH = $(srcdir) $(builddir)

After that you might end up with a no rule to make target ... error, which you should be able to fix by updating your source-generating rule as in:

$(builddir)/whatever.cpp: whatever.swig
        # ...

A better solution

You might notice that in your current setup, the release tarball (as created by make dist) will contain the whatever.cpp file as part of your sources, since you added this file to the myprogram_SOURCES. If you don't want this (e.g. because it might mean that the build-process will really take the pregenerated file rather than generating it again), you might want to use something like the following. It uses a wrapper source-file (whatever_includer.cpp) that simply includes the generated file, and it uses -I$(builddir) to then find the generated file.

Makefile.am:

dist_noinst_DATA = whatever.swig

whatever.cpp: whatever.swig
    swig -c++ -php $^

whatever_includer.cpp: whatever.cpp

myprogram_SOURCES = ... whatever_includer.cpp
myprogram_CPPFLAGS = ... -I$(builddir)

clean-local::
    rm -f $(builddir)/whatever.cpp

whatever_includer.cpp:

#include "whatever.cpp"

Answer 2

Usually, you want to keep $srcdir readonly, so that if for instance the source is distributed unpacked on a CDROM, you can still run /.../configure from some other part of the file-system.

However if you are using SWIG to generate source code for a wrapper library, you probably want to distribute that SWIG-generated code as well so that your users do not need to install SWIG to compile your code. Then you have indeed a choice: you can decide that the SWIG-generated code should end in $builddir (it's OK: make dist will collect it there and include it in the tarball), or you could decide to output SWIG-generated code in $srcdir since it is really a source from the point of view of the distributed package. An advantage of keeping it in $srcdir is that when make distcheck attempts to build your package from a read-only source directory, it will fail on any attempt to call SWIG to regenerate the wrapper source. If you have your wrapper source in $builddir, you might not notice you have some broken rule that cause SWIG to be run on the user's host; by generating in $srcdir you ensure that SWIG is not needed by your users.

So my preference is to output SWIG wrapper sources in $srcdir. My setup for Python wrappers looks as follows:

EXTRA_DIST = spot.i
python_PYTHON = $(srcdir)/spot.py    # _PYTHON is distributed by default
pyexec_LTLIBRARIES = _spot.la

MAINTAINERCLEANFILES = $(srcdir)/spot_wrap.cxx $(srcdir)/spot.py

_spot_la_SOURCES = $(srcdir)/spot_wrap.cxx $(srcdir)/spot_wrap.h
_spot_la_LDFLAGS = -avoid-version -module
_spot_la_LIBADD = $(top_builddir)/src/libspot.la

$(srcdir)/spot_wrap.cxx: $(srcdir)/spot.i
        $(SWIG) -c++ -python -I$(srcdir) -I$(top_srcdir)/src $(srcdir)/spot.i

# Handle the multi-file output of SWIG.
$(srcdir)/spot.py: $(srcdir)/spot.i
        $(MAKE) $(AM_MAKEFLAGS) spot_wrap.cxx

Note that I use $(srcdir) for all targets, because of limitations of the VPATH feature on various flavors of make. My setup to deal with the multiple files output by SWIG could be improved, but as these rules are not run by users and it has never caused me any problem, I do not bother.