Makefile handling auto-generated source and Makefile

Question

I have a problem with a Makefile that's using an auto-generated Makefile to compile .o files from auto-generated .c and .s (assembler) source files.

The context is embedded programming for STM32 microcontrollers, where a Makefile and hardware initialization source code is generated from a hardware configuration file (.ioc file).

Suppose, we have the following directory structure:

.
├── archive
│   ├── a
│   │   └── a.c
│   ├── b
│   │   └── b.s
│   └── Makefile
├── Makefile
├── source
│   └── generate.sh

The files archive/a/a.c and archive/b/b.s may be empty here. My main ./Makefile looks like this:

.PHONY: default all clean generate objects

BUILD_DIR := build
SOURCE_DIR := source
OBJECTS := build/source/a/a.o build/source/b/b.o

default: all

all: objects

objects: $(OBJECTS)

$(BUILD_DIR)/%.o: %.c $(SOURCE_DIR)/Makefile
    mkdir -p $(shell dirname $@)
    $(MAKE) -C $(SOURCE_DIR) BUILD_DIR=../$(shell dirname $@) VPATH=../$(shell dirname $<) ../$@

$(BUILD_DIR)/%.o: %.s $(SOURCE_DIR)/Makefile
    mkdir -p $(shell dirname $@)
    $(MAKE) -C $(SOURCE_DIR) BUILD_DIR=../$(shell dirname $@) VPATH=../$(shell dirname $<) ../$@

$(SOURCE_DIR)/%.c :: generate

$(SOURCE_DIR)/%.s :: generate

$(SOURCE_DIR)/Makefile :: generate

generate: $(SOURCE_DIR)/generate.sh
    cd $(SOURCE_DIR) ; bash generate.sh

clean:
    $(RM) -r build $(SOURCE_DIR)/Makefile $(SOURCE_DIR)/a $(SOURCE_DIR)/b

The source/generate.sh looks like this:

cp -r ../archive/. ./

In reality, this process much more complicated and takes a few minutes.

Finally, the archive/Makefile looks like this:

BUILD_DIR := build

$(BUILD_DIR)/%.o: %.c
    cp $^ $@

$(BUILD_DIR)/%.o: %.s
    cp $^ $@

In reality, we use gcc instead of a simple cp to compile the C and ASM source files.

The auto-generated Makefile uses VPATH to find the required input file and is actually meant to compile everything in a flat build folder (which I don't want).

I have no control over the contents of the auto-generated Makefile! The source/generate.sh should be treated as a black-box that I have no influence on.

Therefore, the superordinate Makefile has to slightly abuse the auto-generated Makefile by overwriting the BUILD_DIR and VPATH accordingly.

Now, if I first run make generate and then, e.g. make build/source/a/a.o, everything works as planned:

First, the source directory is populated with the auto-generated stuff, and then build/source/a/a.o is "compiled" from source/a/a.c.

However, if I don't run make generate first, I get:

$ make build/source/a/a.o
make: *** No rule to make target 'build/source/a/a.o'.  Stop.

My assumption was that the recipes would have been resolved in the following manner:

$(BUILD_DIR)/%.o: $(SOURCE_DIR)/%.c $(SOURCE_DIR)/Makefile
    -> $(SOURCE_DIR)/%.c: generate
        -> generate: $(SOURCE_DIR)/generate.sh
    -> $(SOURCE_DIR)/Makefile: generate
        -> generate: $(SOURCE_DIR)/generate.sh

After a bit of trial and error, I figured that I could somewhat fix this behaviour by defining the recipes as follows:

$(SOURCE_DIR)/%.c: generate
    true

$(SOURCE_DIR)/%.s: generate
    true

$(SOURCE_DIR)/Makefile: generate
    true

This yields:

$ make build/source/a/a.o
cd source ; bash generate.sh
true
true
mkdir -p build/source/a
make -C source BUILD_DIR=../build/source/a VPATH=../source/a ../build/source/a/a.o
make[1]: Entering directory '/[...]/source'
cp ../source/a/a.c ../build/source/a/a.o
make[1]: Leaving directory '/[...]/source'
rm source/a/a.c

However, this leads to the generate step being run every time I run make build/source/a/a.o. Since the generate step is expensive in real life, this is not an option.

Also, source/a/a.c is treated as a temporary file that is meant to be deleted which I don't want.

How can I design my superordinate ./Makefile so that the generate step is being run automatically, but only if necessary?

I also need to be able to build non-auto-generated source files from the root directory.

---

A few more strange make behaviours:

---

Simply running make (i.e. make objects) with this Makefile yields

cd source ; bash generate.sh
true
true
mkdir -p build/source/a
make -C source BUILD_DIR=../build/source/a VPATH=../source/a ../build/source/a/a.o
make[1]: Entering directory '/[...]/source'
cp ../source/a/a.c ../build/source/a/a.o
make[1]: Leaving directory '/[...]/source'
true
mkdir -p build/source/b
make -C source BUILD_DIR=../build/source/b VPATH=../source/b ../build/source/b/b.o
make[1]: Entering directory '/[...]/source'
cp ../source/b/b.s ../build/source/b/b.o
make[1]: Leaving directory '/[...]/source'
rm source/a/a.c

Why does make remove source/a/a.c, but not source/b/b.c? For the record: I want neither of these files to be auto-removed.

An issue that I am sadly unable to reproduce with my example is that make basically outputs the following line at the start (adpated to the example):

make: Circular source/generate.sh.c <- generate dependency dropped.

---

Changing the recipe to the following

$(SOURCE_DIR)/%.c: generate
    true

$(SOURCE_DIR)/%.s: generate
    true

$(SOURCE_DIR)/Makefile: generate

yields:

$ make build/source/a/a.o
cd source ; bash generate.sh
true
cc    -c -o source/Makefile.o source/Makefile.c
cc1: fatal error: source/Makefile.c: No such file or directory
compilation terminated.
make: *** [<builtin>: source/Makefile.o] Error 1

Why does make think that it needs to build a Makefile.o here?