SCons mwe for a generic Fortran project

Question

I am having a hard time to understand the logic of SCons, even after reading several sources and spending quite some time coding. I have not found any reasonably understandable Fortran project using SCons related to what I want to accomplish. I also found any in-depth tutorial or explanation (the official manual doesn't really tell how things work on the backend).

My goal is to automatically compile a Fortran project with source files scattered in a directory-tree starting from some root directory.

An outline of the code:

1. Find all Fortran files in a dir-tree
2. Use fortdepend to generate the dependencies and discriminate programs and modules
3. Tell SCons the objects, dependencies and programs/targets

The dependencies among the sources are resolved with fortdepend. But I'm stuck with telling SCons the dependencies. Since SCons has explicit Fortran support, my solution feels way to complicated for such a generic use case.

Here is my SConstruct file:

import os
import fortdepend as fd
from  SCons.Environment import Environment

fexts = ['.f', '.for', '.f90', '.f95', '.f03', '.f08']

def find_fortran_files(root_dir, fortran_extensions, abspath=False):
  """Find all Fortran source files in src directory and its subdirectories"""
  source_files = []
  for dir, _, files in os.walk(root_dir):
    for file in files:
      if any(file.endswith(ext) for ext in fortran_extensions):
        source_files.append( os.path.join(dir, file) if abspath else os.path.join(dir.replace(root_dir, '.'), file))
  return source_files

def fortran_source_to_object(source, fortran_extensions):
  """replace the fortran ext with .o"""
  for ext in fortran_extensions:
    if source.endswith(ext):
      return source.replace(ext, '.o')
s2o = lambda src: fortran_source_to_object(str(src), fexts)
ss2os = lambda srcs: [fortran_source_to_object(str(src), fexts) for src in srcs]


def generate_fortran_dependencies(source_files, fortran_extensions, **kwargs):
  """ use fortdepend to get dependencies """
  fproj = fd.FortranProject(files=source_files, **kwargs)
  source_deps, prog_deps = {}, {}
  for key, val in fproj.depends_by_module.items():
    _key = key.source_file.filename
    _val = [v.source_file.filename for v in val] 
    if key.unit_type=='module':
      source_deps[_key] = _val
    elif key.unit_type=='program':
      prog_deps[_key] = _val
  return source_deps, prog_deps

env = Environment(tools=['default', 'gfortran'], F90='gfortran', LINK='gfortran', LINKFLAGS='', F90FLAGS='')

all_files = find_fortran_files(os.getcwd(), fortran_extensions=fexts, abspath=False)
print('- all files found with fortran ext: ', all_files)

source_deps, prog_deps = generate_fortran_dependencies(all_files, fexts)
source_files = list(source_deps.keys())
prog_files = list(prog_deps.keys())
print('- all source deps (no progs)       ', source_deps)
print('- all program deps found           ', prog_deps)
print('- all source files (no progs)      ', source_files)
print('- all prog files                   ', prog_files)

objects = []
all_deps = {**source_deps, **prog_deps}
for src in all_deps:
    obj_path = os.path.splitext(src)[0] + '.o'
    object = env.Object(obj_path, src)
    print(f'-- tell SCons {obj_path} also depends on {all_deps[src]}')
    env.Depends(target=object, dependency = all_deps[src])
    objects.append(object)
print('- all SCons objects', [str(o) for o in objects])

for prog in prog_deps:
  prog_name = os.path.splitext(os.path.basename(prog))[0]
  print(f'-- tell SCons {prog_name} also depends on {ss2os(prog_deps[prog])}')
  env.Depends(target=prog_name, dependency = ss2os(prog_deps[prog]))

# Filter out .mod files from the objects list
objects_for_linking = [str(o[0]) for o in objects] # get only the first entry which is the object file 
print('- all SCons objects for linking', objects_for_linking)

prog_to_make = 'main'
env.Program(target=prog_to_make, source=objects_for_linking)

In the below repo https://github.com/alexksr/SconsMWE.git are two python scripts to set up a root directory with example Fortran code. init1.py one has dependencies, which are satisfied when compiling in alphabetical order (easy, non-use case). The second init2.py has dependencies, that must explicitly be taken into account (module0 depends on module1 and module2). For the ladder my code does not work, but results in:

scons: Reading SConscript files ...
- all files found with fortran ext:  ['./src/module0.f90', './src/main.f90', './src/module1.f90', './src/module2.f90', './src/utils/util_module.f90']
- all source deps (no progs)        {'./src/module0.f90': ['./src/module1.f90', './src/module2.f90'], './src/module1.f90': [], './src/module2.f90': [], './src/utils/util_module.f90': []}
- all program deps found            {'./src/main.f90': ['./src/module0.f90', './src/module1.f90', './src/module2.f90', './src/utils/util_module.f90']}
- all source files (no progs)       ['./src/module0.f90', './src/module1.f90', './src/module2.f90', './src/utils/util_module.f90']
- all prog files                    ['./src/main.f90']
-- tell SCons ./src/module0.o also depends on ['./src/module1.f90', './src/module2.f90']
-- tell SCons ./src/module1.o also depends on []
-- tell SCons ./src/module2.o also depends on []
-- tell SCons ./src/utils/util_module.o also depends on []
-- tell SCons ./src/main.o also depends on ['./src/module0.f90', './src/module1.f90', './src/module2.f90', './src/utils/util_module.f90']
- all SCons objects ["['src/module0.o', 'module0.mod']", "['src/module1.o', 'module1.mod']", "['src/module2.o', 'module2.mod']", "['src/utils/util_module.o', 'util_module.mod']", "['src/main.o']"]
-- tell SCons main also depends on ['./src/module0.o', './src/module1.o', './src/module2.o', './src/utils/util_module.o']
- all SCons objects for linking ['src/module0.o', 'src/module1.o', 'src/module2.o', 'src/utils/util_module.o', 'src/main.o']
scons: done reading SConscript files.
scons: Building targets ...
gfortran -o src/module0.o -c src/module0.f90
src/module0.f90:3:8:

     use module1
        1
Fatal Error: Can't open module file 'module1.mod' for reading at (1): No such file or directory
compilation terminated.
scons: *** [src/module0.o] Error 1
scons: building terminated because of errors.

Any help is appreciated, before I abandon SCons and go back to make.

UPDATE Issues solved in this discussion. Some Fortran-related issues with SCons make this case a special one and SCons does not sticking to the full dependencies.

Answer 1

Here's the simplest way to build your test tree:

env = Environment(tools=['default', 'gfortran'], F90='gfortran', 
                  LINK='gfortran', LINKFLAGS='', FORTRANMODDIR='fortran_mods')

sources = env.Glob('src/*.f90') + env.Glob('src/utils/*.f90')

env.Program('main', sources)

No need for fortdepend. Note it was necessary to set FORTRANMODDIR, and to not clear F90FLAGS