Modularization: Use config.yaml file from GitHub repository for module

134 Views Asked by At

I want to use an existing snakemake workflow (technology_data) as a module into another workflow (let's call it w). At the moment I manually download (whenever the workflow is setup new) the GitHub repository of technology_data into the desired location and have it included into w as a subworkflow.

Now I want to include technology_data as-is, i.e. including the config.yaml that is contained in the GitHub repository into workflow w as a module.

What I have in my Snakefile of workflow w for now is:

module technology_data:
    snakefile: github("PyPSA/technology-data", path="Snakefile", commit="5a10981")

use rule * from technology_data as technology_data_*

Which fails when I call one of the rules of the workflow with the following error:

> snakemake -call technology_data_compile_cost_assumptions
...
WorkflowError in line 2 of https://github.com/PyPSA/technology-data/raw/5a10981/Snakefile:
Workflow defines configfile config.yaml but it is not present or accessible

I don't want to download the config.yaml from the repository but I want Snakemake to handle it automatically with the rest of the remote module.

I tried in the module definition to use:

  • configfile: github("PyPSA/technology-data", path="config.yaml", commit="5a10981") which did not work (configfile is not understood by module)
  • config: snakemake.io.load_configfile(github("PyPSA/technology-data", path="config.yaml", commit="5a10981")) which doesn't work either

Is there a good solution to have snakemake use the remote config.yaml automatically?

0

There are 0 best solutions below