Hydra combine lists in config

3.4k Views Asked by At

Say I have the following hydra config test.yaml:

list1 : [0]
list2 : [1,2,3]

is it possible to merge list1 and list2 into a new list that contains [0,1,2,3], perhaps using variable interpolation?

Here is the hydra code:

import hydra
from omegaconf import OmegaConf
@hydra.main(config_name="test.yaml", config_path="./")
def main(cfg):
    OmegaConf.resolve(cfg)
    print(cfg)
if __name__ == "__main__":
    main()

Attempt (failed):

list1 : [0]
list2 : [1,2,3]
list3 : 
  - ${list1} 
  - ${list2}

list3 gives [0,[1,2,3]]

The reason I want this is because I have some lists of unknown length in other config files and want to merge them to create an argument for object creation. I'd prefer not to do this in the code and just rely directly on hydras object instantiation (otherwise I'll be doing list merging everywhere!).

1

There are 1 best solutions below

1
On BEST ANSWER

Turns out this is not too difficult, one can indeed rely on OmegaConfs variable interpolation with a custom resolver.

import hydra
from omegaconf import OmegaConf
# custom list merge resolver
OmegaConf.register_new_resolver("merge", lambda x, y : x + y)

@hydra.main(config_name="test.yaml", config_path="./")
def main(cfg):
    OmegaConf.resolve(cfg)
    print(cfg)

if __name__ == "__main__":
    main()

Config file test.yaml

list1 : [0]
list2 : [1,2,3]

list3 : ${merge:${list1},${list2}}

list3 is now [0,1,2,3] as desired.