I am trying to do a basic hyperparameter tuning. By default Hydra creates a permutation of each hyperparameter.
hydra:
mode: MULTIRUN
sweeper:
params:
+n: 5,10,15
+a_lower: 0.5, 0.7, 0.9
Here it will run the same python file 3*3=9 times with all the permutations of n and a_lower.
In my case, some hyperparameters are related to other. For example:
hydra:
mode: MULTIRUN
sweeper:
params:
+n: 5,10,15
+a_lower: 0.5, 0.7, 0.9
+a_upper: 1.1, 1.15
+a: 0.91, 1.05, 1.09
+pde_coeff: 0.1, 1.0
[a_lower, a_upper, a] should stay together, not permutate. This should not be allowed: [a_lower, a_upper, a] = [0.5, 1.15, 1.05].
I tried using a list or a dictionary but the error is always the same. For example, I used this dictionary to prevent the permutations of [a_lower, a_upper, a]:
+matrix:['case 1':[0.5, 1.1, 1.05],'case 2':[0.7, 1.15, 1.09], 'case 3':[0.9, 1.15, 1.0]]
The error is
[s.1915438@sl1(sunbird) 6. inverse + logger]$ python 1.single_notebook.py
Traceback (most recent call last):
File "/scratch/s.1915438/env/modulus/lib/python3.9/site-packages/hydra/_internal/utils.py", line 213, in run_and_report
return func()
File "/scratch/s.1915438/env/modulus/lib/python3.9/site-packages/hydra/_internal/utils.py", line 453, in <lambda>
lambda: hydra.run(
File "/scratch/s.1915438/env/modulus/lib/python3.9/site-packages/hydra/_internal/hydra.py", line 105, in run
cfg = self.compose_config(
File "/scratch/s.1915438/env/modulus/lib/python3.9/site-packages/hydra/_internal/hydra.py", line 594, in compose_config
cfg = self.config_loader.load_configuration(
File "/scratch/s.1915438/env/modulus/lib/python3.9/site-packages/hydra/_internal/config_loader_impl.py", line 141, in load_configuration
return self._load_configuration_impl(
File "/scratch/s.1915438/env/modulus/lib/python3.9/site-packages/hydra/_internal/config_loader_impl.py", line 235, in _load_configuration_impl
self._process_config_searchpath(config_name, parsed_overrides, caching_repo)
File "/scratch/s.1915438/env/modulus/lib/python3.9/site-packages/hydra/_internal/config_loader_impl.py", line 158, in _process_config_searchpath
loaded = repo.load_config(config_path=config_name)
File "/scratch/s.1915438/env/modulus/lib/python3.9/site-packages/hydra/_internal/config_repository.py", line 349, in load_config
ret = self.delegate.load_config(config_path=config_path)
File "/scratch/s.1915438/env/modulus/lib/python3.9/site-packages/hydra/_internal/config_repository.py", line 92, in load_config
ret = source.load_config(config_path=config_path)
File "/scratch/s.1915438/env/modulus/lib/python3.9/site-packages/hydra/_internal/core_plugins/file_config_source.py", line 31, in load_config
cfg = OmegaConf.load(f)
File "/scratch/s.1915438/env/modulus/lib/python3.9/site-packages/omegaconf/omegaconf.py", line 190, in load
obj = yaml.load(file_, Loader=get_yaml_loader())
File "/scratch/s.1915438/env/modulus/lib/python3.9/site-packages/yaml/__init__.py", line 81, in load
return loader.get_single_data()
File "/scratch/s.1915438/env/modulus/lib/python3.9/site-packages/yaml/constructor.py", line 49, in get_single_data
node = self.get_single_node()
File "/scratch/s.1915438/env/modulus/lib/python3.9/site-packages/yaml/composer.py", line 36, in get_single_node
document = self.compose_document()
File "/scratch/s.1915438/env/modulus/lib/python3.9/site-packages/yaml/composer.py", line 55, in compose_document
node = self.compose_node(None, None)
File "/scratch/s.1915438/env/modulus/lib/python3.9/site-packages/yaml/composer.py", line 84, in compose_node
node = self.compose_mapping_node(anchor)
File "/scratch/s.1915438/env/modulus/lib/python3.9/site-packages/yaml/composer.py", line 133, in compose_mapping_node
item_value = self.compose_node(node, item_key)
File "/scratch/s.1915438/env/modulus/lib/python3.9/site-packages/yaml/composer.py", line 84, in compose_node
node = self.compose_mapping_node(anchor)
File "/scratch/s.1915438/env/modulus/lib/python3.9/site-packages/yaml/composer.py", line 133, in compose_mapping_node
item_value = self.compose_node(node, item_key)
File "/scratch/s.1915438/env/modulus/lib/python3.9/site-packages/yaml/composer.py", line 84, in compose_node
node = self.compose_mapping_node(anchor)
File "/scratch/s.1915438/env/modulus/lib/python3.9/site-packages/yaml/composer.py", line 133, in compose_mapping_node
item_value = self.compose_node(node, item_key)
File "/scratch/s.1915438/env/modulus/lib/python3.9/site-packages/yaml/composer.py", line 84, in compose_node
node = self.compose_mapping_node(anchor)
File "/scratch/s.1915438/env/modulus/lib/python3.9/site-packages/yaml/composer.py", line 127, in compose_mapping_node
while not self.check_event(MappingEndEvent):
File "/scratch/s.1915438/env/modulus/lib/python3.9/site-packages/yaml/parser.py", line 98, in check_event
self.current_event = self.state()
File "/scratch/s.1915438/env/modulus/lib/python3.9/site-packages/yaml/parser.py", line 428, in parse_block_mapping_key
if self.check_token(KeyToken):
File "/scratch/s.1915438/env/modulus/lib/python3.9/site-packages/yaml/scanner.py", line 116, in check_token
self.fetch_more_tokens()
File "/scratch/s.1915438/env/modulus/lib/python3.9/site-packages/yaml/scanner.py", line 173, in fetch_more_tokens
return self.fetch_stream_end()
File "/scratch/s.1915438/env/modulus/lib/python3.9/site-packages/yaml/scanner.py", line 377, in fetch_stream_end
self.remove_possible_simple_key()
File "/scratch/s.1915438/env/modulus/lib/python3.9/site-packages/yaml/scanner.py", line 318, in remove_possible_simple_key
raise ScannerError("while scanning a simple key", key.mark,
yaml.scanner.ScannerError: while scanning a simple key
in "/scratch/s.1915438/2. inverse+forward/6. inverse + logger/saved_data/conf.yaml", line 21, column 11
could not find expected ':'
in "/scratch/s.1915438/2. inverse+forward/6. inverse + logger/saved_data/conf.yaml", line 21, column 45
[s.1915438@sl1(sunbird) 6. inverse + logger]$