Validate classes (hydra-core lists) with pydantic

Question

1. Context

How to validate a specific class in pydantic?

I'm using pydantic to validate yaml list parameters parsed by hydra, to later be passed to modeling routines. The problem is that the hydra dictionary contains not a list of values, but a class that contains those values. How can I validate those params?

2. Example

In the following example, there are 2 files:

• cfg.yaml containing the parameters to be validated
• main.py containing the instructions to load and validate cfg.yaml

2.1 Config File cfg.yaml

params_list:
  - 10
  - 0
  - 20

2.2 Parser/Validator file main.py

import hydra
import pydantic
from omegaconf import DictConfig, OmegaConf
from typing import List

class Test(pydantic.BaseModel):
    params_list: List[int]

@hydra.main(config_path=".", config_name="cfg.yaml")
def go(cfg: DictConfig):
    parsed_cfg = Test(**cfg)
    print(parsed_cfg)

if __name__ == "__main__":
    go()

3. Problem

When executing python3 main.py the following error arises

value is not a valid list (type=type_error.list)

That is because hydra has a specific class for dealing with lists, called omegaconf.listconfig.ListConfig, which can be checked by adding

print(type(cfg['params_list']))

right after the go() function definition.

4. Guidance

I know that I probably have to tell pydantic to validate this specific thing, but I just don't know exactly how.

• Here are provided some tips, but it seems to much for the task I guess.
• Another idea is to create a generic type for the data attribute (like params_list: Generic) and then use the validator decorator to transform it to a list, something along the lines:

class ParamsList(pydantic.BaseModel):
  params_list: ???????? #i don't know that to do here
  @p.validator("params_list")
  @classmethod
    def validate_path(cls, v) -> None:
        """validate if it's a list"""
        if type(list(v)) != list:
            raise TypeError("It's not a list. Make it become a list")
        return list(v)

Help!: Any idea on how to solve it?

How to Recreate Example

1. In a folder add files described in sections 2.1 and 2.2.
2. Also create a requirements.txt file with the packages pydantic and hydra-core
3. After creating and activating the env, run python3 main.py

Answer 1

Pydantic doesn't accept the DictConfig format. When you try to parse the hydra config with the pydantic model, you must first convert the DictConfig to a native Python Dict. You do this with OmegaConf.to_object(cfg).

I assume you use Python 3.10 or higher. Note the use of version_base="1.2" to get the latest hydra version.

This should work:

import hydra
import pydantic
from omegaconf import DictConfig, OmegaConf

class Test(pydantic.BaseModel):
    params_list: list[int]


@hydra.main(config_path=".", config_name="cfg.yaml", version_base="1.2")
def go(cfg: DictConfig):
    print(cfg)
    d_cfg = OmegaConf.to_object(cfg)
    parsed_cfg = Test(**d_cfg)
    print(parsed_cfg)


if __name__ == "__main__":
    go()