Sneak peek: pydantic parsing & validation for Hydra apps

[rsokl]

Edit: check it out! https://mit-ll-responsible-ai.github.io/hydra-zen/changes.html#rc1-2024-04-01

tl;dr: hydra_zen.instantiate is going to have a custom validation-wrapper argument. And hydra-zen will also provide a pydantic wrapper that will enable pydantic's rich parsing capabilities for converting/validating arguments on all config-instantiation calls. This is very powerful. It is basically Hydra's CLI x pydantic's parsing & validation.

Really excited about this. I figured out how to add a custom validation layer to instantiate.

Typically if you have cfg = builds(target, **kwargs), calling instantiate(cfg) ultimately calls target(**kwargs). Now you will be able to call instantiate(cfg, target_wrapper=w), and it will call w(target)(**kwargs). Critically, this works recursively -- applying w within all recursive instantiate calls, and it doesn't change any of your configs.

Who cares?

Well, if you set target_wrapper=pydantic.validate_call, this means that you get pydantic's parsing and validation logic across all instantiated configs!

So what?

Now you get much more sophisticated runtime type-checking. E.g. support for Literal, which Hydra doesn't support.

def func(x: Literal[1, 2]):
    return x

cfg = builds(func, x=1)
instantiate(cfg)  # returns 1

bad_cfg = builds(func, x=3)
instantiate(bad_cfg)  # raises ValidationError: Input should be 1 or 2

You also get annotation-based parsing & conversion. E.g., Hydra is only ever able to produce list-type sequences from the CLI, but now we can be more sophisticated:

Given:

def func(x: tuple[int, int]):
    return x

cfg = builds(func, x=(1, 2))

Before:

>>> instantiate(cfg)
[1, 2]

After:

>>> instantiate(cfg)  # pydantic converts to tuple based on annotation
(1, 2)

You also get to use pydantic types in your annotations, and now you can validate against them. E.g.

from pydantic import AnyHttpUrl

def needs_url(x: AnyHttpUrl):
    return x

>>> cfg = builds(needs_url, x="http://www.google.com")
>>> instantiate(cfg)
Url('http://www.google.com/')

>>> instantiate(cfg(x="bad"))
ValidationError: 1 validation error for needs_url

This is great. Hydra's type checking happens at the wrong layer of abstraction and is very bare bones. This moves the checking logic to the right place -- the point of instantiation -- and opts for pydantic's much more complete support for type annotations.

And obviously you don't have to use pydantic. You can use whatever validation/conversion logic you want here.

[rsokl]

#666