https://github.com/fidelity/spock

[removed]

This is much lighter but it's a pure-python config flow manager I made where you can chain experiment classes by adding them (xp1()+xp2() ) https://github.com/sileod/xpflow

Looks interesting, a bit more lightweight than hydra. But also misses a lot of cool features like composing multiple yaml configs

Most of the cool stuff is buried in the docs under advanced features :-)

https://fidelity.github.io/spock/advanced_features/Composition

(full transparency I'm the author/maintainer/core-developer. I know the docs need a re-org to surface more of the useful features)

This being my favorite hidden one:

https://fidelity.github.io/spock/advanced_features/Evolve#maintaining-cli-and-python-api-configuration-parity

go closer to the metal and use omegaconf directly.

second favorite:

https://fidelity.github.io/spock/advanced_features/Post-Hooks

Basically lets you do any validation necessary on your configs. Spock provides some basics (greater than, within bounds, etc) but it's totally up to the user via any simple asserts or validation functions a user wants to write.

As the developer of Spock (posted in another comment) -- OmegaConf is also an awesome choice and super useful. I'd suggest checking it out too!

You can go even closer to metal and use the attrs library as well (https://www.attrs.org/en/stable/)

I'm also using Omega conf. It's a great lib: full of feature, not opiniated, perfect for Mlops !

> https://github.com/fidelity/spock

This looks quite promising, and I like the Post hooks you linked below, but I do not see any way of running a series of experiment in a non-combinatoric way? There is Optuna api (though I can't tell whether early pruning is supported in this?), but I don't see any way of grouping parameters for a set of experiments.

That is certainly an option that I was considering, but then I would have to make my own job planner / multirunner, (which I actually already have done for my current project, but this whole refactoring was to try and move away from my own custom functions and try to use some more standardized methods)

i misunderstood, i thought you were looking for an alternative config component. if you're looking for an atlernative for managing hyperparameter search jobs, consider https://docs.ray.io/en/latest/tune/index.html . I think hydra actually might even integrate with ray.

Thanks, looks like your library isn't far behind hydra in terms of functionality. Will definitely look into it more closely the next time I set up a project.

What would you say are the pros and cons between hydra and spock?

dataclasses from python standard library + dacite.from_dict

I have a lightweight package that I use that has all the main things I wanted from hydra or gin-config. It's here and it's pretty tiny in terms of lines of code: https://github.com/pseeth/argbind

Isn't just using python flexible enough for you?

Mainly that Spock is much lighter weight and really focuses on just configuration management and stateful ness. Hydra has all these crazy bells and whistles (Ray integration etc) that could be useful for certain things but kinda starts meandering from the original purpose of configuration management imo. Hydra is great and if it works for you then use it. We built Spock internally when I was at Fidelity because Hydra didn’t exist… just so happens that FB/Meta was doing the same thing at the same time so both libraries end up covering a very similar usage space

Thanks, I basically use only the config part of hydra and am regularly annoyed that its so clunky, so spock might be a good alternative. Gonna check it out :)

Pyrallis quite nicely builds on top of that to support e.g. command line arguments.

I'm quite happy with it.