You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Much of the core MultiQC code was written early on when (a) I didn't know very much Python and (b) everything had to work on Python 2.7. I also wrote a lot of the code to be super forgiving. Things like needing a list but allowing a string (and coercing it into a list) if the user wanted. Also tonnes of defaults for everything, even when it doesn't always make sense. This sounds like a nice thing to do, but actually tends to confuse things because stuff doesn't always fail when it probably should do.
Now that MultiQC supports only Python 3 we can use static typing, and specifically the Pydantic library. I think that this it's time to do a major code clean up in the core library to sort out a lot of the mess that has crept in. We can use this opportunity to be much less forgiving with the code and make it clear about what is expected (type hints will help developers a lot). By using Pydantic data models we can get a tonne of nice validation for free, without using much in the way of additional code. We can also extend the validation methods for custom classes easily.
pydantic-settings specifically (docs) has a lot of functionality that we currently do ourselves, or would like to have.
Pydantic should also be able to help with #1256 by providing access to a wide range of nice output formats. Of particular interest is Apache Parquet and JSON Schema, which I think we should be well positioned for.
The content you are editing has changed. Please copy your edits and refresh the page.
Much of the core MultiQC code was written early on when (a) I didn't know very much Python and (b) everything had to work on Python 2.7. I also wrote a lot of the code to be super forgiving. Things like needing a list but allowing a string (and coercing it into a list) if the user wanted. Also tonnes of defaults for everything, even when it doesn't always make sense. This sounds like a nice thing to do, but actually tends to confuse things because stuff doesn't always fail when it probably should do.
Now that MultiQC supports only Python 3 we can use static typing, and specifically the Pydantic library. I think that this it's time to do a major code clean up in the core library to sort out a lot of the mess that has crept in. We can use this opportunity to be much less forgiving with the code and make it clear about what is expected (type hints will help developers a lot). By using Pydantic data models we can get a tonne of nice validation for free, without using much in the way of additional code. We can also extend the validation methods for custom classes easily.
pydantic-settings
specifically (docs) has a lot of functionality that we currently do ourselves, or would like to have.Pydantic should also be able to help with #1256 by providing access to a wide range of nice output formats. Of particular interest is Apache Parquet and JSON Schema, which I think we should be well positioned for.
Progress
pydantic-settings
to read config from different sources #2535Series
for line plot data andextra_series
#2573The text was updated successfully, but these errors were encountered: