Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: Make Quantity an abstract base class #822

Draft
wants to merge 17 commits into
base: v1.x.x
Choose a base branch
from

Conversation

llucax
Copy link
Contributor

@llucax llucax commented Dec 18, 2023

This is a draft PR just to make sure there are no big objections to moving forward with the idea.

The draft is there mostly to show how the new quantity and SupportFloatT is used, the _quantities module in particular has a lot of noise, so it shouldn't be reviewed in detail.

The code in this PR passes mypy and flake8 checks on src, but not in tests or benchmarks, as these are not updated yet.

The most important things to note are:

  • It uses SupportFloatT as a TypeVar bound to SupportsFloat to allow Sample and most other types working with either float or Quantities to be used.

  • This means for the generic types needing to actually use the underlaying value, we sometimes need an extra conversion to float when some operation is needed. This only happens in Sample3Phase to get the max/min, and the resampling function. It happens also implicitly when checking for NaN of infinity.

  • Added a value constructor to the resampler to be able to produce new samples. This has the nice side effect that if you need to resample something that you know is in, say, kW instead of W, you can use Power.from_kilowatts as the constructor.

  • There are quite a few (unfinished) fixes/improvements to Quantity and its sub types. The most important changes to Quantity are:

    • The trick to disable the default constructor was replaced by a more straightforward implementation in the __init__ and the utility _new method is added to create instances.
    • The .base_value was replaced by __float__, as this was only exposed to be able to work with a Quantity as if it was an unknown float.
    • QuantityT is not needed anymore and removed.
    • Quantity is now an ABC.
    • The _base_value is moved to the constructor, to have better rendering for the docs.
    • Removed isnan() and isinf() as now the math functions can be used directly instead (maybe we can keep them for convenience).
    • There are some random migrations to match instead of if, which needs to be cleaned up.

Of course once this have a rough approval, I'll split in smaller commits (and even PRs) and update tests and docs.

Fixes #821.

@github-actions github-actions bot added part:docs Affects the documentation part:tooling Affects the development tooling (CI, deployment, dependency management, etc.) part:data-pipeline Affects the data pipeline part:actor Affects an actor ot the actors utilities (decorator, etc.) part:microgrid Affects the interactions with the microgrid labels Dec 18, 2023
@llucax llucax requested a review from shsms December 18, 2023 15:14
@github-actions github-actions bot removed part:docs Affects the documentation part:tooling Affects the development tooling (CI, deployment, dependency management, etc.) labels Dec 18, 2023
@llucax llucax force-pushed the quantity-abc branch 2 times, most recently from 1d66c40 to 2ebed70 Compare January 4, 2024 15:27
@llucax llucax added this to the v1.0.0-rc5 milestone Jan 29, 2024
@github-actions github-actions bot added part:docs Affects the documentation part:tests Affects the unit, integration and performance (benchmarks) tests labels Feb 12, 2024
@github-actions github-actions bot removed the part:docs Affects the documentation label Feb 15, 2024
Otherwise unhandled exceptions will be silently swallowed by the task,
which makes debugging really hard.

Signed-off-by: Leandro Lucarella <luca-frequenz@llucax.com>
Signed-off-by: Leandro Lucarella <luca-frequenz@llucax.com>
We are going to make `Quantity` a abstract base class, so we don't need
to do the trick to only allow to call the default constructor for the
`Quantity` class but not sub-classes. We just make `Quantity.__init__()`
always raise a `TypeError`, as subclasses can use `_new()` to create
instances.

Signed-off-by: Leandro Lucarella <luca-frequenz@llucax.com>
We don't want `Quantity` to be instantiated by using `zero()` or
`from_string()` either.

Signed-off-by: Leandro Lucarella <luca-frequenz@llucax.com>
This was possible before too, but using the `base_value` property, which
is removed by this commit. Now the more idiomatic `float(quantity)`
conversion is provided, although it should only be used for generics,
when functions or classes are parametrized with a `SupportsFloat` type.

`float(quantity)` always returns the quantity in the `base_unit`.

This allows to make generic classes and functions that can operate on
both `Quantity` or raw `float`s (or any other type that supports float).

The methods `isnan()` and `isinf()` are removed because now
`math.isnan()` and `math.isinf()` (as well as other `math` utility
functions) can be directly used with `Quantity`.

Signed-off-by: Leandro Lucarella <luca-frequenz@llucax.com>
Now that `Quantity` is an abstract base class, we can't really
instantiate the base class directly in tests, so we need to make some
adjustements and test that `Quantity` can't be constructed in any way.

Signed-off-by: Leandro Lucarella <luca-frequenz@llucax.com>
Now classes and functions will be parametrized in a way that can take
any type that can be converted to `float`, including `float` itself.

Signed-off-by: Leandro Lucarella <luca-frequenz@llucax.com>
Instead of using the old `QuantityT`, we use `SupportsFloatT`, which
allows us to use both quantities and raw floats for samples.

Signed-off-by: Leandro Lucarella <luca-frequenz@llucax.com>
Instead of using the `QuantityT` type variable, we use the
`SupportsFloatT` which allows for more flexibility in the type of
samples that can be stored in the ring buffer, for example, supporting
raw `float` values.

Signed-off-by: Leandro Lucarella <luca-frequenz@llucax.com>
The resampler can now be paremetrized on both quantities and `float`.
Even more, the values a resampler resamples are now attached to the
specific type, not to the base `Quantity` class, so users can be sure
the resampler will always return the appropriate quantity.

If a resampler must be used to resample different types of quantities,
it can now be parametrized using plain `float`s, which should also yield
better performance as it avoids creating `Quantity` objects.

Signed-off-by: Leandro Lucarella <luca-frequenz@llucax.com>
The data sourcing actor will now use `Sample[float]` to send samples
instead of a unit-less `Quantity` base class that provides no value and
decreases performance.

Signed-off-by: Leandro Lucarella <luca-frequenz@llucax.com>
Same as with the data sourcing actor, the resampling actor will now use
`Sample[float]` to send samples instead of a unit-less `Quantity` base
class that provides no value and decreases performance.

Signed-off-by: Leandro Lucarella <luca-frequenz@llucax.com>
This is just a subclass of `ResamplerConfig` that is specialized with
`float`. Since the `ResamplerConfig` now needs more configuration
(the type and the value constructor), and the `ResamplerConfig` is
passed around quite a bit during initialization, it makes sense to have
a class that already provides the basic stuff for `float`, which is what
will always be used by the resampling actor because it needs to deal
with different types of quantities (and is also good for performance
reasons).

Signed-off-by: Leandro Lucarella <luca-frequenz@llucax.com>
Because the resampler is now generic, we can also make the
`MovingWindow` and `PeriodicFeatureExtractor` generic so they can return
a specialized quantity instead of a unit-less quantity, again improving
performance and safety.

Signed-off-by: Leandro Lucarella <luca-frequenz@llucax.com>
@llucax
Copy link
Contributor Author

llucax commented Feb 15, 2024

When fixing the tests I bumped into an issue that might be tricky to solve.

If I understand correctly, all the FormulaEngine stuff is parametrized on a QuantityT mainly because of the high level formula engine, which allows to create formulas from formulas.

So far it seems like there is a lot of code and complexity to deal with QuantityT, which can be a Quantity or float, or even 3-phase stuff. For what I've seen these classes also don't differentiate very well between the input and output type, and these are usually mixed (one QuantityT TypeVar parameter for both things, which can resolve to Quantity and thus have a different type for the input and output, but the class is then specialized with Quantity, not the specific subclass).

As it is taking me a quite a bit of time trying to understand all the formula engine stuff, in particular when thinking about input and output types, I will delay the work on this PR.

It is not very clear to me which stuff should work with float and which should use quantities. For example, the evaluation is all done with plain floats by the FormulaEvaluator, which also means that if we are mixing different types.

We need to make sure to use the base_value in a way that is compatible, in particular for multiplication (like current * voltage). In this sense I think we might need to have a more explicit definition that base_values are stored in the base SI unit or something like that, so we can operate on them safely as pure floats.

To progress with this PR I will probably add a second parameter for formula engine stuff that could have a different input and output type, so we have something like InputT and OutputT.

@llucax
Copy link
Contributor Author

llucax commented Feb 15, 2024

I thought that maybe to reduce the complexity of this formula engine stuff we could pin the type to float and consider it a low-level construct, and only convert to a quantity after a value is retrieved from a formula engine, but I guess this won't work.

The problem is formula engines are really "leaked" to the user, so users can do (microgrid.grid.power + microgrid.pv.power).new_receiver() and it would be bad if that receiver just gets floats, it would mean users will need to convert to Power manually which kills the built-in safety we have now. :(

I guess this is also a shame in terms of performance as we might have a lot of conversion going on inside the pipeline that is not strictly necessary.

Not sure if there is a way to have all the internal formula engine stuff working with floats and only convert to quantity when leaving the engine, I guess we would have to put (a lot) more thought into it to find out.

@llucax
Copy link
Contributor Author

llucax commented Feb 15, 2024

FYI, I cleaned up this PR a bit, it is still a WIP, but except from the commits with WIP in it, they should be much more readable and give a much better idea of what the intention of this PR is and how things are supposed to work.

@llucax llucax modified the milestones: v1.0.0-rc5, v1.0.0-rc6 Feb 19, 2024
@llucax llucax modified the milestones: v1.0.0-rc6, v1.0.0-rc7 Mar 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
part:actor Affects an actor ot the actors utilities (decorator, etc.) part:data-pipeline Affects the data pipeline part:microgrid Affects the interactions with the microgrid part:tests Affects the unit, integration and performance (benchmarks) tests
Projects
Status: To do
Development

Successfully merging this pull request may close these issues.

Quantity should be an abstract base class
1 participant