Adding Training Strategies to dae_solvers.jl #838
+189
−2
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Added in StochasticTraining strategy for generate_loss function
hippyhippohops
changed the title
Added WeightedIntervalTraining
src/dae_solve.jl
Outdated
Comment on lines
97
to
119
| autodiff && throw(ArgumentError("autodiff not supported for WeightedIntervalTraining.")) | ||
| minT = tspan[1] | ||
| maxT = tspan[2] | ||
|
|
||
| weights = strategy.weights ./ sum(strategy.weights) | ||
|
|
||
| N = length(weights) | ||
| points = strategy.points | ||
|
|
||
| difference = (maxT - minT) / N | ||
|
|
||
| data = Float64[] | ||
| for (index, item) in enumerate(weights) | ||
| temp_data = rand(1, trunc(Int, points * item)) .* difference .+ minT .+ | ||
| ((index - 1) * difference) | ||
| data = append!(data, temp_data) | ||
| end | ||
|
|
||
| ts = data | ||
| function loss(θ, _) | ||
| sum(abs2, inner_loss(phi, f, autodiff, ts, θ, p, differential_vars)) | ||
| end | ||
| return loss |
There was a problem hiding this comment.
the formatting is off here.
There was a problem hiding this comment.
Okay, I will look into it.
src/dae_solve.jl
Outdated
Comment on lines
85
to
96
| function generate_loss(strategy::StochasticTraining, phi, f, autodiff::Bool, tspan, p, differential_vars::AbstractVector) | ||
| autodiff && throw(ArgumentError("autodiff not supported for StochasticTraining.")) | ||
| function loss(θ, _) | ||
| ts = adapt(parameterless_type(θ), | ||
| [(tspan[2] - tspan[1]) * rand() + tspan[1] for i in 1:(strategy.points)]) | ||
| sum(abs2, inner_loss(phi, f, autodiff, ts, θ, p, differential_vars)) | ||
| end | ||
| return loss | ||
| end | ||
|
|
||
|
|
||
| function generate_loss(strategy::WeightedIntervalTraining, phi, f, autodiff::Bool, tspan, p,differential_vars::AbstractVector) |
There was a problem hiding this comment.
Looks like a repeat of the ODE code. Is it actually necessary to repeat or could it be refactored so DAEs and ODEs use the same function?
There was a problem hiding this comment.
Yes, I did draw inspiration from the ODE code. I will look into refactoring it so that the same function will be used.
I added the test function for weighted interval training
hippyhippohops
commented
Apr 1, 2024
|
No, that was not the issue. The issue was to refactor common computations in |
In the previous commit, I merged all the generate_loss function into one function, but I refactored the code incorrectly. I need to refactor each generate loss function between ode_solve and dae_solve
hippyhippohops
commented
May 6, 2024
|
I think we should add rest of the strategies before merging. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Added in StochasticTraining strategy for generate_loss function
Checklist
contributor guidelines, in particular the SciML Style Guide and
COLPRAC.
Additional context
Add any other context about the problem here.