unable to train correct result (gives incorrect answers even when everything is set correct)

[Liza23]

I've been trying to solve some ordinary differential equation (in one variable) but each time I run the model, I get different results -- sometime exactly as expected and other times entirely opposite to what is expected. How can this be reliable in such a case?

The training & validation error is usually pretty low and hence doesn't add to model misbehaving.

[shuheng-liu]

Hi Liza, usually, there's randomness involved when network weights are initialized. Try manually setting the random see for torch and numpy will guarantee reproducible results.

If you're worried about the model being susceptible to random seeding, can you share your code so I can know which ODE you're solving and what hyperparameters you are using?

[Liza23]

Hey thanks,
This is the ODE that I'm trying to solve:
diff(alpha, r, order=2) + 2 * diff(alpha, r, order=1) * diff(alpha, r, order=1) + 2 * (1/r) * diff(alpha, r, order=1)
and these are the intial value parameters that I'm setting
init_val = IVP(t_0=5, u_0=0.0)

And this is what I'm solving in the "ode solver":
ode=ricci_tensor_one_var, condition=init_val, t_min=1.0, t_max=5

I even tried setting the train and validation data myself to eliminate randomness.
The solution I'm getting is completely opposite of what I'm supposed to get.

Also, the training and validation errors are considerably low still I'm getting incorrect results.

Any suggestion as to why this is happening would be helpful! Thanks

[shuheng-liu]

It's because of the solution is not unique. Each time, your network can learn a different solution under given constraint. See my comment below for more details.

[shuheng-liu]

This is a 2nd order ODE. Typically, we need two constraints to uniquely determine the solution.

In addition to u(t0)=u0, you need another constraint, such as

1. u'(t0)=u0': Use IVP but with a third argument, E.g.: IVP(t0, u0, u0prime)
2. u(t1)=u1: Use DirichletBVP from neurodiffeq.conditions. E.g.: DirichletBVP(t0, u0, t1, u1)
3. Some other constraint, checkout DoubleEndedBVP1D, etc. in the documentation.

[Liza23]

Hey thanks, this makes sense.
However, even after specifying enough conditions, I'm getting bizarre results. Again, the opposite of what is expected.

This is the modified IVP condition that we're using:
init_val = IVP(t_0=10, u_0=10e-5, u_0_prime=0.0)

We know the solution analytically and hence able to compare that what we're getting is not correct.
The analytical solution is:
alpha = 1/2 (log(1/r - 1))

[shuheng-liu]

This is a rather peculiar equation. Can you share the background of this equation (e.g., what alpha and r stands for, what is the system being modelled here)? In the meantime, I would recommend using the monitor feature to plot the solution. The code will be similar to Cell 16 of this tutorial. Note that you can pass fig_dir=... to to_callback(), which will save the images to disk.

For example,

monitor_callback = monitor.to_callback(fig_dir='my_images/')

...

solver.fit(max_epochs=..., callbacks=[monitor_callback])

[shuheng-liu]

Upon second look at the equation, why don't we just do a variable substitution? Namely, let v = diff(alpha, r, orde=1). The equation therefore becomes a first order nonlinear ODE : diff(v, r) + 2*v**2 - 2/r*v = 0, with initial condition v(10)=0.0. This is a Bernoulli differential equation and can be solved with some substitution u = 1/v. If you model u with a neural network and use neurodiffeq to solve it, it should probably work.

[Liza23]

Hey, thanks for suggesting this approach. This seems to give us a rough plot of what the actual alpha is.

For, what the background of the equation is -- it is the Ricci Tensor equations to obtain Schwarzschild Solution. For more reference you can check this link: https://arxiv.org/pdf/gr-qc/9712019.pdf (from page - 168: 170)

However, our doubt as to why we are getting peculiar results even after specifying the conditions? Is there a way to understand the model. We want to model more complex equations using the package and wanted to try on the easier known ones first and hence it is important to understand why are we getting such results.

[Liza23]

See this is graph of alpha that we are getting which is exactly opposite of what is desired. And on one unusual run, it does give the right answer as well. Hence, we're just dangling with the right & the wrong answers with each iteration.

[Liza23]

Running the same set of equations again, I obtain this:)