Employ new halo-filling functions to update halos of prognostic variables on the cubed sphere

[siddharthabishnu]

This PR incorporates significant updates from PR #3306. Most importantly, it employs the new halo-filling functions to update the halos of the prognostic variables. Additionally, it removes the replace_horizontal_vector_halos! function, enhances existing methods and function calls related to the cubed sphere grid, and refines the vorticity computation function. It also implements a new test to verify that the norm of the prognostic variables of the Rossby-Haurwitz wave remains almost constant as it propagates over the cubed sphere.

[navidcy]

Let me know when this is ready to review.

Could you rename this to a title the just refers to what has happened in this PR?

[navidcy]

this seems like a hack

also, what does ssh refers to?

[siddharthabishnu]

Yes, it is a temporary hack. I have outlined the issue arising in the absence of this hack in issue #3572 and created PR #3573 to resolve it. In the comment, ssh refers to fields like ssh defined on the xy plane, for which the parent array takes the z-halo into consideration even though it does not exist. In case of a field like the meridional average of a scalar quantity which is defined on the xz plane, the same issue arises without an equivalent hack.

[navidcy]

OK. Well we won't merge a hack, right?

[siddharthabishnu]

No, we won't. It's only temporary. Once I fix it in PR #3573, we will import the solution (not hack) in this PR.

[navidcy]

OK, I see. This PR requires first #3573. Gotcha!

[navidcy]

I don't understand though why filling halos in z for free surface has a problem now; that was OK up to now, right? Why is #3573 holding us here? What I'm saying is that filling halos in the vertical shouldn't be any different that how we were doing it for other grids, right?

[navidcy]

@siddharthabishnu could you merge main now that #3573 is merged? I think then this comment is not relevant any more and can be resolved ;)

[glwagner]

@siddharthabishnu, what capability does this PR add?

[siddharthabishnu]

Let me know when this is ready to review.

Could you rename this to a title the just refers to what has happened in this PR?

I just renamed the title to reflect the main functionality of this PR. The description outlines the additional features included in this PR. I will notify you when it is ready for review.

[siddharthabishnu]

@siddharthabishnu, what capability does this PR add?

@glwagner, this PR uses the new halo-filling functions to update halos for prognostic variables and refines the vorticity computation function on the cubed sphere. Additional features introduced in this PR are detailed in the description.

[navidcy]

Does this PR resolved #1584 (amongst other things)?

[siddharthabishnu]

Does this PR resolve #1584 (amongst other things)?

Yes, it does, in commit 41c991f.

[siddharthabishnu]

To compare the numerical solution of the Rossby-Haurwitz wave on the cubed sphere against an established benchmark, I chose the exact solution of the non-linear barotropic vorticity equation on the sphere (not the solution of the non-linear shallow water equations). This was achieved by replacing the initial phase Rλ with the time-dependent phase Rλ−νt in Equations (143)–(146) of Williamson et al., using the angular velocity ν given by Equation (142). The results at a resolution of 128x128 cells per panel are as follows:

• Since we prescribe the initial stream function and numerically take its derivatives to specify the initial velocities on the cubed sphere, numerical errors are present even in the initial velocities and the diagnosed initial vorticity. The vorticity error is significantly pronounced near the cubed sphere corners due to the deviation from orthogonality. For instance, in the domain's interior, the vorticity error norm is three orders of magnitude less than the vorticity norm, whereas at the corners, it is nearly the same order of magnitude.
• As the simulation progresses, the amplified error from the corners propagates into the interior as expected. After 500 time steps, the vorticity error in the interior increases from three orders of magnitude less to one order of magnitude less than the vorticity norm.

I have attached panel-wise plots of the exact vorticity and the error of its numerical counterpart on the cubed sphere for the initial condition and after 500 time steps for reference. Given these circumstances, @navidcy and I are contemplating whether we should continue using the exact solution for verification in the test script or consider alternative verification methods, such as ensuring that the norm of the numerical solution of the Rossby-Haurwitz wave remains bounded within a specified range (as mentioned in the introduction to this PR). @glwagner, @simone-silvestri and @jm-c, let us know your thoughts.

Exact initial vorticity

Error in initial vorticity

Exact vorticity after 500 time steps

Error in vorticity after 500 time steps

[navidcy]

So to be honest @siddharthabishnu phrasing regarding me "contemplating" is a bit generous. Personally, I'm not sure whether this can be a dynamics test; something for which the error is just an order of magnitude less than the solution itself seems a bit loose for me...

isn't there anything else we can use for a dynamics test? feels like we've been stuck on this for ages... we need a dynamics test to merge the cubed sphere code and move on I think. Perhaps there is no dynamics test we can have?

The solution @siddharthabishnu uses above is not an exact solution of the HydrostaticFreeSurfaceModel equations... thus the big error. (It's an exact solution of the barotropic vorticity equation...) But whether this big of an error is expected or is it because the code has a bug I don't know and we've been in this vagueness for long. Anybody has any ideas?

[navidcy]

What if we have free_surface=nothing? Would that mean that we are solving the barotropic vorticity eq with Nz=1?