Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Failing elixirs on Apple Silicon #1838

Open
efaulhaber opened this issue Feb 7, 2024 · 3 comments
Open

Failing elixirs on Apple Silicon #1838

efaulhaber opened this issue Feb 7, 2024 · 3 comments
Labels
bug Something isn't working parallelization Related to MPI, threading, tasks etc.

Comments

@efaulhaber
Copy link
Member

efaulhaber commented Feb 7, 2024 •
edited

As I mentioned in #1830, many elixirs are still not working on Apple Silicon on multiple threads. To determine a definite list of elixirs, I started running all tests on 2 threads (ignoring allocation tests, which will fail on multiple threads due to extra allocations).

Here is the summary for TreeMesh. I'll keep running more tests in the background. The failing tests are due to slight differences in the L2/Linf errors. The errors are all due to LoadError: cfunction: closures are not supported on this platform (see #1462 for a solution that has to be applied to the remaining indicators).

Test Summary:                                                                          | Pass  Fail  Error  Total     Time
TreeMesh1D                                                                             |  973     2      4    979  4m50.9s
  Examples 1D                                                                          |  900     2      4    906  4m37.0s
    Linear scalar advection                                                            |   21                  21     0.9s
    Inviscid Burgers                                                                   |   20                  20     8.1s
    Hyperbolic diffusion                                                               |   14                  14     1.8s
    Compressible Euler                                                                 |  186            1    187    25.1s
      elixir_euler_source_terms.jl                                                     |    9                   9     0.1s
      elixir_euler_convergence_pure_fv.jl                                              |    9                   9     0.1s
      elixir_euler_density_wave.jl                                                     |    9                   9     0.1s
      elixir_euler_density_wave.jl with initial_condition_constant                     |    9                   9     0.1s
      elixir_euler_source_terms_nonperiodic.jl                                         |    9                   9     0.1s
      elixir_euler_ec.jl                                                               |    9                   9     0.1s
      elixir_euler_ec.jl with flux_kennedy_gruber                                      |    9                   9     0.0s
      elixir_euler_ec.jl with flux_shima_etal                                          |    9                   9     0.0s
      elixir_euler_ec.jl with flux_chandrashekar                                       |    9                   9     0.1s
      elixir_euler_ec.jl with flux_hll                                                 |    9                   9     0.1s
      elixir_euler_shockcapturing.jl                                                   |    9                   9     0.1s
      elixir_euler_sedov_blast_wave.jl                                                 |    9                   9     2.8s
      elixir_euler_sedov_blast_wave.jl (HLLE)                                          |    9                   9     3.1s
      elixir_euler_sedov_blast_wave_pure_fv.jl                                         |    9                   9     2.6s
      elixir_euler_sedov_blast_wave.jl with pressure                                   |    9                   9     2.7s
      elixir_euler_sedov_blast_wave.jl with density                                    |    9                   9     2.6s
      elixir_euler_positivity.jl                                                       |                 1      1     2.5s
      elixir_euler_blast_wave.jl                                                       |    9                   9     1.8s
      elixir_euler_quasi_1d_source_terms.jl                                            |   11                  11     0.1s
      elixir_euler_quasi_1d_discontinuous.jl                                           |   11                  11     2.2s
      elixir_euler_quasi_1d_ec.jl                                                      |   11                  11     1.7s
    Compressible Euler Multicomponent                                                  |   80                  80     9.9s
    MHD                                                                                |  209                 209    53.6s
    MHD Multicomponent                                                                 |  113                 113    20.2s
    Compressible Euler with self-gravity                                               |    9                   9     6.4s
    Shallow Water                                                                      |  148     2      3    153  1m19.3s
      elixir_shallowwater_ec.jl                                                        |    9                   9     4.1s
      elixir_shallowwater_ec.jl with initial_condition_weak_blast_wave                 |    9                   9     2.3s
      elixir_shallowwater_well_balanced.jl                                             |    9                   9     2.4s
      elixir_shallowwater_well_balanced.jl with FluxHydrostaticReconstruction          |    9                   9     4.2s
      elixir_shallowwater_well_balanced.jl with flux_nonconservative_ersing_etal       |    9                   9     3.7s
      elixir_shallowwater_well_balanced_wet_dry.jl with FluxHydrostaticReconstruction  |                 1      1     6.1s
      elixir_shallowwater_source_terms.jl                                              |    9                   9     4.7s
      elixir_shallowwater_source_terms.jl with flux_hll                                |    9                   9     4.2s
      elixir_shallowwater_source_terms.jl with flux_nonconservative_ersing_etal        |    9                   9     4.0s
      elixir_shallowwater_source_terms_dirichlet.jl                                    |    9                   9     2.0s
      elixir_shallowwater_source_terms_dirichlet.jl with FluxHydrostaticReconstruction |    9                   9     4.3s
      elixir_shallowwater_well_balanced_nonperiodic.jl with Dirichlet boundary         |    9                   9     3.8s
      elixir_shallowwater_well_balanced_nonperiodic.jl with wall boundary              |    9                   9     2.5s
      elixir_shallowwater_shock_capturing.jl                                           |    9                   9     5.9s
      elixir_shallowwater_beach.jl                                                     |                 1      1     3.4s
      elixir_shallowwater_parabolic_bowl.jl                                            |                 1      1     5.8s
      elixir_shallow_water_quasi_1d_source_terms.jl                                    |   11                  11     4.4s
      elixir_shallowwater_quasi_1d_well_balanced.jl                                    |    9     2            11     4.8s
      elixir_shallowwater_quasi_1d_discontinuous.jl                                    |   11                  11     4.0s
    Shallow Water Two layer                                                            |   39                  39    13.1s
    Linear scalar advection                                                            |   10                  10    15.8s
    Inviscid Burgers                                                                   |   15                  15    17.2s
    Compressible Euler                                                                 |   36                  36    24.9s
  Tests for initial conditions                                                         |   20                  20     7.1s
  Displaying components 1D                                                             |   45                  45     3.1s
  Additional tests in 1D                                                               |    6                   6     0.0s
  Nonconservative terms in 1D (linear advection)                                       |    1                   1     3.6s
ERROR: LoadError: Some tests did not pass: 973 passed, 2 failed, 4 errored, 0 broken.
in expression starting at /Users/erik/git/Trixi.jl/test/test_tree_1d.jl:1

test_tree_2d_part1.jl passed.

Test Summary:                                                             | Pass  Fail  Error  Total     Time
TreeMesh2D Part 2                                                         |  592    14      3    609  7m32.4s
  Examples 2D                                                             |  591    14      3    608  7m32.3s
    Acoustic Perturbation                                                 |   85                  85    39.3s
    Linearized Euler Equations 2D                                         |   22                  22    19.8s
    Compressible Euler                                                    |  366     8      2    376  4m44.5s
      elixir_euler_source_terms.jl                                        |   11                  11     7.9s
      elixir_euler_convergence_pure_fv.jl                                 |   11                  11     9.1s
      elixir_euler_density_wave.jl                                        |   11                  11     7.4s
      elixir_euler_source_terms_nonperiodic.jl                            |   11                  11     3.9s
      elixir_euler_ec.jl                                                  |   11                  11     7.7s
      elixir_euler_ec.jl with flux_kennedy_gruber                         |   11                  11     7.9s
      elixir_euler_ec.jl with flux_chandrashekar                          |   11                  11     8.3s
      elixir_euler_shockcapturing.jl                                      |   11                  11    12.6s
      elixir_euler_shockcapturing_subcell.jl                              |   11                  11    16.6s
      elixir_euler_blast_wave.jl                                          |   11                  11    10.7s
      elixir_euler_blast_wave_pure_fv.jl                                  |   11                  11     5.2s
      elixir_euler_blast_wave_amr.jl                                      |   11                  11     7.0s
      elixir_euler_blast_wave_sc_subcell_nonperiodic.jl                   |   11                  11     3.4s
      elixir_euler_sedov_blast_wave.jl                                    |   11                  11    12.7s
      elixir_euler_sedov_blast_wave_sc_subcell.jl                         |    3     8            11    14.2s
      elixir_euler_sedov_blast_wave.jl (HLLE)                             |   11                  11    13.5s
      elixir_euler_positivity.jl                                          |                 1      1     6.0s
      elixir_euler_blob_mortar.jl                                         |   11                  11    10.6s
      elixir_euler_blob_amr.jl                                            |   11                  11    15.0s
      elixir_euler_kelvin_helmholtz_instability_fjordholm_etal.jl         |   11                  11    15.9s
      elixir_euler_kelvin_helmholtz_instability.jl                        |   11                  11     3.4s
      elixir_euler_kelvin_helmholtz_instability_amr.jl                    |   11                  11     5.2s
      elixir_euler_kelvin_helmholtz_instability_sc_subcell.jl             |   11                  11    18.2s
      elixir_euler_colliding_flow.jl                                      |   11                  11     4.8s
      elixir_euler_colliding_flow_amr.jl                                  |   11                  11     4.1s
      elixir_euler_astro_jet_amr.jl                                       |   11                  11     6.0s
      elixir_euler_vortex.jl                                              |   11                  11     4.2s
      elixir_euler_vortex_mortar.jl                                       |   11                  11     4.4s
      elixir_euler_vortex_mortar_split.jl                                 |   11                  11     7.8s
      elixir_euler_vortex_shockcapturing.jl                               |   11                  11     5.4s
      elixir_euler_vortex_mortar_shockcapturing.jl                        |   11                  11     3.1s
      elixir_euler_vortex_amr.jl                                          |   11                  11     6.9s
      elixir_euler_ec.jl with boundary_condition_slip_wall                |   11                  11     3.6s
      elixir_euler_warm_bubble.jl                                         |                 1      1     0.4s
      Compressible Euler: Tests for initial conditions                    |   22                  22     7.0s
    Compressible Euler Multicomponent                                     |  104     6           110  1m26.7s
      Testing entropy2cons and cons2entropy                               |    2                   2     0.5s
      elixir_eulermulti_shock_bubble.jl                                   |   13                  13    14.8s
      elixir_eulermulti_shock_bubble_shockcapturing_subcell_positivity.jl |   15                  15    15.5s
      elixir_eulermulti_shock_bubble_shockcapturing_subcell_minmax.jl     |    7     6            13     3.9s
      elixir_eulermulti_ec.jl                                             |   11                  11     9.0s
      elixir_eulermulti_es.jl                                             |   17                  17    11.6s
      elixir_eulermulti_convergence_ec.jl                                 |   13                  13    10.4s
      elixir_eulermulti_convergence_es.jl                                 |   13                  13     9.2s
      elixir_eulermulti_convergence_es.jl with flux_chandrashekar         |   13                  13    10.7s
    Polytropic Euler                                                      |    9                   9     9.2s
    Acoustic perturbation coupled with compressible Euler                 |                 1      1     1.0s
      elixir_euleracoustics_co-rotating_vortex_pair.jl                    |                 1      1     0.7s
    KPP                                                                   |    5                   5    11.4s
ERROR: LoadError: Some tests did not pass: 592 passed, 14 failed, 3 errored, 0 broken.
in expression starting at /Users/erik/git/Trixi.jl/test/test_tree_2d_part2.jl:1

Test Summary:                                                                         | Pass  Error  Total      Time
TreeMesh2D Part 3                                                                     |  648      3    651  10m34.6s
  Examples 2D                                                                         |  647      3    650  10m34.6s
    MHD                                                                               |  210           210   3m09.1s
    MHD Multicomponent                                                                |  117           117   1m40.1s
    Lattice-Boltzmann                                                                 |  105           105     49.9s
    Shallow Water                                                                     |  110      3    113   2m22.7s
      elixir_shallowwater_ec.jl                                                       |   11            11     13.2s
      elixir_shallowwater_well_balanced.jl                                            |   11            11      4.7s
      elixir_shallowwater_well_balanced_wall.jl                                       |   11            11      4.8s
      elixir_shallowwater_well_balanced.jl with FluxHydrostaticReconstruction         |   11            11     15.6s
      elixir_shallowwater_well_balanced.jl with flux_nonconservative_ersing_etal      |   11            11     11.7s
      elixir_shallowwater_well_balanced_wet_dry.jl with FluxHydrostaticReconstruction |           1      1     16.3s
      elixir_shallowwater_source_terms.jl                                             |   11            11     11.2s
      elixir_shallowwater_source_terms_dirichlet.jl                                   |   11            11      4.1s
      elixir_shallowwater_source_terms.jl with flux_hll                               |   11            11     11.6s
      elixir_shallowwater_source_terms.jl with flux_nonconservative_ersing_etal       |   11            11     10.6s
      elixir_shallowwater_conical_island.jl                                           |           1      1     19.8s
      elixir_shallowwater_parabolic_bowl.jl                                           |           1      1      4.2s
      elixir_shallowwater_wall.jl                                                     |   11            11     12.2s
    Two-Layer Shallow Water                                                           |   51            51     47.9s
    Linear scalar advection                                                           |   10            10     27.0s
    Compressible Euler                                                                |   44            44   1m16.9s
ERROR: LoadError: Some tests did not pass: 648 passed, 0 failed, 3 errored, 0 broken.
in expression starting at /Users/erik/git/Trixi.jl/test/test_tree_2d_part3.jl:1

Test Summary:                                                              | Pass  Error  Total      Time
TreeMesh3D Part 1                                                          |  222      1    223  10m08.4s
  Examples 3D                                                              |  221      1    222  10m08.3s
    Compressible Euler                                                     |  221      1    222  10m07.9s
      elixir_euler_source_terms.jl                                         |   13            13   1m09.7s
      elixir_euler_convergence_pure_fv.jl                                  |   13            13     30.9s
      elixir_euler_source_terms.jl with split_form                         |   13            13     28.1s
      elixir_euler_convergence.jl                                          |   13            13     30.7s
      elixir_euler_mortar.jl                                               |   13            13      2.8s
      elixir_euler_amr.jl                                                  |   13            13     36.2s
      elixir_euler_taylor_green_vortex.jl                                  |   13            13     29.2s
      elixir_euler_shockcapturing.jl                                       |   13            13     45.2s
      elixir_euler_shockcapturing_amr.jl                                   |   13            13      9.8s
      elixir_euler_density_pulse.jl                                        |   13            13     29.7s
      elixir_euler_ec.jl                                                   |   13            13     10.2s
      elixir_euler_ec.jl with initial_condition=initial_condition_constant |   13            13     11.2s
      elixir_euler_ec.jl with flux_chandrashekar                           |   13            13     27.7s
      elixir_euler_ec.jl with flux_kennedy_gruber                          |   13            13     32.0s
      elixir_euler_ec.jl with flux_shima_etal                              |   13            13     27.6s
      elixir_euler_blob_amr.jl                                             |           1      1     30.7s
      elixir_euler_sedov_blast_wave.jl                                     |   13            13     46.4s
      elixir_euler_sedov_blast_wave.jl (HLLE)                              |   13            13   1m44.9s
ERROR: LoadError: Some tests did not pass: 222 passed, 0 failed, 1 errored, 0 broken.
in expression starting at /Users/erik/git/Trixi.jl/test/test_tree_3d_part1.jl:1

Test Summary:     | Pass  Total     Time
TreeMesh3D Part 2 |  127    127  4m15.7s

Test Summary:                                              | Pass  Fail  Total     Time
TreeMesh3D Part 2                                          |  320     2    322  9m47.2s
  Examples 3D                                              |  308     2    310  9m46.9s
    MHD                                                    |  147          147  6m38.2s
    Lattice-Boltzmann                                      |  114          114  1m20.7s
    Linear scalar advection                                |    8     2     10    28.1s
      elixir_advection_extended.jl                         |    5            5    16.2s
      elixir_advection_extended.jl with periodic operators |    3     2      5    11.7s
    Compressible Euler                                     |   39           39  1m19.3s
  Additional tests in 3D                                   |   11           11     0.1s
ERROR: LoadError: Some tests did not pass: 320 passed, 2 failed, 0 errored, 0 broken.
in expression starting at /Users/erik/git/Trixi.jl/test/test_tree_3d_part3.jl:1
@efaulhaber efaulhaber added bug Something isn't working parallelization Related to MPI, threading, tasks etc. labels Feb 7, 2024
@sloede
Copy link
Member

sloede commented Feb 7, 2024

Thanks a lot for investigating this. It seems all be related to using some form of positivity limiting

@efaulhaber
Copy link
Member Author

Most, if not all closure errors on Apple Silicon are caused by similar constructs as in #1462 in different indicators used for AMR and limiting.

@efaulhaber
Copy link
Member Author

The current problems in 1D and 2D are caused by:

  • IndicatorHennemannGassner in 1D.
  • IndicatorHennemannGassnerShallowWater in 1D.
  • IndicatorHennemannGassner in 2D.
  • IndicatorHennemannGassnerShallowWater in 2D.
  • The errors in elixir_euler_warm_bubble.jl and elixir_euleracoustics_co-rotating_vortex_pair.jl seem to be caused by the @threded loop in compute_coefficients!. No idea why this is not causing problems elsewhere.

Note that I mentioned the 1D and 2D indicators separately because these are different codes that need to be refactored like I did in #1462.

@sloede sloede mentioned this issue Apr 29, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working parallelization Related to MPI, threading, tasks etc.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@sloede @efaulhaber