Error running navierstokes example

[lindsayad]

Hello, I'm getting the following error:

[0]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------
[0]PETSC ERROR: Error in external library
[0]PETSC ERROR: /home/lindad/libCEED/interface/ceed-operator.c:73 in CeedOperatorCheckField(): Field 'dx' of size 9 and EvalMode gradient: CeedElemRestriction/Basis has 4 components
[0]PETSC ERROR: See https://petsc.org/release/faq/ for trouble shooting.
[0]PETSC ERROR: Petsc Development GIT revision: v3.20.5-735-g38dc5bc9508  GIT Date: 2024-03-08 08:25:36 -0600
[0]PETSC ERROR: ./navierstokes on a arch-moose named pop-os by lindad Wed Mar 13 16:02:02 2024
[0]PETSC ERROR: Configure options --with-64-bit-indices --with-cxx-dialect=C++17 --with-debugging=no --with-fortran-bindings=0 --with-mpi=1 --with-openmp=1 --with-shared-libraries=1 --with-sowing=0 --download-fblaslapack=1 --download-hypre=1 --download-metis=1 --download-mumps=1 --download-ptscotch=1 --download-parmetis=1 --download-scalapack=1 --download-slepc=1 --download-strumpack=1 --download-superlu_dist=1 --download-hdf5=1 --with-hdf5-fortran-bindings=0 --download-hdf5-configure-arguments="--with-zlib" --with-make-np=64  --CFLAGS=-fno-omit-frame-pointer --CXX_CXXFLAGS=-fno-omit-frame-pointer
[0]PETSC ERROR: #1 SetupLibceed() at /home/lindad/libCEED/examples/fluids/src/setuplibceed.c:307
[0]PETSC ERROR: #2 main() at /home/lindad/libCEED/examples/fluids/navierstokes.c:195
[0]PETSC ERROR: PETSc Option Table entries:
[0]PETSC ERROR: -ceed /cpu/self (source: command line)
[0]PETSC ERROR: -degree 1 (source: command line)
[0]PETSC ERROR: -dm_blocking_type field_node (source: code)
[0]PETSC ERROR: -dm_plex_simplex 0 (source: code)
[0]PETSC ERROR: -dm_sparse_localize 0 (source: code)
[0]PETSC ERROR: ----------------End of Error Message -------send entire error message to petsc-maint@mcs.anl.gov----------
application called MPI_Abort(MPI_COMM_SELF, 76) - process 0
[unset]: PMIU_write error; fd=-1 buf=:cmd=abort exitcode=76 message=application called MPI_Abort(MPI_COMM_SELF, 76) - process 0
:
system msg for write_line failure : Bad file descriptor

Any suggestions? I'll probably hop into the debugger and see if I can figure it out

[jeremylt]

What's your command line option to run the example and the yml you're using? Also, what's your git hash for libCEED?

[lindsayad]

lindad@pop-os:~/libCEED/examples/fluids(main)$ ./navierstokes -ceed /cpu/self -degree 1

gets me the same error

[lindsayad]

libCEED hash is fad128ee

[lindsayad]

I feel dense; I don't know what this question means 😅

What's the yml you're using?

[jedbrown]

Usually there will be ./navierstokes -options_file gaussianwave.yml or similar. You can write everything into a command line, but there isn't a meaningful default case so you'd usually have options to set the domain and boundary conditions. (That error message sucks, but fixing it would still just point at the first required option that isn't provided.)

[lindsayad]

I see. I think the documentation at the top of the source file should probably be edited a bit. The "Sample run" language

// Sample runs:
//
//     ./navierstokes -ceed /cpu/self -problem density_current -degree 1

gives the impression to me that it should work. Looking at the Sphinx documentation for that example, I saw that the default problem and degree were density_current and 1 respectively so I thought that I could run with no arguments and get something meaningful.

But yes, ./navierstokes -options_file gaussianwave.yml runs beautifully, thank you!

Tangential to my OP, I'm wondering whether there is (or there is a plan to have) a configuration option for 64 bit CeedInt? I noticed warnings when compiling together with a 64 bit PETSc

[jeremylt]

@jrwrigh we should probably build against Int64 in CI somewhere like Ratel does.

There isn't any technical issues with running with Int64 here, we just sometimes oops an integer conversation when adding a new feature.

It occurs to me that probably we should check if any options are set and point people to the docs if no CL options are used. I assume the sample command you referenced is a bit out of date and should be updated.

[jrwrigh]

@lindsayad Sorry for the rough first experience! The fluids example is kinda my baby, so that's my bad.

Tangential to my OP, I'm wondering whether there is (or there is a plan to have) a configuration option for 64 bit CeedInt? I noticed warnings when compiling together with a 64 bit PETSc

I'm not sure if there are plans/options to make CeedInt 64 bit, but in the vast majority of cases it shouldn't be necessary as CeedInt is more-or-less restricted to "rank-local indexing" type operations. PetscInt should handle the global numbering type operations, so it should handle the move to 64bit ints, e.g. when total number of DoFs across ranks is above 2 billion.

[jrwrigh]

@jrwrigh we should probably build against Int64 in CI somewhere like Ratel does.

The issue there is that the testing suite is based on loading PETSc binary vectors in as the reference solution for regression tests, but PETSc binary files are only loadable by libraries with the same PetscInt size. When PETSc reads the file, it checks the header for the vector size, but it's always written by a PetscInt. So if you write a file with PetscInt == int32_t and read it with PetscInt == int64_t, then it will read the size wrong. Otherwise I'd be testing locally with 64bit PETSc locally at all times.

All that said, most issues can be caught just in the build, so probably at least doing that.

It occurs to me that probably we should check if any options are set and point people to the docs if no CL options are used. I assume the sample command you referenced is a bit out of date and should be updated.

Yeah, I think moving "exclusively" to recommending yaml files that are tested is a good idea.