Trilinos MueLu is not working

[koecher]

During the configuration of deal.II, the setup complains that Trilinos MueLu is found but not working. Therefore, the AMG MueLu Preconditioners are not compiled.

I found this issue today with deal.II v9.3 but it was already present for deal.II v9.2 and maybe even longer.

[koecher]

Status so far: Trilinos in our configuration disables the MueLu support for Epetra. deal.II detects this state correctly during the configuration and disables the wrapper support.

Unfortunately, I can't find the reason for this. I only find solutions where disabling the MueLu support of Epetra is the solution.

It looks like MueLu is still working for Tpetra, but we don't have the wrapper.

Anyhow, the older ML is still working and should be used in case of trouble.

[jpthiele]

I ran into a similar issue when trying to get Trilinos 13.2 with ShyLU_DD to work with our Epetra Wrappers.
MueLu supports Epetra with the same function Convert_Epetra_CrsMatrix_ToXpetra_CrsMatrixWrap that I tried to use,
but I got a linker error.
Setting
-D Tpetra_INST_INT_LONG_LONG=OFF
-D Tpetra_INST_INT_INT=ON
fixed that, so that might be worth a try.

[koecher]

@jpthiele nice that you found that. That would be useful for the next official Trilinos Version and configuration.

Can you provide a PR with the updated Trilinos package?

[jpthiele]

I only have the trilinos.package specific to ShyLU.
Because @kinnewig and I were aware of this issue here we disabled MueLu too.
But adding the two lines should hopefully do the trick.
I thought you might still have the failing config somwhere and could try it.

[Wendi-L]

Just a quick comment: I experienced the same error that Trilinos MueLu was found but is not usable through Epetra, and MueLu support is disabled during deal.ii building. I am using the dealii-9.4 branch. Moreover, the build failed with the error message below:

[ 74%] Linking CXX executable ../bin/step-9.release
/usr/bin/ld: /*/trilinos-release-13-2-0/lib/libstratimikosbelos.so: undefined reference to `Tpetra::MultiVector<float, int, long long, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace> >::getGlobalLength() const'
/usr/bin/ld: /*/trilinos-release-13-2-0/lib/libstratimikosbelos.so: undefined reference to `Tpetra::MultiVector<float, int, long long, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace> >::get1dCopy(Teuchos::ArrayView<float> const&, unsigned long) const'
/usr/bin/ld: /*/trilinos-release-13-2-0/lib/libstratimikosbelos.so: undefined reference to `Tpetra::MultiVector<float, int, long long, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace> >::getVector(unsigned long) const'
/usr/bin/ld: /*/trilinos-release-13-2-0/lib/libstratimikosamesos2.so: undefined reference to `Amesos2::TachoSolver<Tpetra::CrsMatrix<float, int, long long, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace> >, Tpetra::MultiVector<float, int, long long, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace> > >::TachoSolver(Teuchos::RCP<Tpetra::CrsMatrix<float, int, long long, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace> > const>, Teuchos::RCP<Tpetra::MultiVector<float, int, long long, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace> > >, Teuchos::RCP<Tpetra::MultiVector<float, int, long long, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace> > const>)'
/usr/bin/ld: /*/trilinos-release-13-2-0/lib/libstratimikosbelos.so: undefined reference to `Tpetra::MultiVector<float, int, long long, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace> >::getLocalLength() const'
/usr/bin/ld: /*/trilinos-release-13-2-0/lib/libstratimikosbelos.so: undefined reference to `Tpetra::MultiVector<float, int, long long, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace> >::getNumVectors() const'
/usr/bin/ld: /*/trilinos-release-13-2-0/lib/libstratimikosamesos2.so: undefined reference to `Amesos2::KLU2<Tpetra::CrsMatrix<float, int, long long, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace> >, Tpetra::MultiVector<float, int, long long, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace> > >::KLU2(Teuchos::RCP<Tpetra::CrsMatrix<float, int, long long, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace> > const>, Teuchos::RCP<Tpetra::MultiVector<float, int, long long, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace> > >, Teuchos::RCP<Tpetra::MultiVector<float, int, long long, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace> > const>)'
collect2: error: ld returned 1 exit status
make[2]: *** [examples/CMakeFiles/step-9.release.dir/build.make:259: bin/step-9.release] Error 1
make[1]: *** [CMakeFiles/Makefile2:3942: examples/CMakeFiles/step-9.release.dir/all] Error 2

I followed @jpthiele to set the following to fix this issue:

-D Tpetra_INST_INT_LONG_LONG=OFF
-D Tpetra_INST_INT_INT=ON

[jpthiele]

I had the same issue a few days ago with MUMPS in Trilinos.
By default only the scalar type double is instantiated in Tpetra, so an additional
-D Tpetra_INST_FLOAT:BOOL=ON
should and then a rebuild of Trilinos should fix this.
Let us know if this works.

[Wendi-L]

I had the same issue a few days ago with MUMPS in Trilinos. By default only the scalar type double is instantiated in Tpetra, so an additional -D Tpetra_INST_FLOAT:BOOL=ON should and then a rebuild of Trilinos should fix this. Let us know if this works.

Many thanks, @jpthiele, for your kind reply and advise!

I tried two settings. One failed and the other succeeded. Details are as follows:

Configure 1:

  -D Tpetra_INST_INT_LONG_LONG=OFF
  -D Tpetra_INST_INT_INT=ON
  -D Tpetra_INST_FLOAT:BOOL=ON

It failed during the Trilinos installation stage with the error message below:

[100%] Building CXX object packages/muelu/adapters/CMakeFiles/muelu-adapters.dir/ExplicitInstantiation/Thyra_XpetraLinearOp.cpp.o
In file included from /*/trilinos-release-13-2-0/packages/muelu/adapters/ExplicitInstantiation/Thyra_MueLuPreconditionerFactory.cpp:52:
/*/Trilinos-trilinos-release-13-2-0/packages/muelu/adapters/stratimikos/Thyra_MueLuPreconditionerFactory_def.hpp: In instantiation of ‘bool Thyra::replaceWithXpetra(Teuchos::ParameterList&, std::string) [with Scalar = float; LocalOrdinal = int; GlobalOrdinal = int; Node = Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial>; std::string = std::__cxx11::basic_string<char>]’:
/*/Trilinos-trilinos-release-13-2-0/packages/muelu/adapters/stratimikos/Thyra_MueLuPreconditionerFactory_def.hpp:384:66:   required from ‘void Thyra::MueLuPreconditionerFactory<Scalar, LocalOrdinal, GlobalOrdinal, Node>::initializePrec(const Teuchos::RCP<const Thyra::LinearOpSourceBase<Scalar> >&, Thyra::PreconditionerBase<Scalar>*, Thyra::ESupportSolveUse) const [with Scalar = float; LocalOrdinal = int; GlobalOrdinal = int; Node = Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial>]’
/*/Trilinos-trilinos-release-13-2-0/packages/muelu/src/Interface/../Utils/MueLu_ETI_4arg.hpp:32:3:   required from here
/*/Trilinos-trilinos-release-13-2-0/packages/muelu/adapters/stratimikos/Thyra_MueLuPreconditionerFactory_def.hpp:206:60: error: no matching function for call to ‘get_Epetra_Map(const Thyra::VectorSpaceBase<float>&, Teuchos::RCP<const Epetra_Comm>&)’
  206 |           RCP<const Epetra_Map> map = Thyra::get_Epetra_Map(*(thyM->range()), comm);
      |                                       ~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~
In file included from /*/Trilinos-trilinos-release-13-2-0/packages/xpetra/sup/Utils/Xpetra_ThyraUtils.hpp:99,
                 from /*/Trilinos-trilinos-release-13-2-0/packages/xpetra/src/BlockedCrsMatrix/Xpetra_BlockedCrsMatrix.hpp:79,
                 from /*/Trilinos-trilinos-release-13-2-0/packages/xpetra/sup/Utils/Xpetra_MatrixMatrix.hpp:53,
                 from /*/Trilinos-trilinos-release-13-2-0/packages/muelu/src/Interface/../Utils/MueLu_Utilities_decl.hpp:76,
                 from /*/trilinos-release-13-2-0/packages/muelu/src/MueLu_Utilities.hpp:1,
                 from /*/Trilinos-trilinos-release-13-2-0/packages/muelu/src/Interface/../MueCentral/MueLu_Level.hpp:68,
                 from /*/Trilinos-trilinos-release-13-2-0/packages/muelu/adapters/xpetra/MueLu_XpetraOperator_decl.hpp:53,
                 from /*/Trilinos-trilinos-release-13-2-0/packages/muelu/adapters/stratimikos/Thyra_XpetraLinearOp_def.hpp:54,
                 from /*/Trilinos-trilinos-release-13-2-0/packages/muelu/adapters/stratimikos/Thyra_XpetraLinearOp.hpp:2,
                 from /*/Trilinos-trilinos-release-13-2-0/packages/muelu/adapters/stratimikos/Thyra_MueLuPreconditionerFactory_decl.hpp:58,
                 from /*/Trilinos-trilinos-release-13-2-0/packages/muelu/adapters/stratimikos/Thyra_MueLuPreconditionerFactory_def.hpp:49,
                 from /*/trilinos-release-13-2-0/packages/muelu/adapters/ExplicitInstantiation/Thyra_MueLuPreconditionerFactory.cpp:52:
/*/Trilinos-trilinos-release-13-2-0/packages/thyra/adapters/epetra/src/Thyra_EpetraThyraWrappers.hpp:279:1: note: candidate: ‘Teuchos::RCP<const Epetra_Map> Thyra::get_Epetra_Map(const Thyra::VectorSpaceBase<double>&, const Teuchos::RCP<const Epetra_Comm>&)’
  279 | get_Epetra_Map(const VectorSpaceBase<double>& vs, const RCP<const Epetra_Comm>& comm);
      | ^~~~~~~~~~~~~~
/*/Trilinos-trilinos-release-13-2-0/packages/thyra/adapters/epetra/src/Thyra_EpetraThyraWrappers.hpp:279:47: note:   no known conversion for argument 1 from ‘const Thyra::VectorSpaceBase<float>’ to ‘const Thyra::VectorSpaceBase<double>&’
  279 | get_Epetra_Map(const VectorSpaceBase<double>& vs, const RCP<const Epetra_Comm>& comm);
      |                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~
/*/Trilinos-trilinos-release-13-2-0/packages/thyra/adapters/epetra/src/Thyra_EpetraThyraWrappers.hpp:284:1: note: candidate: ‘Teuchos::RCP<const Epetra_Map> Thyra::get_Epetra_Map(const Teuchos::RCP<const Thyra::VectorSpaceBase<double> >&, const Teuchos::RCP<const Epetra_Comm>&)’
  284 | get_Epetra_Map(const RCP<const VectorSpaceBase<double>>& vs,
      | ^~~~~~~~~~~~~~
/*/Trilinos-trilinos-release-13-2-0/packages/thyra/adapters/epetra/src/Thyra_EpetraThyraWrappers.hpp:284:58: note:   no known conversion for argument 1 from ‘const Thyra::VectorSpaceBase<float>’ to ‘const Teuchos::RCP<const Thyra::VectorSpaceBase<double> >&’
  284 | get_Epetra_Map(const RCP<const VectorSpaceBase<double>>& vs,
      |                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~
In file included from /*/trilinos-release-13-2-0/packages/muelu/adapters/ExplicitInstantiation/Thyra_MueLuPreconditionerFactory.cpp:52:
/*/Trilinos-trilinos-release-13-2-0/packages/muelu/adapters/stratimikos/Thyra_MueLuPreconditionerFactory_def.hpp:208:70: error: call of overloaded ‘get_Epetra_Vector(const Epetra_Map&, Teuchos::RCP<const Thyra::VectorBase<float> >&)’ is ambiguous
  208 |             RCP<const Epetra_Vector> eDiag = Thyra::get_Epetra_Vector(*map, diag);
      |                                              ~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~
In file included from /*/Trilinos-trilinos-release-13-2-0/packages/xpetra/sup/Utils/Xpetra_ThyraUtils.hpp:99,
                 from /*/Trilinos-trilinos-release-13-2-0/packages/xpetra/src/BlockedCrsMatrix/Xpetra_BlockedCrsMatrix.hpp:79,
                 from /*/Trilinos-trilinos-release-13-2-0/packages/xpetra/sup/Utils/Xpetra_MatrixMatrix.hpp:53,
                 from /*/Trilinos-trilinos-release-13-2-0/packages/muelu/src/Interface/../Utils/MueLu_Utilities_decl.hpp:76,
                 from /*/trilinos-release-13-2-0/packages/muelu/src/MueLu_Utilities.hpp:1,
                 from /*/Trilinos-trilinos-release-13-2-0/packages/muelu/src/Interface/../MueCentral/MueLu_Level.hpp:68,
                 from /*/Trilinos-trilinos-release-13-2-0/packages/muelu/adapters/xpetra/MueLu_XpetraOperator_decl.hpp:53,
                 from /*/Trilinos-trilinos-release-13-2-0/packages/muelu/adapters/stratimikos/Thyra_XpetraLinearOp_def.hpp:54,
                 from /*/Trilinos-trilinos-release-13-2-0/packages/muelu/adapters/stratimikos/Thyra_XpetraLinearOp.hpp:2,
                 from /*/Trilinos-trilinos-release-13-2-0/packages/muelu/adapters/stratimikos/Thyra_MueLuPreconditionerFactory_decl.hpp:58,
                 from /*/Trilinos-trilinos-release-13-2-0/packages/muelu/adapters/stratimikos/Thyra_MueLuPreconditionerFactory_def.hpp:49,
                 from /*/trilinos-release-13-2-0/packages/muelu/adapters/ExplicitInstantiation/Thyra_MueLuPreconditionerFactory.cpp:52:
/*/Trilinos-trilinos-release-13-2-0/packages/thyra/adapters/epetra/src/Thyra_EpetraThyraWrappers.hpp:310:1: note: candidate: ‘Teuchos::RCP<Epetra_Vector> Thyra::get_Epetra_Vector(const Epetra_Map&, const Teuchos::RCP<Thyra::VectorBase<double> >&)’
  310 | get_Epetra_Vector(
      | ^~~~~~~~~~~~~~~~~
/*/Trilinos-trilinos-release-13-2-0/packages/thyra/adapters/epetra/src/Thyra_EpetraThyraWrappers.hpp:344:1: note: candidate: ‘Teuchos::RCP<const Epetra_Vector> Thyra::get_Epetra_Vector(const Epetra_Map&, const Teuchos::RCP<const Thyra::VectorBase<double> >&)’
  344 | get_Epetra_Vector(
      | ^~~~~~~~~~~~~~~~~
make[2]: *** [packages/muelu/adapters/CMakeFiles/muelu-adapters.dir/build.make:132: packages/muelu/adapters/CMakeFiles/muelu-adapters.dir/ExplicitInstantiation/Thyra_MueLuPreconditionerFactory.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....
make[1]: *** [CMakeFiles/Makefile2:9587: packages/muelu/adapters/CMakeFiles/muelu-adapters.dir/all] Error 2
make: *** [Makefile:166: all] Error 2
Failure with exit status: 2
Exit message: There was a problem building trilinos 13-2-0.

Configure 2:

  -D Tpetra_INST_INT_LONG_LONG=OFF
  -D Tpetra_INST_INT_INT=ON

It compiled and installed both Trilinos and deal.ii smoothly.

[jpthiele]

Ah perfect, then I misread your previous post to it not actually working.
But finding out that MueLU seems to have a problem when Tpetra is build with float and Epetra is also build is interesting.

[LemonBoy68]

/media/softwares/Aspect/dealii-candi-oneapi1/trilinos-release-13-2-0/lib/libstratimikosbelos.so: error: error on 'Tpetra::MultiVector<float, int, long long, Kokkos::Compat: :KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace> >::getVector(unsigned long) const'undefined reference
/media/softwares/Aspect/dealii-candi-oneapi1/trilinos-release-13-2-0/lib/libstratimikosbelos.so: error: error on 'Tpetra::MultiVector<float, int, long long, Kokkos::Compat: :KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace> >::getLocalLength() const' undefined reference
/media/softwares/Aspect/dealii-candi-oneapi1/trilinos-release-13-2-0/lib/libstratimikosbelos.so: error: error on 'Tpetra::MultiVector<float, int, long long, Kokkos::Compat: :KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace> >::getGlobalLength() const'undefined reference
/media/softwares/Aspect/dealii-candi-oneapi1/trilinos-release-13-2-0/lib/libstratimikosbelos.so: error: error on 'Tpetra::MultiVector<float, int, long long, Kokkos::Compat: :KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace> >::getNumVectors() const'undefined reference
/media/softwares/Aspect/dealii-candi-oneapi1/trilinos-release-13-2-0/lib/libstratimikosbelos.so: error: error on 'Tpetra::MultiVector<float, int, long long, Kokkos::Compat: :KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace> >::get1dCopy(Teuchos::ArrayView const&, unsigned long) const'undefined reference
collect2: error: ld returned 1 exit status
gmake[2]: *** [CMakeFiles/CheckCompilerSetupExec.dir/build.make:272: CheckCompilerSetupExec] Error 1
gmake[2]: Leaving directory "/media/softwares/Aspect/dealii-candi-oneapi1/tmp/build/deal.II-v9.5.1/check_compiler_setup/DEAL_II_HAVE_USABLE_FLAGS_DEBUG"
gmake[1]: *** [CMakeFiles/Makefile2:86: CMakeFiles/CheckCompilerSetupExec.dir/all] Error 2
gmake[1]: Leaving directory "/media/softwares/Aspect/dealii-candi-oneapi1/tmp/build/deal.II-v9.5.1/check_compiler_setup/DEAL_II_HAVE_USABLE_FLAGS_DEBUG"
gmake: ***[Makefile:94:all] Error 2