Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Compiler cmake issue #245

Draft
wants to merge 3 commits into
base: master
Choose a base branch
from
Draft

Compiler cmake issue #245

wants to merge 3 commits into from

Conversation

koecher
Copy link
Contributor

@koecher koecher commented Jul 9, 2021 •
edited

This is a PR Draft to figure out the problems we have with setting the compiler and cmake

Updated packages:

  • Trilinos 13.0.1
  • (optional SuperLU_dist 6.4.0 seems to be working with trilinos 13.0.1)

Developments

  • New detection of BLAS, LAPACK, MKL, openBLAS in trilinos.package
  • removes explicit compiler settings for cmake in trilinos.package

Successful tests on freshly set up virtual machines with the recommended installation instructions
@koecher

  • ubuntu 21.04, gcc-10, openmpi from OS, cmake 3.18 from OS, candi sets CC,CXX,etc to mpi compilers
  • ubuntu 21.04, gcc-10, openmpi from OS, cmake 3.20.5 from candi (unrelated superlu problem for deal.II / linking blas)
  • CI / ubuntu 18.04 minimal (pull_request) Successful
  • CI / ubuntu 20.04 (pull_request) Successful
  • centos 7, gcc-8.1.0/mpich-3.2 self-compiled with all env. variables set, cmake 3.20.5 from candi, Successful
  • ubuntu 20.04 (LTS), gcc-9, openmpi from OS, cmake 3.16.3 from OS, Successful
  • ubuntu 20.04 (LTS), gcc-9, openmpi from OS, cmake 3.20.5 from candi, Successful
  • ubuntu 20.04 (LTS), gcc-10 alternative, openmpi from OS, cmake 3.16.3 from OS
  • ubuntu 20.04 (LTS), gcc-10 alternative, openmpi from OS, cmake 3.20.5 from candi
  • opensuse 15 / Leap, gcc-7.5, mpich from OS, cmake 3.17.0 from OS, Successful
  • CI / OSX gcc (pull_request) Has Failure in a test after install, seems unrelated
  • CI / OSX clang (pull_request), Successful
  • macOS 11.4.1 (bigsur), gcc-11, openmpi, cmake 3.21.0 from homebrew, Successful

@gfcas

Clean ups and TODOs

  • remove local.cfg commit
  • Trilinos configuration variables in candi
  • BLAS and LAPACK dirs
  • TODOs in trilinos.package
  • SuperLU_dist BLAS/LAPACK linking problem (only for v7.0.0), resolved

@koecher koecher requested a review from tjhei July 9, 2021 17:54
@koecher koecher self-assigned this Jul 9, 2021
@koecher
Copy link
Contributor Author

koecher commented Jul 9, 2021 •
edited

Related to #207 #159

@koecher
Copy link
Contributor Author

koecher commented Jul 9, 2021

@gfcas can you test this branch on your machines?

This was linked to issues Jul 9, 2021
cmake 3.20 MPI compiler selection broken #244
Closed
configuration with Intel compilers is broken #202
Open
@koecher koecher mentioned this pull request Jul 9, 2021
Merged
@ghost
Copy link

ghost commented Jul 12, 2021

@gfcas can you test this branch on your machines?

@koecher of course, I will also reconfigure the github actions runners. On my personal laptop maybe it will take some days because I got a new one.

@ghost
Copy link

ghost commented Jul 12, 2021

I think it very good that we update the Trilinos package, which is one of the major ones for parallel computing. Should we include also Umfpack and Mumps as described in the dealii readme (https://www.dealii.org/current/external-libs/trilinos.html). For my own work I was recently interested in the trilinos subpackage ShyLU.

@koecher
Copy link
Contributor Author

koecher commented Jul 12, 2021

I think it very good that we update the Trilinos package, which is one of the major ones for parallel computing. Should we include also Umfpack and Mumps as described in the dealii readme (https://www.dealii.org/current/external-libs/trilinos.html). For my own work I was recently interested in the trilinos subpackage ShyLU.

Let us concentrate on Trilinos and deal.II for now to find a solution to the problem

@koecher koecher force-pushed the compiler-cmake-issue branch 2 times, most recently from 7cb5393 to 714625c Compare July 12, 2021 11:40
@koecher
Copy link
Contributor Author

koecher commented Jul 12, 2021

@gfcas no reviews so far, everything is draft for testing

@ghost
Copy link

ghost commented Jul 12, 2021 •
edited by ghost

@koecher:

  • Ubuntu 18.04 (LTS), gcc-7.5, openmpi from OS, cmake 3.20.5 from OS
  • Ubuntu 20.04 (LTS), gcc-9.3, openmpi from OS, cmake 3.20.5 from OS
  • Ubuntu 20.04 (LTS), gcc-10.3, openmpi from OS, cmake 3.20.5 from OS
    just succeded, see https://github.com/gfcas/candi/actions/runs/1022810879

@koecher koecher force-pushed the compiler-cmake-issue branch 3 times, most recently from e2baaad to 79a2ea1 Compare July 12, 2021 15:03
@dealii dealii deleted a comment Jul 12, 2021
@dealii dealii deleted a comment Jul 12, 2021
@dealii dealii deleted a comment Jul 12, 2021
@dealii dealii deleted a comment Jul 12, 2021
@dealii dealii deleted a comment Jul 12, 2021
@dealii dealii deleted a comment Jul 12, 2021
@dealii dealii deleted a comment Jul 12, 2021
@koecher
Copy link
Contributor Author

koecher commented Jul 12, 2021 •
edited

I've cleaned up the discussion. Please only report on the compiler-cmake issue. SuperLU is not of interest anymore here

@koecher koecher mentioned this pull request Jul 19, 2021
Merged
@ghost
Copy link

ghost commented Jul 28, 2021

In order to get this issue ready for v9.3.1 maybe we should consistently test it with DEAL_II_VERSION=v9.3.1?

@koecher
Copy link
Contributor Author

koecher commented Jul 28, 2021

I think we need a test for intel compilers, the other systems look good.

Testing with the fixed v9.3.1 is okay.

@koecher
Copy link
Contributor Author

koecher commented Jul 29, 2021

Remarks for macOS:

Packages:

  • parmetis 4.0.3
  • hdf5 1.10.7
  • superlu_dist 6.4.0
  • p4est 2.2
  • trilinos 13.0.1 (without compiler settings, without -lgfortran)
  • deal.II v9.3.1 (without compiler settings)

Special configuration: LC_ALL=C and unset LANGUAGE since step-32 doesn't work without it

export OMPI_CC=gcc-11
export OMPI_CXX=g++-11
export OMPI_FC=gfortran-11
export CC=mpicc
export CXX=mpicxx
export FC=mpifort
export FF=mpifort
export LC_ALL=C
unset LANGUAGE

Tested step tutorials

  • step-1
  • step-7
  • step-29 (using deal.II complex and umfpack as direct solver)
  • step-32 (needs LC_ALL=C)
  • step-33
  • step-40 (only Trilinos installed)
  • step-72
  • hdf5 file output with own software

Additional system preferences: (if dylib library not found, e.g. for TrilinosWrappers::SolverDirect)

  • System Preferences / Security & Privacy / Developer Tools / Terminal

@koecher
superlu_dist: v6.4.0 as 7.0.0 is not working
6ae1052 
superlu_dist v6.4.0: adds checksum and cleanups
superlu_dist from github repo as tarball
superlu_dist: version 7.0.0
minor styling

superlu_dist: v7.0.0 tarball from github

superlu_dist: add comment for VERSION=7.0.0
@koecher
local.cfg (debug commit)
0dcaff3 
remove superlu
@koecher
trilinos: version 13.0.1 and simplified Compiler settings
3561c6f 
trilinos follow up
trilinos: mayor update for BLAS/LAPACK/openBLAS/MKL
trilinos 13.0.1: adds checksum and completes superlu_dist v6.4.0 integration
trilinos: removes -lgfortran (already disabled)

trilinos: rebasing corrections
@tjhei
Copy link
Member

tjhei commented Jul 29, 2021

Do you think it is a good time to go to Trilinos 13 at this point? I am not sure we have tested it much with deal.ii.

@koecher
Copy link
Contributor Author

koecher commented Jul 29, 2021

Do you think it is a good time to go to Trilinos 13 at this point? I am not sure we have tested it much with deal.ii.

Well this was a try if things are going well so far. I think we should test the parallel features of trilinos 13 a little more.
From the current tests on macos, I'm fine with trilinos 13.

@koecher
Copy link
Contributor Author

koecher commented Jul 29, 2021 •
edited

@tjhei do you have the chance to test this with intel compilers?

@tjhei
Copy link
Member

tjhei commented Jul 29, 2021

Deal.II currently suggests using 12.18, see https://github.com/dealii/dealii/blob/94b2450484e130a74051c7bd7230c8d6f79b98b2/doc/external-libs/trilinos.html#L53
I don't think it is a good idea to go to 13 right now, especially because it does not fix any problem, or does it?

What do you want to me to test, just run with default settings with Intel and see if things compile?

@koecher
Copy link
Contributor Author

koecher commented Jul 29, 2021

Deal.II currently suggests using 12.18, see https://github.com/dealii/dealii/blob/94b2450484e130a74051c7bd7230c8d6f79b98b2/doc/external-libs/trilinos.html#L53
I don't think it is a good idea to go to 13 right now, especially because it does not fix any problem, or does it?

What do you want to me to test, just run with default settings with Intel and see if things compile?

  • can you test if this PR is running smoothly? Here is a local.cfg to set the packages.
  • I suggest to stay with trilinos 12 for deal.II v9.3 and
  • introduce an additional trilinos13.package for master

This PR isn't useful for a merge, the things we learned here should be used for clean PRs in future

@dealii dealii deleted a comment Jul 29, 2021
@dealii dealii deleted a comment Jul 29, 2021
@dealii dealii deleted a comment Jul 29, 2021
@tjhei
Copy link
Member

tjhei commented Jul 29, 2021

@zjiaqi2018 can you please check out this PR of candi on palmetto or frontera and compile with current Intel compilers and MKL? Please set MKL=ON in the local.cfg. Let us know what changes you need to do or if you get any errors.

@zjiaqi2018
Copy link

@zjiaqi2018 can you please check out this PR of candi on palmetto or frontera and compile with current Intel compilers and MKL? Please set MKL=ON in the local.cfg. Let us know what changes you need to do or if you get any errors.

I tested it on frontera, and it seems to stop at:
image

@zjiaqi2018
Copy link

@zjiaqi2018 can you please check out this PR of candi on palmetto or frontera and compile with current Intel compilers and MKL? Please set MKL=ON in the local.cfg. Let us know what changes you need to do or if you get any errors.

I tested it on frontera, and it seems to stop at:
image

with -D DEAL_II_COMPONENT_EXAMPLES=OFF, it doesn't work either:
image

@tjhei
Copy link
Member

tjhei commented Jul 30, 2021

Is this on Frontera? Rene had to disable gold linker, see "-fuse" in https://github.com/geodynamics/aspect/wiki/Installation-on-Frontera

@tjhei
Copy link
Member

tjhei commented Jul 30, 2021

Can you post the deal.ii summary.log and your changes to the settings once it works?

@zjiaqi2018
Copy link

zjiaqi2018 commented Jul 30, 2021 •
edited

It works now. For the local changes, I just follow Rene's instructions you mentioned.
image

@koecher koecher linked an issue Sep 14, 2021 that may be closed by this pull request
Trilinos 12.18.1 and make 4.3 combination fails #252
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tjhei tjhei Awaiting requested review from tjhei

Assignees

@koecher koecher

Labels
bug enhancement help wanted version update updates the version of a certain package work in progress
Projects
None yet
Milestone
No milestone
3 participants
@koecher @tjhei @zjiaqi2018