Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Calls to Kokkos::min and Kokkos:max are incorrect if ENABLE_KOKKOS=On #323

Open
dogunter opened this issue Apr 13, 2023 · 10 comments · May be fixed by #328
Open

Calls to Kokkos::min and Kokkos:max are incorrect if ENABLE_KOKKOS=On #323

dogunter opened this issue Apr 13, 2023 · 10 comments · May be fixed by #328

Comments

@dogunter
Copy link

I've been trying to build RAJAPerf with the cmake option ENABLE_KOKKOS=On but it fails with two error messages,

"/home/dog/kokkos/testing/RAJAPerf/src/basic-kokkos/REDUCE3_INT-Kokkos.cpp", line 58: error: namespace "Kokkos" has no member "min"
        m_vmin = Kokkos::min(m_vmin, static_cast<Int_type>(min_value));
                         ^

"/home/dog/kokkos/testing/RAJAPerf/src/basic-kokkos/REDUCE3_INT-Kokkos.cpp", line 59: error: namespace "Kokkos" has no member "max"
        m_vmax = Kokkos::max(m_vmax, static_cast<Int_type>(max_value));
                         ^

Steps to reproduce:

$ module load cmake
$ module load nvhpc/23.1
$ git clone --recursive https://github.com/LLNL/RAJAPerf.git
$ cd RAJAPerf
$ mkdir build; cd build
$ cmake   -DENABLE_KOKKOS=On ..
$ make -j

It appears to be a simple case issue. The correct call should be to Kokkos::Min and Kokkos:Max, respectively.

@rhornung67
Copy link
Member

pnig @ajpowelsnl

@dogunter
Copy link
Author

I will. I've discovered a more serious flaw in that Kokkos::Min and Kokkos::Max aren't being used as Kokkos defined them.

@ajpowelsnl
Copy link
Contributor

Hi @dogunter and @rhornung67 -- Many thanks for the heads up. I'll look into this.

@ajpowelsnl
Copy link
Contributor

Hi @dogunter and @rhornung67 -- I poked around with this for a couple of hours, and I think the issue might be that you need to update your Kokkos (in $RAJAPERF_ROOT/tpl/kokkos). The code in REDUCE3_INT looks good, and I believe the problems you're having are possibly an effect of somewhat recent namespace changes for Kokkos::min, Kokkos::max.

I would suggest trying the following:

  1. in $RAJAPERF_ROOT:
git submodule update --recursive --init
  1. cd $RAJAPERF_ROOT/tpl/kokkos

  2. git checkout develop

  3. git pull

  4. cd $RAJAPERF_ROOT

  5. mkdir build_kokkos

  6. make yourself a "do_cmake" build script in $RAJAPERF_ROOT

# Use <= clang-12 + cuda-11-ish
cmake \
-DENABLE_KOKKOS=ON \
-DKOKKOS_ENABLE_CXX17=OFF \
-DCMAKE_CXX_COMPILER=clang++ or g++  \
-DRAJA_PERFSUITE_ENABLE_TESTS=OFF \
-DENABLE_CUDA=ON \
-DKokkos_ARCH_VOLTA70=ON \
-DCMAKE_CUDA_ARCHITECTURES=70 \
-DCMAKE_BUILD_TYPE=Release ..
  1. cd build_kokkos
  2. ../do_cmake to configure
  3. make -j20 to build

LMK if this suggestion works. We'll continue to troubleshoot if it doesn't. Apologies in advance for the delayed response.

@rhornung67
Copy link
Member

@ajpowelsnl you should be able to put up a PR to address this issue. RAJA folks are not maintaining the Kokkos part.

@ajpowelsnl
Copy link
Contributor

@ajpowelsnl you should be able to put up a PR to address this issue. RAJA folks are not maintaining the Kokkos part.

OK, I need to figure out how to make git submodule pull in Kokkos develop.

@rhornung67
Copy link
Member

Check out the RAJAPerf develop branch and make sure all submodules are updated recursively. Then make a new branch off that to submit a PR. Go into the Kokkos submodule and checkout the develop branch. Commit that change and make a PR.

@ajpowelsnl
Copy link
Contributor

Check out the RAJAPerf develop branch and make sure all submodules are updated recursively. Then make a new branch off that to submit a PR. Go into the Kokkos submodule and checkout the develop branch. Commit that change and make a PR.

What I would like (I think) is for Kokkos to behave like the other submodules (such that git submodule update --recursive --init updated a specific branch, develop, ideally, all in one swell foop), so folks wouldn't run into these problems. IOW, make this all automatic under the hood.

@rhornung67
Copy link
Member

The root cause of the issue that was reported was that the RAJAPerf code was changed but the kokkos submodule version was not changed to be consistent with that change, I believe.

Running 'git submodule update' as you describe will update each submodule, including submodules of submodules to the specific commit hash, tag, etc. in the associated submodule project repo that the version of RAJAPerf is pinned to. Each submodule has to be updated if you want a new version.

We don't want everything pinned to the latest develop branch of each submodule project. I don't think you can even do that with git submodules.

@ajpowelsnl ajpowelsnl linked a pull request May 22, 2023 that will close this issue
@ajpowelsnl
Copy link
Contributor

Hi @rhornung67 and @dogunter -- I've submitted a #328 with the requested update. LMK if this does not fix you. Apologies for the delay.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants