Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bug: GSI2DART MPI implementation #633

Open
mjs2369 opened this issue Feb 6, 2024 · 0 comments
Open

bug: GSI2DART MPI implementation #633

mjs2369 opened this issue Feb 6, 2024 · 0 comments
Labels
Bug Something isn't working obs_converters converting observations to DART format

Comments

@mjs2369
Copy link
Contributor

mjs2369 commented Feb 6, 2024

Describe the bug

The GSI2DART observation converter will not compile with GNU Fortran 11.2.0 (or later) due to several issues related to MPI:

  • Rank and type mismatches in mpi_utilities_mod.f90
  • Type mismatches in mpi_readobs.f90

These GSI2DART bugs were removed from the Fortran standards pull request #619 (and issue #352) as they are deeper than just a standards issue. In general, the MPI implementation in this code will need all together refactored. This code makes use of code from both mpi_utilities_mod.f90 and its own mpisetup.f90 and calls 2 different MPI initialization subroutines (mpi_initialize and initialize_mpi_utilities).

mpisetup.f90 still uses include 'mpif.h' as well, which should not be used alongside the use of the mpi module in mpi_utilities_mod.f90. In general, using the mpif.h include file is outdated

  1. List the steps someone needs to take to reproduce the bug.
    Run quickbuild.sh with GNU Fortran (GCC) 11.2.0 or later in GSI2DART/work/
  2. What was the expected outcome?
    Program gsi_to_dart compiles successfully
  3. What actually happened?
    Errors out

Error Message

DART/assimilation_code/modules/utilities/mpi_utilities_mod.f90:

/glade/work/masmith/DART/assimilation_code/modules/utilities/mpi_utilities_mod.f90:1964:13:

 1964 | call mpi_get(x, 1, datasize, owner, target_disp, 1, datasize, window, errcode)
      |             1
......
 1988 | call mpi_get(x, num_rows, datasize, owner, target_disp, num_rows, datasize, window, errcode)
      |             2
Error: Rank mismatch between actual argument at (1) and actual argument at (2) (rank-1 and scalar)
/glade/work/masmith/DART/assimilation_code/modules/utilities/mpi_utilities_mod.f90:1502:33:

 1502 | call mpi_allreduce(MPI_IN_PLACE, min_var, num_elements, datasize, MPI_MIN, get_dart_mpi_comm(), errcode)
      |                                 1
......
 2011 | call mpi_allreduce(MPI_IN_PLACE, max, 1, datasize, MPI_MAX, my_local_comm, errcode)
      |                                 2
Error: Rank mismatch between actual argument at (1) and actual argument at (2) (scalar and rank-1)
/glade/work/masmith/DART/assimilation_code/modules/utilities/mpi_utilities_mod.f90:1503:33:

 1503 | call mpi_allreduce(MPI_IN_PLACE, max_var, num_elements, datasize, MPI_MAX, get_dart_mpi_comm(), errcode)
      |                                 1
......
 2011 | call mpi_allreduce(MPI_IN_PLACE, max, 1, datasize, MPI_MAX, my_local_comm, errcode)
      |                                 2
Error: Rank mismatch between actual argument at (1) and actual argument at (2) (scalar and rank-1)
/glade/work/masmith/DART/assimilation_code/modules/utilities/mpi_utilities_mod.f90:1410:19:

 1410 | call MPI_Allreduce(localaddend, localsum, 1, datasize, MPI_SUM, &
      |                   1
......
 2011 | call mpi_allreduce(MPI_IN_PLACE, max, 1, datasize, MPI_MAX, my_local_comm, errcode)
      |                   2
Error: Type mismatch between actual argument at (1) and actual argument at (2) (REAL(8)/INTEGER(4)).
/glade/work/masmith/DART/assimilation_code/modules/utilities/mpi_utilities_mod.f90:1410:32:

 1410 | call MPI_Allreduce(localaddend, localsum, 1, datasize, MPI_SUM, &
      |                                1
......
 2011 | call mpi_allreduce(MPI_IN_PLACE, max, 1, datasize, MPI_MAX, my_local_comm, errcode)
      |                                 2

There are many more errors in addition to these.

GSI2DART/enkf/mpi_readobs.f90:

/glade/scratch/hkershaw/DART/Bugs/mismatch/DART/observations/obs_converters/GSI2DART/enkf/mpi_readobs.f90:172:19:

   74 |     call mpi_bcast(nobs_conv,1,mpi_integer,0,mpi_comm_world,ierr)
      |                   2
......
  172 |     call mpi_bcast(ensmean_ob,nobs_tot,mpi_real4,0,mpi_comm_world,ierr)
      |                   1
Error: Type mismatch between actual argument at (1) and actual argument at (2) (REAL(4)/INTEGER(4)).
/glade/scratch/hkershaw/DART/Bugs/mismatch/DART/observations/obs_converters/GSI2DART/enkf/mpi_readobs.f90:173:19:

   74 |     call mpi_bcast(nobs_conv,1,mpi_integer,0,mpi_comm_world,ierr)
      |                   2
......
  173 |     call mpi_bcast(sprd_ob,nobs_tot,mpi_real4,0,mpi_comm_world,ierr)
      |                   1
Error: Type mismatch between actual argument at (1) and actual argument at (2) (REAL(4)/INTEGER(4)).
Makefile:10: recipe for target 'mpi_readobs.o' failed

Which model(s) are you working with?

GSI2DART observation converter

Screenshots

If applicable, add screenshots to help explain your problem.

Version of DART

v11.0.2

Have you modified the DART code?

No

Build information

Please describe:

  1. The machine you are running on (e.g. windows laptop, NCAR supercomputer Cheyenne).
  2. The compiler you are using (e.g. gnu, intel).

All machines, GNU Fortran (versions 11.2.0 or later)

@hkershaw-brown hkershaw-brown added Bug Something isn't working obs_converters converting observations to DART format labels May 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Something isn't working obs_converters converting observations to DART format
Projects
None yet
Development

No branches or pull requests

2 participants