WIP: Experimental fpm build #753

rouson · 2022-04-23T21:50:14Z

coverage on master

Summary of changes

Greatly restructure/simplify the directory structure and remove unsupported legacy files, e.g., GASNet-1 layer.
Add fpm manifist: fpm.toml.
Add run-fpm.sh script for building with fpm.

To Do

Reconcile the file system restructuring with the windows-support branch.
Replace the install.sh with an installer based the [Caffeine] installer and the new run-fpm.sh script.

Rationale for changes

The Fortran Package Manager has grown greatly in popularity and an fpm manifest so simple to maintain that it's reasonable to support two build systems.
Homebrew has become sufficiently stable on macOS and Linux that it will be much easier to maintain a homebred-based installer than the current homegrown-package-management installer.

Additional info and certifications

This pull request (PR) is a:

Bug fix
Feature addition
Other, Please describe:

I certify that

I certify that:
- I have reviewed and followed the contributing guidelines
- I will wait at least 24 hours before self-approving the PR to give another
  OpenCoarrays developer a chance to review my proposed code
- I have not introduced errant white space (no trailing white space or white space errors may
  be introduced)
- I have added an explanation of what these changes do and why they should be included
- I have checked to ensure there aren't other open Pull Requests for the same change
- I have you written new tests for these changes
- I have successfully tested these changes locally
- I have commented any non-trivial, non-obvious code changes
- The commits are logically atomic, self consistent and coherent
- The commit messages follow best practices
- Test coverage is maintained or increased after this is merged

Code coverage data

git mv test tests

The script 1. Installs OpenMPI via homebrew if mpifort is missing. 2. Builds libopencoarrays.a library. 3. Runs a parallel "Hello, world!" program in four images.

This commit eliminates the hardwired, separate build and run steps in run-fpm.sh and instead passes an command-line arguments along to fpm. The following now works even with no prior build directory present: ./run-fpm.sh --example hello

everythingfunctional

I'm not sure I understand the CMake changes, but I made some comments on the install script.

everythingfunctional · 2022-04-25T16:22:42Z

install.sh

+    echo "For a non-interactive build with the 'yes' utility installed, execute"
+    echo "yes | ./install.sh"


I don't think I'd include this. There are too many scenarios where this won't work. For example, if homebrew asks for sudo access.

@everythingfunctional I was going to remove this, but decided to keep it because it relates to a user request. I think I can borrow code from the original installer to cover the case in which sudo is required.

everythingfunctional · 2022-04-25T16:25:18Z

install.sh

+  --c-flag "-DPREFIX_NAME=_gfortran_caf_ -DGCC_GE_7 -DGCC_GE_8" \
+  --flag "-fcoarray=lib"
+
+CAF_VERSION=$(sed -n '/[0-9]\{1,\}\(\.[0-9]\{1,\}\)\{1,\}/{s/^\([^.]*\)\([0-9]\{1,\}\(\.[0-9]\{1,\}\)\{1,\}\)\(.*\)/\2/p;q;}' .VERSION)


This regular expression deserves an explanatory comment I think.

install.sh

everythingfunctional · 2022-04-25T17:26:17Z

install.sh

+  if [ -z ${MPIFC+x} ] || [ -z ${MPICC+x} ]; then
+    ask_permission_to_install_homebrew_package "'mpifort' and 'mpicc'" "open-mpi"
+    exit_if_user_declines "mpifort and mpicc"
+    "$BREW" install gcc@$GCC_VERSION


I get the following error from mpicc installed with homebrew on Linux

-------------------------------------------------------------------------- The Open MPI wrapper compiler was unable to find the specified compiler gcc-5 in your PATH. Note that this compiler was either specified at configure time or in one of several possible environment variables. -------------------------------------------------------------------------- <ERROR> Compilation failed for object " example_hello.f90.o " <ERROR>stopping due to failed compilation

install.sh

everythingfunctional · 2022-04-25T17:51:54Z

install.sh

+$FPM install \
+  --compiler $FPM_FC \
+  --c-compiler $FPM_CC \
+  --c-flag "-DPREFIX_NAME=_gfortran_caf_ -DGCC_GE_7 -DGCC_GE_8" \
+  --flag "-fcoarray=lib"


I get the following error from this on Linux.

+ /usr/bin/mpicc -c ./src/mpi-runtime/mpi_caf.c -DPREFIX_NAME=_gfortran_caf_ -DGCC_GE_7 -DGCC_GE_8 -o build/mpifort_EA4E6F5B6A428512/opencoarrays/src_mpi-runtime_mpi_caf.c.o ./src/mpi-runtime/mpi_caf.c:46:10: fatal error: mpi.h: No such file or directory 46 | #include <mpi.h> | ^~~~~~~ compilation terminated.

everythingfunctional · 2022-04-25T17:53:46Z

install.sh

+sed -i '' -e "s/@Fortran_COMPILER@/$Fortran_COMPILER/g" "$PREFIX"/bin/caf
+sed -i '' -e "s/@CAF_LIBS@/lib\/libopencoarrays.a/g" "$PREFIX"/bin/caf


I'm pretty sure these sed commands have to have -i'' (without the space) or just -i on Linux.

everythingfunctional · 2022-04-25T17:58:24Z

run-fpm.sh

+fpm $@ \
+  --compiler `which mpifort` \
+  --c-compiler `which mpicc` \
+  --c-flag "-DPREFIX_NAME=_gfortran_caf_ -DGCC_GE_7 -DGCC_GE_8" \
+  --flag "-fcoarray=lib" \
+  --runner "mpirun -n 4"


Is the idea that this is the fpm command to be used if one wants to make use of OpenCoarrays? If so, shouldn't there be a flag that points to the installed opencoarrays library?, or is the idea that it would be listed in the project's fpm.toml file and thus downloaded and compiled by fpm with those flags?

Co-authored-by: Brad Richardson <everythingfunctional@protonmail.com>

everythingfunctional · 2022-05-02T14:26:21Z

test/this_image_test.f90

+          image_numbers = [(merge(0, me, me/=i), i = 1, ni)]
+          call co_sum(image_numbers)
+          result_ = assert_that(all(image_numbers == [(i, i = 1, ni)]) .and. size(image_numbers)>0, "correct image set")


This is a "clever trick", and as such deserves better explanation. There might be a way to make the code more clear, but I'd settle for an explanatory comment.

rouson added 9 commits March 22, 2022 20:07

feat: rm installation script & supporting scripts

9275b82

refac: mv tests to fpm-defualt location

040f5fa

rm legacy gasnet-1 layer

cd15f1b

rm static Makefile/make.inc files

9b88425

rm iso-fortran-bindin dir

574c60d

rm opencoarrays.F90 wrappers

1198df8

temporarily hide tests from fpm

b4ad48d

git mv test tests

build(fpm): add experimental fpm build

cda9e57

feat(fpm): add experimental run-fpm.sh script

fddc89b

The script 1. Installs OpenMPI via homebrew if mpifort is missing. 2. Builds libopencoarrays.a library. 3. Runs a parallel "Hello, world!" program in four images.

rouson requested review from zbeekman and everythingfunctional April 23, 2022 21:50

rouson added 4 commits April 24, 2022 12:44

feat(fpm): run-fpm.sh accepts arguments

1935b64

This commit eliminates the hardwired, separate build and run steps in run-fpm.sh and instead passes an command-line arguments along to fpm. The following now works even with no prior build directory present: ./run-fpm.sh --example hello

feat(fpm): add library installation to manifest

e70895f

feat(fpm): add install.sh

85c699f

WIP: customize caf script

74d7313

everythingfunctional reviewed Apr 25, 2022

View reviewed changes

rouson and others added 14 commits April 26, 2022 07:02

Update install.sh

5983fa9

Co-authored-by: Brad Richardson <everythingfunctional@protonmail.com>

fix(install.sh): rm typ in mpicc invocation

33d2f01

Co-authored-by: Brad Richardson <everythingfunctional@protonmail.com>

feat(install.sh): write working caf script

b9f8ba3

feat(install.sh): write working cafrun script

e7370f9

refac: mv script templates to script-templates dir

aec955e

fix(install.sh): correct cafrun args

5ec6ea2

update version to 3.0.0

d4e463e

doc(install.sh): better post-install output

19a3f04

feat(install.sh): write & verify pkg-config file

1eaad36

doc(AUTHORS.md): add Brad Richardson

67c9598

git mv script-templates to src

36c0b41

WIP: add Garden unit tests; single-image runs pass

448252f

WIP: replace run-fpm.sh with install.rsp-template

2a64b1f

fix(install.sh): support Linux in sed command

ad2bfc8

everythingfunctional reviewed May 2, 2022

View reviewed changes

everythingfunctional and others added 3 commits May 2, 2022 11:08

fix(install.sh): mixed up MPICC and MPIFC

41fd892

feat(install.sh): set -o pipefail

5e7c0a3

install.sh: fix #756

8f1de42

Base automatically changed from windows-support to main May 7, 2022 02:25

fix(install.sh): update user/developer directions

9c9380f

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

WIP: Experimental fpm build #753

WIP: Experimental fpm build #753

rouson commented Apr 23, 2022

everythingfunctional left a comment

everythingfunctional Apr 25, 2022

rouson Apr 25, 2022

everythingfunctional Apr 25, 2022

everythingfunctional Apr 25, 2022

everythingfunctional Apr 25, 2022

everythingfunctional Apr 25, 2022

everythingfunctional Apr 25, 2022

everythingfunctional May 2, 2022

		echo "For a non-interactive build with the 'yes' utility installed, execute"
		echo "yes \| ./install.sh"

		sed -i '' -e "s/@Fortran_COMPILER@/$Fortran_COMPILER/g" "$PREFIX"/bin/caf
		sed -i '' -e "s/@CAF_LIBS@/lib\/libopencoarrays.a/g" "$PREFIX"/bin/caf

WIP: Experimental fpm build #753

Are you sure you want to change the base?

WIP: Experimental fpm build #753

Conversation

rouson commented Apr 23, 2022

Summary of changes

To Do

Rationale for changes

Additional info and certifications

I certify that

Code coverage data

everythingfunctional left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment