Coroutines do work with Apple Clang but not with gcc

[diehlpk]

Expected Behavior

The code should compile with both clang and gcc.

Actual Behavior

However, using gcc results in this error

/home/diehlpk/git/book/Examples/hpx/taylor_hpx_future_coroutine.cpp: In function ‘hpx::future<double> run(size_t, size_t, double)’:
/home/diehlpk/git/book/Examples/hpx/taylor_hpx_future_coroutine.cpp:38:77: error: use of deleted function ‘hpx::future<double>::future(const hpx::future<double>&)’
   38 |   for (size_t i = 0; i < futures2.size(); i++) result += co_await futures2[i];
      |                                                                             ^
In file included from /home/diehlpk/Compile/hpx-1.8.0/libs/core/runtime_local/include/hpx/runtime_local/runtime_local.hpp:12,
                 from /home/diehlpk/Compile/hpx-1.8.0/libs/core/init_runtime_local/include/hpx/init_runtime_local/init_runtime_local.hpp:20,
                 from /home/diehlpk/Compile/hpx-1.8.0/libs/full/init_runtime/include/hpx/hpx_init_params.hpp:13,
                 from /home/diehlpk/Compile/hpx-1.8.0/libs/full/init_runtime/include/hpx/hpx_init.hpp:16,
                 from /home/diehlpk/Compile/hpx-1.8.0/wrap/include/hpx/wrap_main.hpp:11,
                 from /home/diehlpk/Compile/hpx-1.8.0/wrap/include/hpx/hpx_main.hpp:9,
                 from /home/diehlpk/git/book/Examples/hpx/taylor_hpx_future_coroutine.cpp:3:
/home/diehlpk/Compile/hpx-1.8.0/libs/core/futures/include/hpx/futures/future.hpp:830:11: note: ‘hpx::future<double>::future(const hpx::future<double>&)’ is implicitly declared as deleted because ‘hpx::future<double>’ declares a move constructor or move assignment operator
  830 |     class future : public lcos::detail::future_base<future<R>, R>
      |           ^~~~~~
make[2]: *** [hpx/CMakeFiles/taylor_hpx_future_coroutine.dir/build.make:76: hpx/CMakeFiles/taylor_hpx_future_coroutine.dir/taylor_hpx_future_coroutine.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:1136: hpx/CMakeFiles/taylor_hpx_future_coroutine.dir/all] Error 2

Steps to Reproduce the Problem

... Please be as specific as possible while describing how to reproduce your problem.

1. Compile HPX with clang 12 and gcc 12
2. Compile the example code with both of them

Specifications

... Please describe your environment

• HPX Version: 1.8.0
• Platform (compiler, OS): Mac OS X with Apple Clang and Fedora Linux with gcc 12

[diehlpk]

@hkaiser Do you know if we fixed that or is it still open?

[hkaiser]

@hkaiser Do you know if we fixed that or is it still open?

We have not tried recently. @SAtacker: would you be able to check?

[SAtacker]

@hkaiser Do you know if we fixed that or is it still open?

We have not tried recently. @SAtacker: would you be able to check?

The last time I checked was on Rostam with a (slightly) different version than the one that caused the issue, and I wasn't able to reproduce it. Let me try to reproduce it again with the exact same versions.

[SAtacker]

I was able to reproduce this. Thanks! Let me take a look.

[diehlpk]

There is a similar issue with sender and receivers. I will open a new ticket for that.

[SAtacker]

While I wasn't able to use llvm/clang for testing, I made a simple workaround for this.

diff --git a/hpx/CMakeLists.txt b/hpx/CMakeLists.txt
index ed6f08e..bd6102b 100644
--- a/hpx/CMakeLists.txt
+++ b/hpx/CMakeLists.txt
@@ -37,7 +37,7 @@ add_hpx_executable(taylor_hpx_distributed SOURCES taylor_hpx_distributed.cpp)
 add_test(NAME "HPX:_Taylor_distributed" COMMAND taylor_hpx_distributed)
 
 if(${WITH_COROUTINES})
-  #add_hpx_executable(taylor_hpx_future_coroutine SOURCES taylor_hpx_future_coroutine.cpp)
+  add_hpx_executable(taylor_hpx_future_coroutine SOURCES taylor_hpx_future_coroutine.cpp)
   add_test(NAME "HPX:_Taylor_future_coroutine" COMMAND taylor_hpx_future_coroutine 100 0.5 3)
 endif()
 
diff --git a/hpx/taylor_hpx_future_coroutine.cpp b/hpx/taylor_hpx_future_coroutine.cpp
index cb43f08..bb21098 100644
--- a/hpx/taylor_hpx_future_coroutine.cpp
+++ b/hpx/taylor_hpx_future_coroutine.cpp
@@ -40,7 +40,7 @@ hpx::future<double> run(size_t n, size_t amount, double x) {
 
   auto futures2 = co_await hpx::when_all(std::move(futures));
 
-  for (size_t i = 0; i < futures2.size(); i++) result += co_await futures2[i];
+  for (size_t i = 0; i < futures2.size(); i++) result += co_await std::move(futures2[i]);
 
   co_return result;
 }

Output:

satacker@ubuntu:/home/extended/ModernCPPExamples/build$ ./hpx/taylor_hpx_distributed 100 0.5 3
1
100
Difference of Taylor and C++ result -1.11022e-16 after 100 iterations.
satacker@ubuntu:/home/extended/ModernCPPExamples/build$ ./hpx/taylor_hpx_future_coroutine 100 0.5 3
Difference of Taylor and C++ result -1.11022e-16 after 100 iterations.
satacker@ubuntu:/home/extended/ModernCPPExamples/build$ ./hpx/taylor_hpx_future 100 0.5 3
Difference of Taylor and C++ result -1.11022e-16 after 100 iterations.

CC @diehlpk @hkaiser

[hkaiser]

@SAtacker thanks a lot. I think I now remember reading somewhere that clang co_await expects an rvalue reference for non-copyable awaitables.

[SAtacker]

On taking another look futures2[i] has a type of hpx::future<double>& which is a glvalue (“generalized” lvalue) which is an lvalue or an xvalue.
As per cpp reference,

If the expression above is a prvalue, the awaiter object is a temporary materialized from it. Otherwise, if the expression above is an glvalue, the awaiter object is the object to which it refers.

Link

What does this mean?

[hkaiser]

What does this mean?

It means that co_await should be usable on a lvalue (glvalue) and a rvalue (prvalue). If used on a rvalue, a temporary object should be created from it before co_await-ing. Clearly, clang doesn't get this right in this case.

[diehlpk]

There is a similar issue with Sender and Receiver, I will open a new ticket for that.

[diehlpk]

@SAtacker thanks for the fix and I will update the code.

[diehlpk]

@SAtacker I found another issue using gcc 10.3.1 and here I get the following error

/home/jovyan/project/src/taylor_future_coroutine_hpx.cpp: In function 'hpx::future<double> run(size_t, size_t, double)':
/home/jovyan/project/src/taylor_future_coroutine_hpx.cpp:47:19: error: unable to find the promise type for this coroutine
   47 |   auto futures2 = co_await std::move(hpx::when_all(std::move(futures)));
      |                   ^~~~~~~~
make[2]: *** [src/CMakeFiles/taylor_future_coroutine_hpx.dir/build.make:82: src/CMakeFiles/taylor_future_coroutine_hpx.dir/taylor_future_coroutine_hpx.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:986: src/CMakeFiles/taylor_future_coroutine_hpx.dir/all] Error 2
make: *** [Makefile:160: all] Error 2

Exited with code exit status 2

and it is the same without std::move.

[SAtacker]

@SAtacker I found another issue using gcc 10.3.1 and here I get the following error

/home/jovyan/project/src/taylor_future_coroutine_hpx.cpp: In function 'hpx::future<double> run(size_t, size_t, double)':
/home/jovyan/project/src/taylor_future_coroutine_hpx.cpp:47:19: error: unable to find the promise type for this coroutine
   47 |   auto futures2 = co_await std::move(hpx::when_all(std::move(futures)));
      |                   ^~~~~~~~
make[2]: *** [src/CMakeFiles/taylor_future_coroutine_hpx.dir/build.make:82: src/CMakeFiles/taylor_future_coroutine_hpx.dir/taylor_future_coroutine_hpx.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:986: src/CMakeFiles/taylor_future_coroutine_hpx.dir/all] Error 2
make: *** [Makefile:160: all] Error 2

Exited with code exit status 2

and it is the same without std::move.

Thanks, let me take a better look. (At a glance it seemed like a g++-10 issue as I had read somewhere that gcc 10.x does additional move, however if you say that's not the case I would definitely take a better look.)
Thanks

[SAtacker]

@SAtacker I found another issue using gcc 10.3.1 and here I get the following error

/home/jovyan/project/src/taylor_future_coroutine_hpx.cpp: In function 'hpx::future<double> run(size_t, size_t, double)':
/home/jovyan/project/src/taylor_future_coroutine_hpx.cpp:47:19: error: unable to find the promise type for this coroutine
   47 |   auto futures2 = co_await std::move(hpx::when_all(std::move(futures)));
      |                   ^~~~~~~~
make[2]: *** [src/CMakeFiles/taylor_future_coroutine_hpx.dir/build.make:82: src/CMakeFiles/taylor_future_coroutine_hpx.dir/taylor_future_coroutine_hpx.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:986: src/CMakeFiles/taylor_future_coroutine_hpx.dir/all] Error 2
make: *** [Makefile:160: all] Error 2

Exited with code exit status 2

and it is the same without std::move.

Thanks, let me take a better look. (At a glance it seemed like a g++-10 issue as I had read somewhere that gcc 10.x does additional move, however if you say that's not the case I would definitely take a better look.) Thanks

I was able to reproduce it on gcc-10.4.0

[SAtacker]

@SAtacker thanks a lot. I think I now remember reading somewhere that clang co_await expects an rvalue reference for non-copyable awaitables.

Found the link for this bug report on gcc - https://gcc.gnu.org/bugzilla/show_bug.cgi?id=103963

[hkaiser]

@SAtacker does this work, then?

auto futures2 = co_await hpx::when_all(std::move(futures)).share();

[SAtacker]

@SAtacker does this work, then?

auto futures2 = co_await hpx::when_all(std::move(futures)).share();

It does not : (

[SAtacker]

I am not sure but it seems like an issue related to name_lookup?

[SAtacker]

Found a bunch of stuff related to lookup of promise type for coroutines. diff releases/gcc-10.4.1 releases/gcc-11.1.0

2020-02-04  Iain Sandoe  <iain@sandoe.co.uk>
+
+       * coroutines.cc (find_promise_type): Delete unused forward
+       declaration.
+       (struct coroutine_info): Add a bool for no promise type error.
+       (coro_promise_type_found_p): Only emit the error for a missing
+       promise once in each affected coroutine.
+
+2020-02-03  Iain Sandoe  <iain@sandoe.co.uk>
+
+       PR c++/93458
+       * coroutines.cc (struct coroutine_info): Add a bool flag to note
+       that we emitted an error for a bad function return type.
+       (get_coroutine_info): Tolerate an unset info table in case of
+       missing traits.
+       (find_coro_traits_template_decl): In case of error or if we didn't
+       find a type template, note we emitted the error and suppress
+       duplicates.
+       (find_coro_handle_template_decl): Likewise.
+       (instantiate_coro_traits): Only check for error_mark_node in the
+       return from lookup_qualified_name. 
+       (coro_promise_type_found_p): Reorder initialization so that we check
+       for the traits and their usability before allocation of the info
+       table.  Check for a suitable return type and emit a diagnostic for
+       here instead of relying on the lookup machinery.  This allows the
+       error to have a better location, and means we can suppress multiple
+       copies.
+       (coro_function_valid_p): Re-check for a valid promise (and thus the
+       traits) before proceeding.  Tolerate missing info as a fatal error.

I think this is definitely a bug related to promise lookup. Perhaps can we reproduce this on godboldt @hkaiser ?

[hkaiser]

I think this is definitely a bug related to promise lookup.

Yes, I agree.

Perhaps can we reproduce this on godboldt @hkaiser ?

Please feel free to try ;-)

[SAtacker]

@diehlpk I compiled the same code with gcc-10.4.0 and with gcc-11.x and it works. This must be a bug, however I am not able to reproduce it on godbolt.