Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

C++20 std::views::reverse #28687

Draft
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

stickies-v
Copy link
Contributor

@stickies-v stickies-v commented Oct 19, 2023

C++20 introduces std::ranges::views::reverse, which allows us to drop our own reverse_iterator.h implementation and also makes it easier to chain views (even though I think we currently don't use this).

In draft until our minimum dependencies are bumped to pass CI:

  • no wallet, libbitcoinkernel: requires clang-16

@DrahtBot
Copy link
Contributor

DrahtBot commented Oct 19, 2023

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage

For detailed information about the code coverage, see the test coverage report.

Reviews

See the guideline for information on the review process.
A summary of reviews will appear here.

Conflicts

Reviewers, this pull request conflicts with the following ones:

  • #29700 (kernel, refactor: return error status on all fatal errors by ryanofsky)
  • #29071 (refactor: Remove Span operator==, Use std::ranges::equal by maflcko)
  • #28052 (blockstorage: XOR blocksdir *.dat files by maflcko)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

@fanquake
Copy link
Member

cc @sipa you might have some insight here in regards to prevector.

@maflcko
Copy link
Member

maflcko commented Dec 11, 2023

WIP: our prevector implementation requires modification for std::ranges::views::reverse to work. More specifically: it needs to implement the bidirectional_range concept, which currently it seems not to (see static_assert(std::ranges::bidirectional_range<prevector>) output below) as we're not satisfying range. Once that's done, we can update the last usage of reverse_iterator.h here and here and remove the header entirely.

That is also required by std::span.

@stickies-v stickies-v force-pushed the 2023-10/cpp20-use-ranges-reverseview branch from 4eec99d to 95f9af8 Compare December 11, 2023 18:50
@stickies-v stickies-v changed the title [POC][WIP] C++20 std::views::reverse [WIP] C++20 std::views::reverse Dec 11, 2023
@stickies-v
Copy link
Contributor Author

Needs rebase

Thx, rebased to include #28349 & updated PR description

@fanquake
Copy link
Member

fanquake commented Dec 12, 2023

https://github.com/bitcoin/bitcoin/pull/28687/checks?check_run_id=19528202548

/usr/bin/ccache clang++-13 -stdlib=libc++ -std=c++20 -DHAVE_CONFIG_H -I. -I../src/config  -fmacro-prefix-map=/ci_container_base/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu=. -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3 -DHAVE_BUILD_INFO -DPROVIDE_FUZZ_MAIN_FUNCTION -I. -I./minisketch/include -I./secp256k1/include -I./univalue/include -I./leveldb/include -isystem /ci_container_base/depends/x86_64-pc-linux-gnu/include -DBOOST_MULTI_INDEX_DISABLE_SERIALIZATION -DBOOST_NO_CXX98_FUNCTION_BASE   -isystem /ci_container_base/depends/x86_64-pc-linux-gnu/include -pthread -I/ci_container_base/depends/x86_64-pc-linux-gnu/include  -I/ci_container_base/depends/x86_64-pc-linux-gnu/include/  -fdebug-prefix-map=/ci_container_base/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu=. -Wstack-protector -fstack-protector-all -fcf-protection=full -fstack-clash-protection   -Werror    -fvisibility=hidden -fPIE -pipe -std=c++20 -O2  -c -o libbitcoin_node_a-net_processing.o `test -f 'net_processing.cpp' || echo './'`net_processing.cpp
net_processing.cpp:2074:62: error: no member named 'reverse' in namespace 'std::views'
            for (const uint256& hash : vHashes | std::views::reverse) {
                                                 ~~~~~~~~~~~~^
net_processing.cpp:2760:69: error: no member named 'reverse' in namespace 'std::views'
            for (const CBlockIndex *pindex : vToFetch | std::views::reverse) {
                                                        ~~~~~~~~~~~~^
2 errors generated.

Looks like ranges didn't become complete / non-experimental in LLVM until version 16: https://releases.llvm.org/16.0.0/projects/libcxx/docs/ReleaseNotes.html

The C++20 ranges library has been completed and is no longer experimental (with the exception of ranges::join_view which is still marked as experimental because it is about to undergo an ABI-breaking change in the Standard due to D2770). Work on C++23 ranges has started.

Note that prior versions of LLVM required distros/venders to compile with special flags, so I assume it's just not going to be generally available until 16? i.e from the LLVM 14 release notes:

More parts of ranges have been implemented. Since we still expect to make some API and ABI breaking changes, those are disabled by default. However, vendors that wish to enable in their distribution may do so by defining -DLIBCXX_ENABLE_INCOMPLETE_FEATURES=ON when configuring their build.

@maflcko
Copy link
Member

maflcko commented Dec 12, 2023

Hmm, there's no clang-16 in the current LTS release of Ubuntu, so I guess this will have to wait for some time.

https://packages.ubuntu.com/jammy/clang-16

I wonder whether std::span will be usable, given that it internally assumes ranges?

@DrahtBot
Copy link
Contributor

Guix builds (on x86_64)

File commit f0e8290
(master)
commit 2a92fac
(master and this pull)
SHA256SUMS.part 7997d8e0220138cc... 3e66821911a14934...
*-aarch64-linux-gnu-debug.tar.gz c2d1d518af6b6690... 71cd2844ef1594e9...
*-aarch64-linux-gnu.tar.gz 24b6219549eb167f... 62103870da8375e6...
*-arm-linux-gnueabihf-debug.tar.gz 43d2f9e984064b30... 95c903521454832f...
*-arm-linux-gnueabihf.tar.gz 62029d495b06ae94... 2bec136fd3d597e5...
*-arm64-apple-darwin-unsigned.tar.gz 5940ccb123abcb6f... 80071654801bc8bb...
*-arm64-apple-darwin-unsigned.zip 5f9ad735e43975c2... cadd46af1fd698b9...
*-arm64-apple-darwin.tar.gz 4fb329d17e653ace... 09bc2dd4f449aaa0...
*-powerpc64-linux-gnu-debug.tar.gz b89080b50e79f418... 1e1c09db73cbc3cf...
*-powerpc64-linux-gnu.tar.gz 958122ba55627d09... 71e4c7cea959d564...
*-powerpc64le-linux-gnu-debug.tar.gz 3b193a6facbfaca1... 55c83a7292bcdef5...
*-powerpc64le-linux-gnu.tar.gz 133288419542c1bc... c1987c5dc27c19a0...
*-riscv64-linux-gnu-debug.tar.gz 497e98ac63114289... 6ec70782d4801883...
*-riscv64-linux-gnu.tar.gz 46bbcb20a8f21119... 7d0da4d25a6f60c5...
*-x86_64-apple-darwin-unsigned.tar.gz ab343d4c4677f67c... 64c41cb674a66651...
*-x86_64-apple-darwin-unsigned.zip ac24b8017268700b... 99a2559f0b3845b0...
*-x86_64-apple-darwin.tar.gz eb8a6f37e0bb38de... 0b2972137b3a2a6b...
*-x86_64-linux-gnu-debug.tar.gz 6b25557dcc1c1ac2... 31655748d794c403...
*-x86_64-linux-gnu.tar.gz 7edcc22d840d9362... 9d6ba3a813312f33...
*.tar.gz 846962d7c9f05011... d351b109a6580b69...
guix_build.log 7c8dff6c68055a73... 79e8210da9e68cc4...
guix_build.log.diff 1520c94a16f066b4...

@maflcko
Copy link
Member

maflcko commented Dec 14, 2023

Interesting that the guix macOS build passed, given that it is on clang-15, no?

@fanquake
Copy link
Member

Interesting that the guix macOS build passed, given that it is on clang-15, no?

I'd guess this is because our macOS SDK (libc++) is from Xcode 15, which is based on LLVM ~16, and includes support for ranges. So as long as clang-15 can compile it, which it should given vanilla LLVM supports the use of ranges (experimentally) in 15, it probably works.

@maflcko
Copy link
Member

maflcko commented Dec 14, 2023

Oh, I guess clang-15 can use libc++-16, but not libstdc++, according to https://godbolt.org/z/dbKWeE8Te and llvm/llvm-project@babdef2 ?

@maflcko
Copy link
Member

maflcko commented Dec 20, 2023

Though, on GHA it says XCode 14.3, see https://github.com/bitcoin/bitcoin/actions/runs/7171999335/job/19528203033?pr=28687#step:3:14

clang --version
Apple clang version 14.0.3 (clang-1403.0.22.14.1)
Target: x86_64-apple-darwin22.6.0
Thread model: posix
InstalledDir: /Applications/Xcode_14.3.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

@fanquake
Copy link
Member

Though, on GHA it says XCode 14.3

According to https://developer.apple.com/xcode/cpp/#c++20, "Ranges" have been available on macOS since Xcode 14.3.

@maflcko
Copy link
Member

maflcko commented Jan 23, 2024

WIP: our prevector implementation requires modification for std::ranges::views::reverse to work. More specifically: it needs to implement the bidirectional_range concept, which currently it seems not to (see static_assert(std::ranges::bidirectional_range<prevector>) output below) as we're not satisfying range. Once that's done, we can update the last usage of reverse_iterator.h here and here and remove the header entirely.

That is also required by std::span.

(Possibly?) fixed in #29275

@fanquake
Copy link
Member

(Possibly?) fixed in #29275

Looks like it does, this compiles for me locally atleast: https://github.com/fanquake/bitcoin/tree/28687_29275.

@stickies-v can you rebase this branch on top of #29275.

@fanquake fanquake added this to the 28.0 milestone Mar 6, 2024
@stickies-v stickies-v force-pushed the 2023-10/cpp20-use-ranges-reverseview branch from aabe87f to a198735 Compare March 11, 2024 14:44
@stickies-v
Copy link
Contributor Author

Force pushed to address merge conflict from #29483

@DrahtBot
Copy link
Contributor

🚧 At least one of the CI tasks failed. Make sure to run all tests locally, according to the
documentation.

Possibly this is due to a silent merge conflict (the changes in this pull request being
incompatible with the current code in the target branch). If so, make sure to rebase on the latest
commit of the target branch.

Leave a comment here, if you need help tracking down a confusing failure.

Debug: https://github.com/bitcoin/bitcoin/runs/22798928866

@stickies-v
Copy link
Contributor Author

stickies-v commented Mar 18, 2024

Force pushed to:

no wallet, libbitcoinkernel is now the only remaining unadressed failing CI task, afaict requiring a bump to clang-16 too

@stickies-v
Copy link
Contributor Author

Rebased on top of #29765. One more task to go 🤞

Use std::ranges::views::reverse instead of the implementation in
reverse_iterator.h, and remove it as it is no longer used.
@stickies-v stickies-v force-pushed the 2023-10/cpp20-use-ranges-reverseview branch from de52629 to f81565f Compare April 29, 2024 17:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants