backport: random.h/.cpp cherry-picks up to #10821 #3521
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…o std from boost. Commit 1. This code was written by @TheBlueMatt in the following branch: * https://github.com/TheBlueMatt/bitcoin/commits/2017-08-test-10923 This commit message was written by me (@practicalswift) who also squashed @TheBlueMatt's commits into one and tried to summarize the changes made. Commit 2. Remove boost include. Remove boost mentions in comments. Cherry-picked from: 7e319d6
Cherry-picked from: bba9bd0
Cherry-picked from: f7f7e2c
Cherry-picked from: 41b88e9
Move AnnotatedMixin closer to where it's used, and after the DEBUG_LOCKORDER function declarations so it can call them. Cherry-picked from: ba1f095
They should also work with any other mutex type which std::unique_lock supports. There is no change in behavior for current code that calls these macros with CCriticalSection mutexes. Cherry-picked from: 1382913
Instead of std::unique_lock. Cherry-picked from: 9c4dc59
Call sync.h primitives "locks" and "mutexes" instead of "blocks" and "waitable
critical sections" to match current coding conventions and c++11 standard
names.
This PR does not rename the "CCriticalSection" class (though this could be done
as a followup) because it is used everywhere and would swamp the other changes
in this PR. Plain mutexes should mostly be preferred instead of recursive
mutexes in new code anyway.
-BEGIN VERIFY SCRIPT-
set -x
set -e
ren() { git grep -l $1 | xargs sed -i s/$1/$2/; }
ren CCriticalBlock UniqueLock
ren CWaitableCriticalSection Mutex
ren CConditionVariable std::condition_variable
ren cs_GenesisWait g_genesis_wait_mutex
ren condvar_GenesisWait g_genesis_wait_cv
perl -0777 -pi -e 's/.*typedef.*condition_variable.*\n\n?//g' src/sync.h
-END VERIFY SCRIPT-
Cherry-picked from: 190bf62
Cherry-picked from: c21cbe6
Cherry-picked from: 663fbae
Cherry-picked from: e04326f
These are available in sandboxes without access to files or devices. Also [they are safer and more straightforward](https://en.wikipedia.org/wiki/Entropy-supplying_system_calls) to use than `/dev/urandom` as reading from a file has quite a few edge cases: - Linux: `getrandom(buf, buflen, 0)`. [getrandom(2)](http://man7.org/linux/man-pages/man2/getrandom.2.html) was introduced in version 3.17 of the Linux kernel. - OpenBSD: `getentropy(buf, buflen)`. The [getentropy(2)](http://man.openbsd.org/cgi-bin/man.cgi/OpenBSD-current/man2/getentropy.2) function appeared in OpenBSD 5.6. - FreeBSD and NetBSD: `sysctl(KERN_ARND)`. Not sure when this was added but it has existed for quite a while. Alternatives: - Linux has sysctl `CTL_KERN` / `KERN_RANDOM` / `RANDOM_UUID` which gives 16 bytes of randomness. This may be available on older kernels, however [sysctl is deprecated on Linux](https://lwn.net/Articles/605392/) and even removed in some distros so we shouldn't use it. Add tests for `GetOSRand()`: - Test that no error happens (otherwise `RandFailure()` which aborts) - Test that all 32 bytes are overwritten (initialize with zeros, try multiple times) Discussion: - When to use these? Currently they are always used when available. Another option would be to use them only when `/dev/urandom` is not available. But this would mean these code paths receive less testing, and I'm not sure there is any reason to prefer `/dev/urandom`. Closes: #9676 Cherry-picked from: 224e6eb Contains squashed commit of aa09ccb squashme: comment that NUM_OS_RANDOM_BYTES should not be changed lightly
Move the OS random test to a sanity check function that is called every time bitcoind is initialized. Keep `src/test/random_tests.cpp` for the case that later random tests are added, and keep a rudimentary test that just calls the sanity check. Cherry-picked from: 7cad849
If the code was compiled with newer (>=3.17) kernel headers but executed on a system without the system call, every use of random would crash the program. Add a fallback for that case. Cherry-picked from: 7e6dcd9
Cherry-picked from: 1632922
Cherry-picked from: 4fd2d2f
FastRandomContext now provides all functionality that the real Rand* functions provide. Cherry-picked from: 37e864e
Cherry-picked from: 124d13a
Cherry-picked from: 1119927
-BEGIN VERIFY SCRIPT- sed -i "s/\<GetRandHash(/insecure_rand256(/" src/test/*_tests.cpp sed -i "s/\<GetRand(/insecure_randrange(/" src/test/*_tests.cpp src/test/test_bitcoin.cpp sed -i 's/\<insecure_rand() % \([0-9]\+\)/insecure_randrange(\1)/g' src/test/*_tests.cpp -END VERIFY SCRIPT- Cherry-picked from: efee1db
Cherry-picked from: 3ecabae
Cherry-picked from: 5f0b04e
Cherry-picked from: 2ada678
-BEGIN VERIFY SCRIPT- sed -i 's/insecure_randbits(1)/insecure_randbool()/g' src/test/*_tests.cpp sed -i 's/insecure_randrange(2)/insecure_randbool()/g' src/test/*_tests.cpp sed -i 's/insecure_randrange(4)/insecure_randbits(2)/g' src/test/*_tests.cpp sed -i 's/insecure_randrange(32)/insecure_randbits(5)/g' src/test/*_tests.cpp sed -i 's/insecure_randrange(256)/insecure_randbits(8)/g' src/test/*_tests.cpp -END VERIFY SCRIPT- Cherry-picked from: 2fcd9cc
-BEGIN VERIFY SCRIPT- sed -i 's/\<insecure_randbits(/InsecureRandBits(/g' src/test/*.cpp src/test/*.h src/wallet/test/*.cpp sed -i 's/\<insecure_randbool(/InsecureRandBool(/g' src/test/*.cpp src/test/*.h src/wallet/test/*.cpp sed -i 's/\<insecure_randrange(/InsecureRandRange(/g' src/test/*.cpp src/test/*.h src/wallet/test/*.cpp sed -i 's/\<insecure_randbytes(/InsecureRandBytes(/g' src/test/*.cpp src/test/*.h src/wallet/test/*.cpp sed -i 's/\<insecure_rand256(/InsecureRand256(/g' src/test/*.cpp src/test/*.h src/wallet/test/*.cpp sed -i 's/\<insecure_rand(/InsecureRand32(/g' src/test/*.cpp src/test/*.h src/wallet/test/*.cpp sed -i 's/\<seed_insecure_rand(/SeedInsecureRand(/g' src/test/*.cpp src/test/*.h src/wallet/test/*.cpp -END VERIFY SCRIPT- Cherry-picked from: e945848
Cherry-picked from: ee2d10a
Remove it. Make this change, so in a future commit, we can combine #ifdefs, and avoid duplicate <sys/random.h> includes once we switch to using getrandom directly. Also remove the comment about macOS 10.12. We already require macOS > 10.15, so it is redundant. Cherry-picked from: c13c97dbf846cf0e6a5581ac414ef96a215b0dc6
Cherry-picked from: f544094
Cherry-picked from: 33f853d
Cherry-picked from: 2c0a6f1
Cherry-picked from: cb24c85
Call RandomInit() in bench_bitcoin to initialize the RNG so that it does not cause an assertion error. Cherry-picked from: 5155d11
rbx needs to be stashed in a 64bit register on 64bit platforms. With this crash in particular, it was holding a stack canary which was not properly restored after the cpuid. Split out the x86+PIC case so that x86_64 doesn't have to worry about it. Cherry-picked from: 9af207c
Cherry-picked from: a9e82f6
Cherry-picked from: 674848f
Cherry-picked from: a8ae0b2
Cherry-picked from: fbf327b
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is rebased off of #3511, #3520, #3487. It includes commits to randcom.h/.cpp among others but is precursor to https://github.com/xanimo/dogecoin/tree/1.15.0-dev-14955-15224 and https://github.com/xanimo/dogecoin/tree/1.15.0-dev-14955-15224-15512-24926