backport: Switch RNG to PRNG/Add RNG strengthening (10ms once every minute) #3522
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Completely removes BIP70 support from Dogecoin Qt code. Also removes protobuf-compatibility tests as these rely on paymentserverplus code and any TLS related configuration.
Protobuf was only needed for BIP70 payment requests
…o std from boost. Commit 1. This code was written by @TheBlueMatt in the following branch: * https://github.com/TheBlueMatt/bitcoin/commits/2017-08-test-10923 This commit message was written by me (@practicalswift) who also squashed @TheBlueMatt's commits into one and tried to summarize the changes made. Commit 2. Remove boost include. Remove boost mentions in comments. Cherry-picked from: 7e319d6
Cherry-picked from: bba9bd0
Cherry-picked from: f7f7e2c
Cherry-picked from: 41b88e9
Move AnnotatedMixin closer to where it's used, and after the DEBUG_LOCKORDER function declarations so it can call them. Cherry-picked from: ba1f095
They should also work with any other mutex type which std::unique_lock supports. There is no change in behavior for current code that calls these macros with CCriticalSection mutexes. Cherry-picked from: 1382913
Instead of std::unique_lock. Cherry-picked from: 9c4dc59
Call sync.h primitives "locks" and "mutexes" instead of "blocks" and "waitable
critical sections" to match current coding conventions and c++11 standard
names.
This PR does not rename the "CCriticalSection" class (though this could be done
as a followup) because it is used everywhere and would swamp the other changes
in this PR. Plain mutexes should mostly be preferred instead of recursive
mutexes in new code anyway.
-BEGIN VERIFY SCRIPT-
set -x
set -e
ren() { git grep -l $1 | xargs sed -i s/$1/$2/; }
ren CCriticalBlock UniqueLock
ren CWaitableCriticalSection Mutex
ren CConditionVariable std::condition_variable
ren cs_GenesisWait g_genesis_wait_mutex
ren condvar_GenesisWait g_genesis_wait_cv
perl -0777 -pi -e 's/.*typedef.*condition_variable.*\n\n?//g' src/sync.h
-END VERIFY SCRIPT-
Cherry-picked from: 190bf62
Cherry-picked from: c21cbe6
Cherry-picked from: 663fbae
Cherry-picked from: e04326f
These are available in sandboxes without access to files or devices. Also [they are safer and more straightforward](https://en.wikipedia.org/wiki/Entropy-supplying_system_calls) to use than `/dev/urandom` as reading from a file has quite a few edge cases: - Linux: `getrandom(buf, buflen, 0)`. [getrandom(2)](http://man7.org/linux/man-pages/man2/getrandom.2.html) was introduced in version 3.17 of the Linux kernel. - OpenBSD: `getentropy(buf, buflen)`. The [getentropy(2)](http://man.openbsd.org/cgi-bin/man.cgi/OpenBSD-current/man2/getentropy.2) function appeared in OpenBSD 5.6. - FreeBSD and NetBSD: `sysctl(KERN_ARND)`. Not sure when this was added but it has existed for quite a while. Alternatives: - Linux has sysctl `CTL_KERN` / `KERN_RANDOM` / `RANDOM_UUID` which gives 16 bytes of randomness. This may be available on older kernels, however [sysctl is deprecated on Linux](https://lwn.net/Articles/605392/) and even removed in some distros so we shouldn't use it. Add tests for `GetOSRand()`: - Test that no error happens (otherwise `RandFailure()` which aborts) - Test that all 32 bytes are overwritten (initialize with zeros, try multiple times) Discussion: - When to use these? Currently they are always used when available. Another option would be to use them only when `/dev/urandom` is not available. But this would mean these code paths receive less testing, and I'm not sure there is any reason to prefer `/dev/urandom`. Closes: #9676 Cherry-picked from: 224e6eb Contains squashed commit of aa09ccb squashme: comment that NUM_OS_RANDOM_BYTES should not be changed lightly
Move the OS random test to a sanity check function that is called every time bitcoind is initialized. Keep `src/test/random_tests.cpp` for the case that later random tests are added, and keep a rudimentary test that just calls the sanity check. Cherry-picked from: 7cad849
If the code was compiled with newer (>=3.17) kernel headers but executed on a system without the system call, every use of random would crash the program. Add a fallback for that case. Cherry-picked from: 7e6dcd9
Cherry-picked from: 1632922
Cherry-picked from: 4fd2d2f
FastRandomContext now provides all functionality that the real Rand* functions provide. Cherry-picked from: 37e864e
Cherry-picked from: 124d13a
Cherry-picked from: 1119927
-BEGIN VERIFY SCRIPT- sed -i "s/\<GetRandHash(/insecure_rand256(/" src/test/*_tests.cpp sed -i "s/\<GetRand(/insecure_randrange(/" src/test/*_tests.cpp src/test/test_bitcoin.cpp sed -i 's/\<insecure_rand() % \([0-9]\+\)/insecure_randrange(\1)/g' src/test/*_tests.cpp -END VERIFY SCRIPT- Cherry-picked from: efee1db
Cherry-picked from: 3ecabae
Cherry-picked from: 5f0b04e
Cherry-picked from: 2ada678
-BEGIN VERIFY SCRIPT- sed -i 's/insecure_randbits(1)/insecure_randbool()/g' src/test/*_tests.cpp sed -i 's/insecure_randrange(2)/insecure_randbool()/g' src/test/*_tests.cpp sed -i 's/insecure_randrange(4)/insecure_randbits(2)/g' src/test/*_tests.cpp sed -i 's/insecure_randrange(32)/insecure_randbits(5)/g' src/test/*_tests.cpp sed -i 's/insecure_randrange(256)/insecure_randbits(8)/g' src/test/*_tests.cpp -END VERIFY SCRIPT- Cherry-picked from: 2fcd9cc
-BEGIN VERIFY SCRIPT- sed -i 's/\<insecure_randbits(/InsecureRandBits(/g' src/test/*.cpp src/test/*.h src/wallet/test/*.cpp sed -i 's/\<insecure_randbool(/InsecureRandBool(/g' src/test/*.cpp src/test/*.h src/wallet/test/*.cpp sed -i 's/\<insecure_randrange(/InsecureRandRange(/g' src/test/*.cpp src/test/*.h src/wallet/test/*.cpp sed -i 's/\<insecure_randbytes(/InsecureRandBytes(/g' src/test/*.cpp src/test/*.h src/wallet/test/*.cpp sed -i 's/\<insecure_rand256(/InsecureRand256(/g' src/test/*.cpp src/test/*.h src/wallet/test/*.cpp sed -i 's/\<insecure_rand(/InsecureRand32(/g' src/test/*.cpp src/test/*.h src/wallet/test/*.cpp sed -i 's/\<seed_insecure_rand(/SeedInsecureRand(/g' src/test/*.cpp src/test/*.h src/wallet/test/*.cpp -END VERIFY SCRIPT- Cherry-picked from: e945848
Cherry-picked from: ee2d10a
Remove it. Make this change, so in a future commit, we can combine #ifdefs, and avoid duplicate <sys/random.h> includes once we switch to using getrandom directly. Also remove the comment about macOS 10.12. We already require macOS > 10.15, so it is redundant. Cherry-picked from: c13c97dbf846cf0e6a5581ac414ef96a215b0dc6
Cherry-picked from: f544094
Cherry-picked from: 33f853d
Cherry-picked from: 2c0a6f1
Cherry-picked from: cb24c85
Call RandomInit() in bench_bitcoin to initialize the RNG so that it does not cause an assertion error. Cherry-picked from: 5155d11
rbx needs to be stashed in a 64bit register on 64bit platforms. With this crash in particular, it was holding a stack canary which was not properly restored after the cpuid. Split out the x86+PIC case so that x86_64 doesn't have to worry about it. Cherry-picked from: 9af207c
Cherry-picked from: a9e82f6
Cherry-picked from: 674848f
Cherry-picked from: a8ae0b2
Cherry-picked from: fbf327b
This makes it possible to plug it into the various standard C++11 random distribution algorithms and other functions like std::shuffle. Cherry-picked from: 1ec1602
These are hard to deal with, as in a follow-up this function can get called before the logging infrastructure is initialized. Cherry-picked from: 2d1cc50
Cherry-picked from: 05fde14
Cherry-picked from: d3f54d1
Cherry-picked from: aae8b9b
Cherry-picked from: 2ccc3d3
This guarantees that OpenSSL is initialized properly whenever randomness is used, even when that randomness is invoked from global constructors. Note that this patch uses Mutex directly, rather than CCriticalSection. This is because the lock-detection code is not necessarily initialized during global constructors. Cherry-picked from: 16e40a8
It includes the following policy changes: * All GetRand* functions seed the stack pointer and rdrand result (in addition to the performance counter) * The periodic entropy added by the idle scheduler now seeds stack pointer, rdrand and perfmon data (once every 10 minutes) in addition to just a sleep timing. * The entropy added when calling GetStrongRandBytes no longer includes the once-per-10-minutes perfmon data on windows (it is moved to the idle scheduler instead, where latency matters less). Other changes: * OpenSSL is no longer seeded directly anywhere. Instead, any generated randomness through our own RNG is fed back to OpenSSL (after an additional hashing step to prevent leaking our RNG state). * Seeding that was previously done directly in RandAddSeedSleep is now moved to SeedSleep(), which is indirectly invoked through ProcRand from RandAddSeedSleep. * Seeding that was previously done directly in GetStrongRandBytes() is now moved to SeedSlow(), which is indirectly invoked through ProcRand from GetStrongRandBytes(). Cherry-picked from: 9d7032e
All access to hwrand is now gated by GetRNGState, which initializes the hwrand code. Cherry-picked from: 4ea8e50
Cherry-picked from: a1f252e
Cherry-picked from: 152146e
Cherry-picked from: cddb31b
Cherry-picked from: f2e60ca
Cherry-picked from: 223de8d
Once every minute, this will feed the RNG state through repeated SHA512 for 10ms. The timings of that operation are used as entropy source as well. Cherry-picked from: 1d207bc
Cherry-picked from: 3cb9ce8
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Cherry-picked from:
bitcoin/bitcoin#14955
bitcoin/bitcoin#15224
Rebased on #3521, #3511, #3520, #3487 and #3466.