`utxo_total_supply` extremely slow #158

murchandamus · 2023-11-07T15:01:28Z

While I had noticed utxo_total_supply being extremely slow on a merge before, my nightly fuzzer has been on the utxo_total_supply target for 6h51m at this point, even though the limit on the fuzzer is 3600 seconds. It seems to me that either it is stuck in an endless loop, or just extremely slow.

My […]/qa-assets-active-fuzzing/fuzz_seed_corpus/utxo_total_supply directory has 3250 seeds.

It looks to me like it hasn’t even loaded my corpus yet:

Here are two of the worst offenders:

slow-unit-ec3af16aeb3b967f9b63edc7266a71f52825c616

echo "egV+iYmJCy56C0HRX/7+cP7+/v////////7+/v7+/v7+/v7+/v78/v7+/v7+/v7+/r7+/v5cXFxcXFxcXFwpKSkpKSkpKf7+/v7+/v7+/v6+/v4pKSkpKSkpKSkpXClcXP7+JiAOWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZsDQ/J55ZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWbA1PyeewVkoWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVkODg4OWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWQ4ODg5ZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZsDQ/J55ZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWbA1PyeewVkoWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVkODg4OWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlQWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWQ4ODg5ZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVmwND8nnllZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZsDU/J57BWShZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWQ4ODg5ZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZDg4ODllZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVmwND8nnllZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZsDU/J57BWShZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWQ4ODg5ZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVBZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWf7+/v7+/v7+C0HRX/7+/v7+/v5ZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWQ4ODg5ZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWbA0PyeeWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZsDQ/J55ZWVlZWVn+/v7+/v6+WVlZWVlZWVlZWVlZWVlZWVlZWVlZ/v7+/v741y4LQVlZWVlZWVlZWVlZsDQ/J57BWShZWVkODg7RDg==" | base64 -d > slow-unit-ec3af16aeb3b967f9b63edc7266a71f52825c616

/slow-unit-9c08c09c1d630fade4f699fdd1eeaa53325dcce3

echo "JiAOWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZDg4ODllZWVlZWVlZWVlZWVlZWVlZWVlZWVkZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWTJZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVkODg4OWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlTWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWQ4ODllZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZDg4ODllZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVkgDllZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWQ4ODg5ZWVlZWVlZWVlZWVlZWVlZWVlZWVlZGVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVkyWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZDg4ODllZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVkODg5ZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWQ4ODg5ZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVxcXFxcXFxcXFxcXFxcXFxcXFwIvLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8XFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcvLy8t7y8vLy8XAj19Ts7Ox0mJiYmOzs7JiYmJiYmJiYmJiYmJiYmJiYmJiYmXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXAi8vLy8vLy8vLy8vLy8vLy8vLy8vLxZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWQ4ODllZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVmwOD8nnllZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZDg4OWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWbA4PyeeWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVmwNT8nnsFZKFlZWQ4ODg4=" | base64 -d > slow-repro-9c08c09c

The text was updated successfully, but these errors were encountered:

maflcko · 2023-11-07T15:13:02Z

What is tail ./fuzz-*.log ?

murchandamus · 2023-11-07T15:20:26Z

Sorry, was working on adding that. See OP, here’s fuzz-0.log:

INFO: Running with entropic power schedule (0xFF, 100).
INFO: Seed: 3732496129
INFO: Loaded 1 modules   (360765 inline 8-bit counters): 360765 [0x561b6de24550, 0x561b6de7c68d), 
INFO: Loaded 1 PC tables (360765 PCs): 360765 [0x561b6de7c690,0x561b6e3fda60), 
INFO:     3247 files found in ../qa-assets-active-fuzzing/fuzz_seed_corpus/utxo_total_supply
INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 1048576 bytes
INFO: seed corpus: files: 3247 min: 1b max: 4194303b total: 11985543b rss: 145Mb
#16 pulse  cov: 12694 ft: 12725 corp: 5/5b exec/s: 8 rss: 203Mb
#32 pulse  cov: 12694 ft: 12764 corp: 8/8b exec/s: 8 rss: 261Mb
#64 pulse  cov: 12698 ft: 12822 corp: 14/18b exec/s: 8 rss: 314Mb
#128    pulse  cov: 12859 ft: 17302 corp: 28/57b exec/s: 7 rss: 316Mb
#256    pulse  cov: 13075 ft: 21971 corp: 61/179b exec/s: 6 rss: 316Mb
#512    pulse  cov: 13323 ft: 30130 corp: 209/1145b exec/s: 5 rss: 316Mb
#1024   pulse  cov: 13360 ft: 35448 corp: 466/3858b exec/s: 3 rss: 316Mb
#2048   pulse  cov: 13450 ft: 53069 corp: 1156/25Kb exec/s: 1 rss: 316Mb
Slowest unit: 10 s:
artifact_prefix='./'; Test unit written to ./slow-unit-e1eb74194ce0614d81ac490af1a6583a66d1d2c4
Base64: EQj19Ts7OyYmJiYmOzs7JiYmJiYmJiYmJiYmJiYmJiYmJiYmXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXAi8vLy8eby8vLy8vLy8vLy8YmxvY2u8vLy8vLy8vLy8vLy8vLy8vLy8vLxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFy8vLy8vLy8vLxcXFxcDlwJ
Slowest unit: 12 s:
artifact_prefix='./'; Test unit written to ./slow-unit-cb6bebf823aa3326525d2cc1aa6cbf58b487970d
Base64: Jrm5uSa5VlZWVlZWVlZWVlZWVlYmJiYmJiwsLCwsLCwsLCwsLCwsLCwsLCwsLCwsLCwsLCwsLCwsLCwsLCwsLCwsLCwsLCwsLCwsLCwsLCwsLCwsLCwsLCwsLCwsJiYB/yYmJoBWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlYmJiYmJiYmJiYmJiYmJiYmJFsmQUEl
Slowest unit: 14 s:
artifact_prefix='./'; Test unit written to ./slow-unit-b625b9409852885d51fdba131cf90164fc8178d1
Base64: CwcKxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFoaGhoaGhoaGhoaGhoaGhoaGhoaGhocXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcWhoaGhoaGhoaGhoaGhoaGhoaGhoaGhocXFxSOhxcXFIwoa
Slowest unit: 16 s:
artifact_prefix='./'; Test unit written to ./slow-unit-a9e98ee3db1eaac45162745e9e950c48097250e0
Base64: BUOPj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+PjwVDj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+P
Slowest unit: 18 s:
artifact_prefix='./'; Test unit written to ./slow-unit-802520d1fc4cc129532de6b468e78d9e59160aaa
Slowest unit: 20 s:
artifact_prefix='./'; Test unit written to ./slow-unit-e20f483cbf3d4b7449bac7dda8c5a46121e9c139
Slowest unit: 23 s:
artifact_prefix='./'; Test unit written to ./slow-unit-1ef5e38d73fdf03b9c54ca5634a10b5a16fe91e2
Slowest unit: 26 s:
artifact_prefix='./'; Test unit written to ./slow-unit-2c01e1724129117881e5b8860ec15c6cc0ac852c
Slowest unit: 29 s:
artifact_prefix='./'; Test unit written to ./slow-unit-346622c0f7de92ac1969ae91f8b030d0dc2ddf0a
Slowest unit: 32 s:
artifact_prefix='./'; Test unit written to ./slow-unit-993881544bb66c65ae6a206250eacbb655511ad7
Slowest unit: 36 s:
artifact_prefix='./'; Test unit written to ./slow-unit-6901a6b6a7915ed339cc6e5d0c7f93045ed7da2f
Slowest unit: 40 s:
artifact_prefix='./'; Test unit written to ./slow-unit-49b3b886c91ccb51bf356eef02d51dda4af2ee1a
Slowest unit: 46 s:
artifact_prefix='./'; Test unit written to ./slow-unit-9ab2a6b61a346db624def4553bf9841a8aeb2712
Slowest unit: 61 s:
artifact_prefix='./'; Test unit written to ./slow-unit-75d0af5eeadc76bd8866cec81a5f60a942f904e7
Slowest unit: 75 s:
artifact_prefix='./'; Test unit written to ./slow-unit-973e0cccb7e5db0a29874b2122691a26de0558f6
Slowest unit: 87 s:
artifact_prefix='./'; Test unit written to ./slow-unit-bc0fe6f6e4a65c9da5be25be7d5536bad6f8e24c
Slowest unit: 113 s:
artifact_prefix='./'; Test unit written to ./slow-unit-e92f8cda91b50f98d77dcb7e56b3b41fceb5e78b
Slowest unit: 127 s:
artifact_prefix='./'; Test unit written to ./slow-unit-59a4542a4657116270ab3457d83c217c1996d7c8
Slowest unit: 140 s:
artifact_prefix='./'; Test unit written to ./slow-unit-82b80732dc85e5c11f5a8255b82d31f3f13c9244
Slowest unit: 156 s:
artifact_prefix='./'; Test unit written to ./slow-unit-280ddb2c92b53aa298cb88d11c77ccd22e1de7ab
Slowest unit: 182 s:
artifact_prefix='./'; Test unit written to ./slow-unit-9c08c09c1d630fade4f699fdd1eeaa53325dcce3
Slowest unit: 211 s:
artifact_prefix='./'; Test unit written to ./slow-unit-ec3af16aeb3b967f9b63edc7266a71f52825c616

maflcko · 2023-11-07T15:54:54Z

Ah, so I guess this is expected, because it is loading all fuzz inputs. (The same would happen on merge or on -runs=1).

This should be fixed the next time you pull from current Bitcoin Core master.

Leave a comment here, if not.

maflcko · 2023-11-07T15:54:58Z

See bitcoin/bitcoin@2b3f43b

murchandamus · 2023-11-07T17:55:52Z

I see, I’ll update to the latest upstream/master and try fuzzing on utxo_total_supply a bit

murchandamus · 2023-11-07T20:25:37Z

➜  qa-fuzz git:(fuzz-master) ✗ FUZZ=utxo_total_supply src/test/fuzz/fuzz -runs=1 ../qa-assets-active-fuzzing/fuzz_seed_corpus/utxo_total_supply
INFO: Running with entropic power schedule (0xFF, 100).
INFO: Seed: 2198204795
INFO: Loaded 1 modules   (361490 inline 8-bit counters): 361490 [0x55b85189bf90, 0x55b8518f43a2), 
INFO: Loaded 1 PC tables (361490 PCs): 361490 [0x55b8518f43a8,0x55b851e784c8), 
INFO:     3247 files found in ../qa-assets-active-fuzzing/fuzz_seed_corpus/utxo_total_supply
INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 1048576 bytes
INFO: seed corpus: files: 3247 min: 1b max: 4194303b total: 11985543b rss: 146Mb
#32	pulse  cov: 12753 ft: 12865 corp: 21/21b exec/s: 10 rss: 261Mb
#64	pulse  cov: 12767 ft: 12939 corp: 37/47b exec/s: 10 rss: 313Mb
#128	pulse  cov: 12931 ft: 17427 corp: 59/107b exec/s: 9 rss: 318Mb
#256	pulse  cov: 13150 ft: 22006 corp: 104/266b exec/s: 8 rss: 318Mb
#512	pulse  cov: 13401 ft: 30279 corp: 251/1233b exec/s: 8 rss: 318Mb
#1024	pulse  cov: 13444 ft: 35515 corp: 529/4212b exec/s: 8 rss: 318Mb
#2048	pulse  cov: 13538 ft: 53104 corp: 1220/25Kb exec/s: 4 rss: 318Mb
#3249	INITED cov: 14240 ft: 63278 corp: 1644/151Kb exec/s: 0 rss: 409Mb
#3249	DONE   cov: 14240 ft: 63278 corp: 1644/151Kb lim: 12691 exec/s: 0 rss: 409Mb
Done 3249 runs in 4797 second(s)

That’s better, but still a bit slow. Given that the fuzz test changed, I would expect that the runtime should already be reduced but even so. :-/

maflcko · 2023-11-08T07:59:56Z

I don't know how to reduce the runtime further. Maybe assumeutxo can be used? If not, the only alternative would be to remove the target.

murchandamus · 2024-05-17T17:49:54Z

I got a new faster computer. I fuzzed utxo_total_supply a bit, and still only got about 2 exec/s on that fuzz target. Then I fuzzed an hour each on a number of very short max_len (1 byte, 4 bytes, 16 bytes, 64 bytes, and 256 bytes), and built a new corpus from my fuzz seeds via set_cover_merge=1.

Doing a single run of all seeds on the main branch takes me 217 seconds:

➜  qa-fuzz git:(fuzz-master) ✗ time FUZZ=utxo_total_supply src/test/fuzz/fuzz ../qa-assets/fuzz_seed_corpus/utxo_total_supply -runs=1 
INFO: Running with entropic power schedule (0xFF, 100).
INFO: Seed: 1937175595
INFO: Loaded 1 modules   (546442 inline 8-bit counters): 546442 [0x5e51a781cc60, 0x5e51a78a22ea), 
INFO: Loaded 1 PC tables (546442 PCs): 546442 [0x5e51a78a22f0,0x5e51a80f8b90), 
INFO:      981 files found in ../qa-assets/fuzz_seed_corpus/utxo_total_supply
INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 316441 bytes
INFO: seed corpus: files: 981 min: 1b max: 316441b total: 851769b rss: 130Mb
#128	pulse  cov: 15662 ft: 47334 corp: 123/2130b exec/s: 42 rss: 288Mb
#256	pulse  cov: 15739 ft: 54301 corp: 251/9253b exec/s: 17 rss: 290Mb
#512	pulse  cov: 16422 ft: 64810 corp: 494/38Kb exec/s: 7 rss: 290Mb
#983	INITED cov: 16510 ft: 68164 corp: 764/145Kb exec/s: 4 rss: 315Mb
#983	DONE   cov: 16510 ft: 68164 corp: 764/145Kb lim: 16384 exec/s: 4 rss: 315Mb
Done 983 runs in 217 second(s)
FUZZ=utxo_total_supply src/test/fuzz/fuzz  -runs=1  145.60s user 47.53s system 88% cpu 3:37.46 total

Doing a single run on my new corpus took only 101 seconds:

➜  qa-fuzz git:(fuzz-master) ✗ time FUZZ=utxo_total_supply src/test/fuzz/fuzz /tmp/utxo_total_supply -runs=1
INFO: Running with entropic power schedule (0xFF, 100).
INFO: Seed: 964102661
INFO: Loaded 1 modules   (546442 inline 8-bit counters): 546442 [0x5fb20591ec60, 0x5fb2059a42ea), 
INFO: Loaded 1 PC tables (546442 PCs): 546442 [0x5fb2059a42f0,0x5fb2061fab90), 
INFO:      444 files found in /tmp/utxo_total_supply
INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 131076 bytes
INFO: seed corpus: files: 444 min: 1b max: 131076b total: 263162b rss: 130Mb
#128	pulse  cov: 15709 ft: 53620 corp: 126/5616b exec/s: 14 rss: 273Mb
#256	pulse  cov: 16407 ft: 64414 corp: 251/23Kb exec/s: 5 rss: 273Mb
#446	INITED cov: 16441 ft: 67734 corp: 434/125Kb exec/s: 4 rss: 286Mb
#446	DONE   cov: 16441 ft: 67734 corp: 434/125Kb lim: 22550 exec/s: 4 rss: 286Mb
Done 446 runs in 101 second(s)
FUZZ=utxo_total_supply src/test/fuzz/fuzz /tmp/utxo_total_supply -runs=1  65.42s user 22.74s system 87% cpu 1:41.30 total

I do notice that my new corpus only achieved a cov: 16441 vs the main branch getting cov: 16510, but I’m gonna fuzz a bit more with low max_len to see if I can speed up the corpus by replacing longer seeds with short stuff.

dergoegge · 2024-05-17T17:53:11Z

iirc utxo_total_supply is mostly bottle-necked by IO, are you using a ramdisk/tmpfs for /tmp? (that should help a little)

murchandamus · 2024-05-17T19:10:40Z

I was not, but either way, one being twice as fast as the other wouldn’t change much, should it?

This was referenced Jan 4, 2024

Add Murch’s fuzz seed in January 2024 #164

Merged

fuzz: Fix timeouts bitcoin/bitcoin#28812

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`utxo_total_supply` extremely slow #158

`utxo_total_supply` extremely slow #158

murchandamus commented Nov 7, 2023 •

edited

maflcko commented Nov 7, 2023

murchandamus commented Nov 7, 2023 •

edited

maflcko commented Nov 7, 2023

maflcko commented Nov 7, 2023

murchandamus commented Nov 7, 2023

murchandamus commented Nov 7, 2023 •

edited

maflcko commented Nov 8, 2023

murchandamus commented May 17, 2024 •

edited

dergoegge commented May 17, 2024

murchandamus commented May 17, 2024

utxo_total_supply extremely slow #158

utxo_total_supply extremely slow #158

Comments

murchandamus commented Nov 7, 2023 • edited

maflcko commented Nov 7, 2023

murchandamus commented Nov 7, 2023 • edited

maflcko commented Nov 7, 2023

maflcko commented Nov 7, 2023

murchandamus commented Nov 7, 2023

murchandamus commented Nov 7, 2023 • edited

maflcko commented Nov 8, 2023

murchandamus commented May 17, 2024 • edited

dergoegge commented May 17, 2024

murchandamus commented May 17, 2024

`utxo_total_supply` extremely slow #158

`utxo_total_supply` extremely slow #158

murchandamus commented Nov 7, 2023 •

edited

murchandamus commented Nov 7, 2023 •

edited

murchandamus commented Nov 7, 2023 •

edited

murchandamus commented May 17, 2024 •

edited