Benchmarks #44
Comments
|
after replacing all the |
|
The script could probably do with a little more error checking and also the ability to benchmark against some real search terms as these do impact the benchmarks quite significantly when you have a number of terms to search for. All this said the -B option is typically faster with my current searches (10 search terms, between 8-12 characters) |
correct.
keygen rate isn't significantly affected by search terms, unless the list is like really huge and linear search method is used which would start being slowdown at that point, or keys are being found at like really high rate in which case resets of keygen state would also start affecting performance. so idk if adding different search term stuff would affect much. |
|
Ubuntu20.04 Results: |
|
I cleaned the script up a little bit to use loops and be easier to modify, and to not leave lots of files in the top-level dir. I also adapted it to work on a macOS Big Sur system. I am pasting it here in case anybody else wants it. It also still works on Linux (but you have to comment-out the macOS stuff and uncomment the Linux stuff at top of file). Results: #!/usr/bin/env bash
set -e
BINDIR=`pwd`/bin
BENCHDIR=`pwd`/benchmarks
#Uncomment for Linux:
#FLAGS="ref10 amd64-51-30k amd64-64-24k donna donna-sse2"
#libdir="/usr/local/lib"
#includedir="/usr/local/include"
#Below is for macOS Big Sur with MacPorts, comment it out if on Linux
FLAGS="ref10 donna donna-sse2"
libdir="/opt/local/lib"
includedir="/opt/local/include"
if [[ -z $NO_BUILD ]]; then
echo "Building mkp224o"
mkdir -p "$BINDIR"
for flag in $FLAGS; do
OUTFILE="${BINDIR}"/mkp224o-"$flag"
if [[ ! -f "$OUTFILE" ]]; then
make clean
./autogen.sh
CFLAGS="-fomit-frame-pointer -O3 -I${includedir}" LDFLAGS="-L${libdir}" \
./configure --enable-"${flag}"
make
cp -fpv mkp224o "$OUTFILE"
else
echo "already built $OUTFILE"
fi
done
echo " done!"
fi
files=""
if [[ -z $NO_CREATE ]]; then
echo "Creating benchmarks"
mkdir -p "$BENCHDIR"
rm -fv "$BENCHDIR"/mkp224o*
fi
for flag in $FLAGS; do
[[ -z $NO_CREATE ]] && echo "--${flag} -s -T"
[[ -z $NO_CREATE ]] && "$BINDIR"/mkp224o-"$flag" -s -T benchmarkkeyhardtofind 2>&1 | head -n 8 | tail -n 3 > "$BENCHDIR"/mkp224o-"$flag"
files="${files} $BENCHDIR/mkp224o-${flag}"
done
for flag in $FLAGS; do
[[ -z $NO_CREATE ]] && echo "--${flag} -B -s -T"
[[ -z $NO_CREATE ]] && "$BINDIR"/mkp224o-"$flag" -B -s -T benchmarkkeyhardtofind 2>&1 | head -n 8 | tail -n 3 > "$BENCHDIR"/mkp224o-"$flag"-B
files="${files} $BENCHDIR/mkp224o-${flag}-B"
done
[[ -z $NO_CREATE ]] && echo "done!"
echo -n "Calculating benchmark results"
regex='^>calc/sec:([0-9]+\.[0-9]+).*'
values=()
for file in $files; do
for i in $(cat "$file"); do
if [[ $i =~ $regex ]]; then
values+=(${BASH_REMATCH[1]})
fi
done
n="(${values[@]})/3"
line=" "
BASE_NAME=$(basename $file)
calc=$(bc <<< "${n// /+}")
line="${line:${#file}}"
result=$(printf "%s %s %s" $BASE_NAME $line $calc)
results+=("$result")
values=()
echo -n "."
done
echo " done!"
echo -e
echo "Results:"
mapfile -t sorted < <(printf "%s\n" "${results[@]}" | sort -r -k2,2n -k1,1)
for result in "${sorted[@]}"; do
echo ${result}
done |
|
Would love to see a windows version of this :) |
|
I can try and hack it to work on windows but with mingw & bash probably.. since.. well .. powershell is beyond my capabilities. |
That's perfectly fine, I'd appreciate that! |
#! /usr/bin/env bash
# mkp224o-benchmark.sh - a benchmarking wrapper for mkp224o
# print live stats in a pretty format
# loop different binaries and extra_options
# to find the fastest combination
# MIT license
# select range of samples
samples_from=1; samples_to=5 # slow
samples_from=1; samples_to=1 # too fast. shows speed 0 for some commands
samples_from=2; samples_to=2 # show only the second sample
extra_options_list=()
#extra_options_list+=("")
extra_options_list+=("-B")
set -e
bindir=$(dirname $(which mkp224o-donna)) # also works on nixos linux
printf "# args:"; for arg in "$@"; do printf " %q" "$arg"; done; echo
echo -e "# speed\tround\tsample\tcommand"
round_num=0
while true
do
round_num=$((round_num + 1))
for bin in $bindir/mkp224o-*
do
binname=$(basename $bin)
if [[ $# == 0 ]]
then
$bin -h # show help
echo
echo "$(basename "$0"): hint: run this script with extra_options: -qxs"
echo "$(basename "$0"): hint: pipe the output through: | tee -a stats.txt"
exit 1
fi
for extra_options in "${extra_options_list[@]}"
do
command="$binname"
[[ "$extra_options" != "" ]] && command+=" $extra_options"
sample_num=0
while read line
do
speed=$(echo "$line" | sed -E "s/^>calc\/sec:([0-9]+)\..*$/\1/")
if [[ "$speed" == "" ]]; then echo "# $line"; continue; fi
sample_num=$((sample_num + 1))
(( sample_num < samples_from )) && continue
echo -e "${speed}\t${round_num}\t${sample_num}\t${command}"
(( sample_num >= samples_to )) && break # stop the binary
done < <(
# run the binary
$bin "$@" $extra_options 2>&1
)
done
done
doneexample output 4 core x 2.5 GHz Intel Core i5 128 core x 2.9 GHz AMD Ryzen |
This script gives you a build for each flag and a benchmark sorted for each flag and with the -B option for an average over 30s. It excludes the first 10s of results. The mkp224o builds are copied into /usr/local/bin. Requires the
bccommand.If you set the NO_CREATE env variable before you run the command it won't recreate the benchmark results
e.g.
NO_CREATE=1 ./benchmark.shIf you set the NO_BUILD env variable it wont attempt to build mkp224o.
NO_BUILD=1 ./benchmark.shOutput looks like:
Script
The text was updated successfully, but these errors were encountered: