- Remove genshi module? Last release in January 2014!
- upload:
- add Q1 and Q3: keys 'q1' and 'q3'
- "description"
- bm_python_startup: add again python metadata, to get back the Python version in benchmark suite metadata. Same for bm_hg_startup.
- Remove compare command or reimplement it using pyperf compare_to
- Add a --log option to create a log file. Use the logging module and replace print() with logger.error().
- Write a test to ensure that benchmarks listed in groups exist
- Decide if sqlalchemy benchmarks should only benchmark SELECT ALL or INSERT+SELECT?
- Run pep8 on Travis
- compile: add metadata on compiler option, especially LTO and PGO flags
- Python 3.5.0beta2: SyntaxError in parse() of chameleon/astutil.py, on parsing "def func(target):". Failed commits:
- CPython 8a8f453c5a6d
- Skip bm_django_template on Python 3 older than 3.4: Python 3.3.6 final: django_template: django/utils/module_loading.py: "from importlib.util import find_spec as importlib_find" ImportError: cannot import name find_spec CPython 75e3630c6071
- sympy uses deprecated inspect.getargspec(), function removed from 3.6a0 then reintroduced. TODO: fix sympy upstream? sympy: sympy/core/function.py "inspect.getargspec(cls.eval)" module 'inspect' has no attribute 'getargspec' CPython 0d30940dd256
- https://morepypy.blogspot.fr/2016/11/vectorization-extended-powerpc-and-s390x.html
- https://bitbucket.org/plan_rich/numpy-benchmark fork of https://bitbucket.org/mikefc/numpy-benchmark/src to use pyperf
Repository: https://bitbucket.org/pypy/benchmarks/
Different from pyperformance?
- json_bench
Todo:
- pypy_interp
pyxl_bench
PyPI version 1.0: https://pypi.python.org/pypi/pyxl (2012) Old repo: https://github.com/awable/pyxl (last commit: Feb 2013) Python2: https://github.com/dropbox/pyxl (latest commit: Aug 2016) Python3: https://github.com/gvanrossum/pyxl3 (latest commit: Apr 2016)
sphinx
benchmarks.py: invoke sphinx-build.py on lib/cpython-doc/ cpython-doc/: 24 MB
- trans2_annotate
- trans2_backendopt
- trans2_database
- trans2_rtype
- trans2_source
- twisted_iteration
- twisted_names
- twisted_pb
- twisted_tcp
Deliberate choice to not add it:
- eparse: https://pypi.python.org/pypi/Monte 0.0.11 was released in 2013, no tarball on PyPI, only on SourceForge
- krakatau: https://github.com/Storyyeller/Krakatau is not on PyPI, but it seems actively developed
- slowspitfire, spitfire, spitfire_cstringio: not on PyPI
- rietveld: not on PyPy
Done:
- ai (called bm_nqueens in pyperformance)
- bm_chameleon
- bm_mako
- chaos
- crypto_pyaes
- deltablue
- django (called django_template in pyperformance)
- dulwich_log
- fannkuch
- float
- genshi_text
- genshi_xml
- go
- hexiom2
- html5lib
- mdp
- meteor-contest
- nbody_modified (called nbody in pyperformance)
- nqueens
- pidigits
- pyflate-fast (called pyflate in pyperformance)
- raytrace-simple (called raytrace in pyperformance)
- richards
- scimark_fft
- scimark_lu
- scimark_montecarlo
- scimark_sor
- scimark_sparsematmult
- spectral-norm
- sqlalchemy_declarative
- sqlalchemy_imperative
- sqlitesynth (called pyflate in sqlite_synth)
- sympy_expand
- sympy_integrate
- sympy_str
- sympy_sum
- telco
Add benchmarks from the Pyston benchmark suite: https://github.com/dropbox/pyston-perf and convince Pyston to use pyperformance :-)
TODO:
- django_lexing
- django_migrate
- django_template2
- django_template3_10x
- django_template3
- django_template
- fasta (it's different than pyperformance "regex_dna")
- interp2
- pyxl_bench_10x
- pyxl_bench2_10x
- pyxl_bench2
- pyxl_bench
- sre_parse_parse
- virtualenv_bench2
- virtualenv_bench
Done:
- chaos
- deltablue
- fannkuch, fannkuch_med
- nbody
- pidigits: pyston has a flat implementation, single function
- raytrace, raytrace_small: use "--width=80 --height=60" cmdline option to get raytrace_small profile
- richards
- sqlalchemy_imperative, sqlalchemy_imperative2, sqlalchemy_imperative2_10x: use --rows cmdline option to control the number of SQL rows
- sre_compile_ubench: pyperformance has a much more complete benchmark on regex