A low-level OpenQASM benchmark suite for NISQ evaluation and simulation. Please see our paper (attached and arXiv) for details.
Latest version: 1.0
In this repository you will find a low-level and light-weighted benchmark suite based on IBM OpenQASM language (see spec). It collects commonly seen quantum algorithms and routines from various domains including chemistry, simulation, linear algebra, searching, optimization, arithmetic, machine learning, fault tolerance, cryptography, etc. QASMBench trades-off between generality and usability, covering the number of qubits ranging from 2 to 60K, and the circuit depth from 4 to 12M. We set most of the benchmarks with qubits less than 16 so they can be directly verified on IBM's public-available quantum machine -- IBM Quantum Experience.
OpenQASM (Open Quantum Assembly Language) is a low-level quantum intermediate representation (IR) for quantum instructions, similar to the traditional Hardware-Description-Language (HDL) like Verilog and VHDL. OpenQASM is the open-source unified low-level assembly language for IBM quantum machines publically available on cloud that have been investigated and verified by many existing research works. Several popular quantum software frameworks use OpenQASM as one of their output-formats, including Qiskit, Cirq, Scaffold, ProjectQ, etc.
The Quantum Information Software Kit (Qiskit) is a quantum software developed by IBM. It is based on Python. OpenQASM can be generated from Qiskit via:
QuantumCircuit.qasm()
Cirq is a quantum software framework from Google. OpenQASM can be generated from Cirq (not fully compatible) via:
cirq.Circuit.to_qasm()
Scaffold is a quantum programming language embedded in the C/C++ programming language based on the LLVM compiler toolchain. A Scaffold program can be compiled by Scaffcc to OpenQASM via "-b" compiler option.
ProjectQ is a quantum software platform developed by Steiger et al. from ETH Zurich. The official website is here. ProjectQ can generate OpenQASM when using IBM quantum machines as the backends:
IBMBackend.get_qasm()
Depending on the number of qubits used, QASMBench includes three categories. For the introduction of the benchmarking routines under each category, please see our paper for detail. For each benchmark in the following tables, we list its name, brief description, and the algorithm category it belongs to, which is based on this Nature paper by adding the categories of quantum arithmetic, quantum machine learning and quantum communication.
The 'Gates' here refers to the number of Standard OpenQASM gates (see our paper) but excluding those gates in a branching if statement. It is known that physical qubits in an NISQ device follow a certain topology. Since the 2-qubit gates such as CNOT or CX can only be performed between two adjacent physical qubits, a series of SWAP operations can be required to move the relevant qubits until they become directly-connected. This is an important issue in machine-specific mapping and optimization, implying a significant potential overhead. Consequently, we also list the number of CNOT gates in the tables.
Qunatum circuits using 2 to 5 qubits.
| Benchmark | Description | Algorithm | Qubits | Gates | CNOT | Source |
|---|---|---|---|---|---|---|
| wstate | W-state preparation and assessment | Logical Operation | 3 | 30 | 9 | OpenQASM |
| adder | Quantum ripple-carry adder | Quantum Arithmetic | 4 | 23 | 10 | Scaffold |
| basis_change | Transform the single-particle baseis of an linearly connected electronic structure | Quantum Simulation | 3 | 53 | 10 | OpenFermion |
| basis_trotter | Implement Trotter steps for molecule LiH at equilibrium geometry | Quantum Simulation | 4 | 1626 | 582 | OpenFermion |
| cat_state | Coherent superposition of two coherent states with opposite phase | Logical Operation | 4 | 4 | 3 | Scaffold |
| deutsch | Deutsch algorithm with 2 qubits for f(x) = x | Hidden Subgroup | 2 | 5 | 1 | OpenQASM |
| error_correctiond3 | Error correction with distance 3 and 5 qubits | Error Correction | 5 | 114 | 49 | Ref |
| fredkin | Controlled-swap gate | Logical Operation | 3 | 19 | 8 | Scaffold |
| grover | Grover’s algorithm | Search and Optimization | 2 | 16 | 2 | AgentANAKIN |
| hs4 | Hidden subgroup problem | Hidden Subgroup | 4 | 28 | 4 | Scaffold |
| inverseqft | Performs an exact inversion of quantum Fourier tranform | Hidden Subgroup | 4 | 8 | 0 | OpenQASM |
| ipea | Iterative phase estimation algorithm | Hidden Subgroup | 2 | 68 | 30 | OpenQASM |
| iswap | An entangling swapping gate | Logical Operation | 2 | 9 | 2 | OpenQASM |
| linearsolver | Solver for a linear equation of one qubit | Linear Equation | 3 | 19 | 4 | Ref |
| lpn | Learning parity with noise | Machine Learning | 5 | 11 | 2 | sampaio96 |
| pea | Phase estimation algorithm | Hidden Subgroup | 5 | 98 | 42 | OpenQASM |
| qec_sm | Repetition code syndrome measurement | Error Correction | 5 | 5 | 4 | OpenQASM |
| qft | Quantum Fourier transform | Hidden Subgroupe | 4 | 36 | 12 | OpenQASM |
| qec_en | Quantum repetition code encoder | Error Correction | 5 | 25 | 10 | sampaio96 |
| teleportation | Quantum teleportation | Quantum Communication | 3 | 8 | 2 | Ref |
| toffoli | Toffoli gate | Logical Operation | 3 | 18 | 6 | Scaffold |
| variational | Variational ansatz for a Jellium Hamiltonian with a linear-swap network | Quantum Simulation | 4 | 54 | 16 | OpenFermion |
| vqe_uccsd | Variational quantum eigensolver with UCCSD | Linear Equation | 4 | 220 | 88 | Scaffold |
| shor | Shor’s algorithm | Hidden Subgroup | 5 | 64 | 30 | Qiskit |
| bell | Circuit equivalent to Bell inequality test | Logic Operation | 4 | 33 | 7 | Cirq |
Quantum circutis using 6 to 15 qubits.
| Benchmark | Description | Algorithm | Qubits | Gates | CNOT | Source |
|---|---|---|---|---|---|---|
| adder | Quantum ripple-carry adder | Quantum Arithmetic | 10 | 142 | 65 | OpenQASM |
| bv | Bernstein-Vazirani algorithm | Hidden Subgroup | 14 | 41 | 13 | OpenQASM |
| cc | Counterfeit coin finding problem | Search and Optimization | 12 | 22 | 11 | OpenQASM |
| ising | Ising model simulation via QC | Quantum Simulation | 10 | 480 | 90 | Scaffold |
| multiply | Performing 3×5 in a quantum circuit | Quantum Arithmetic | 13 | 98 | 40 | AgentANAKIN |
| qf21 | Using quantum phase estimation to factor the number 21 | Hidden Subgroup | 15 | 311 | 115 | AgentANAKIN |
| qft | Quantum Fourier transform | Hidden Subgroup | 15 | 540 | 210 | OpenQASM |
| qpe | Quantum phase estimation algorithm | Hidden Subgroup | 9 | 123 | 43 | AgentANAKIN |
| seca | Shor's error correction algorithm for teleportation | Error Correction | 11 | 216 | 84 | AgentANAKIN |
| simons | Simon’s algorithm | Hidden Subgroup | 6 | 44 | 14 | AgentANAKIN |
| vqe_uccsd | Variational quantum eigensolver with UCCSD | Linear Equation | 6 | 2282 | 1052 | Scaffold |
| vqe_uccsd | Variational quantum eigensolver with UCCSD | Linear Equation | 8 | 10808 | 5488 | Scaffold |
| qaoa | Quantum approximate optimization algorithm | Search and Optimization | 6 | 270 | 54 | Cirq |
| bb84 | A quantum key distribution circuit | Quantum Communication | 8 | 27 | 0 | Cirq |
| multipler | Quantum multipler | Quantum Arithmetic | 15 | 574 | 246 | Cirq |
Quantum circuits using more than 15 qubits.
| Benchmark | Description | Algorithm | Qubits | Gates | CNOT | Source |
|---|---|---|---|---|---|---|
| ising | Ising model simulation via QC | Quantum Simulation | 500 | 5494 | 998 | Scaffold |
| ising | Ising model simulation via QC | Quantum Simulation | 1000 | 10994 | 1998 | Scaffold |
| bigadder | Quantum ripple-carry adder | Quantum Arithmetic | 18 | 284 | 130 | OpenQASM |
| bv | Bernstein-Vazirani algorithm | Hidden Subgroup | 19 | 56 | 18 | OpenQASM |
| cc | Counterfeit coin finding problem via QC | Hidden Subgroup | 18 | 34 | 17 | OpenQASM |
| qft | Quantum Fourier tranform | Hidden Subgroup | 20 | 970 | 380 | OpenQASM |
| square_root | Computing the square root of an number via amplitude amplification | Quantum Arithmetic | 480 | 16064 | 6274 | Scaffold |
| class_number | Compute the class group of a real quadratic number field | Hidden Subgroups | 60052 | 31110504 | 12460637 | Scaffold |
OpenQASM header file that defines all the gates. Please see OpenQASM and our [paper]((qasmbench.pdf) for details.
Each benchmark folder include the following file:
- bench.qasm: OpenQASM source file.
- bench.png: Visualization of the circuit from IBM QE.
- res_bench.png: Running results in state-vector from IBM QE.
- bench.cuh: Source file for our quantum simulator that will be released later.
If you see "bench_sim" with a "sim" suffix, it means the results are generated by IBM QE simulator rather than the real quantum machines. The real quantum machine could not run successfully with them.
The small-scale benchmarks (except basis-trotter) can be directly uploaded and verified on real Quantum Machines IBM Quantum Experience.
The medium-scale benchmarks can be either validated by real quantum machines or simulators in IBM Quantum Experience.
Some of the large-scale benchmarks can be validated on IBM simulators.
Ang Li, Pacific Northwest National Laboratory (PNNL)
Sriram Krishnamoorthy, Pacific Northwest National Laboratory (PNNL)
And also the original authors that developed these quantum routines.
For research articles, please cite our paper:
- Ang Li, Sriram Krishnamoorthy, "QASMBench: A Low-level QASM Benchmark Suite for NISQ Evaluation and Simulation" [arXiv:2005.13018].
Bibtex:
@article{li2020qasmbench,
title={QASMBench: A Low-level QASM Benchmark Suite for NISQ Evaluation and Simulation},
author={Li, Ang and Krishnamoorthy, Sriram},
journal={arXiv preprint arXiv:2005.13018},
year={2020}
}
This project is licensed under the BSD License, see LICENSE file for details.
We thank the many developers and open-source community for providing these awesome quantum circuits online so we are able to collect and form this benchmark suite. This work was supported by PNNL's Quantum Algorithms, Software, and Architectures (QUASAR) LDRD Initiative. The Pacific Northwest National Laboratory (PNNL) is operated by Battelle for the U.S. Department of Energy (DOE) under contract DE-AC05-76RL01830.
Please contact us If you'd like to add your circuits into the benchmark suite or you'd like to remove your circuits from the suite.