improvements to the non-clifford functionality #259

Krastanov · 2024-04-12T13:25:26Z

No description provided.

Krastanov · 2024-04-12T13:28:15Z

@hongyehu , this is what be built yesterday during the meeting, with a bit of polish. I tried to make the commits well separated and easy to read one by one. Hopefully this will make it easier to follow as you do not need to read the entire diff.

I will wait for tests to pass and then merge. I will add a long list of issues to track necessary enhancements.

github-actions · 2024-04-12T13:46:28Z

Benchmark Result

Judge result

Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl

Job Properties

Time of benchmarks:

Target: 12 Apr 2024 - 13:38
Baseline: 12 Apr 2024 - 13:45

Package commits:

Target: 4fef66
Baseline: 64f695

Julia commits:

Target: 1ae41a
Baseline: 1ae41a

Julia command flags:

Target: None
Baseline: None

Environment variables:

Target: None
Baseline: None

Results

A ratio greater than 1.0 denotes a possible regression (marked with ❌), while a ratio less
than 1.0 denotes a possible improvement (marked with ✅). Only significant results - results
that indicate possible regressions or improvements - are shown below (thus, an empty table means that all
benchmark results remained invariant between builds).

ID	time ratio	memory ratio
`["ecc", "evaluate_decoder", "shor_bp_shorsyn"]`	0.93 (5%) ✅	0.98 (1%) ✅
`["ecc", "evaluate_decoder", "shor_pybp_comm"]`	1.10 (5%) ❌	1.00 (1%)
`["ecc", "evaluate_decoder", "shor_pybp_naivesyn"]`	0.75 (5%) ✅	1.00 (1%)
`["ecc", "evaluate_decoder", "shor_pybp_shorsyn"]`	0.83 (5%) ✅	1.00 (1%)
`["ecc", "evaluate_decoder", "toric8_bp_comm"]`	1.03 (5%)	1.04 (1%) ❌
`["ecc", "evaluate_decoder", "toric8_bp_naivesyn"]`	1.03 (5%)	1.03 (1%) ❌
`["stabilizer", "canon", "diag_rref500"]`	1.06 (5%) ❌	1.00 (1%)
`["stabilizer", "trace", "destabilizer"]`	0.91 (5%) ✅	1.00 (1%)

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

["circuitsim", "mctrajectories"]
["circuitsim", "mctrajectories_sumtype"]
["circuitsim", "mctrajectories_union"]
["circuitsim", "pftrajectories"]
["circuitsim", "pftrajectories_sumtype"]
["circuitsim", "pftrajectories_union"]
["clifford", "dense"]
["clifford", "symbolic"]
["ecc", "evaluate_decoder"]
["pauli", "mul"]
["stabilizer", "canon"]
["stabilizer", "project"]
["stabilizer", "tensor"]
["stabilizer", "trace"]

Julia versioninfo

Target

Julia Version 1.12.0-DEV.334
Commit 1ae41a2c0a3 (2024-04-12 03:14 UTC)
Build Info:
 Official https://julialang.org/ release
Platform Info:
 OS: Linux (x86_64-linux-gnu)
 Ubuntu 22.04.4 LTS
 uname: Linux 6.5.0-1017-azure #17~22.04.1-Ubuntu SMP Sat Mar 9 04:50:38 UTC 2024 x86_64 x86_64
 CPU: AMD EPYC 7763 64-Core Processor: 
 speed user nice sys idle irq
 #1 0 MHz 2544 s 0 s 289 s 6455 s 0 s
 #2 0 MHz 1822 s 0 s 288 s 7164 s 0 s
 #3 0 MHz 2181 s 0 s 292 s 6819 s 0 s
 #4 0 MHz 3122 s 0 s 272 s 5903 s 0 s
 Memory: 15.606498718261719 GB (13358.14453125 MB free)
 Uptime: 935.01 sec
 Load Avg: 1.0 1.09 0.8
 WORD_SIZE: 64
 LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Baseline

Julia Version 1.12.0-DEV.334
Commit 1ae41a2c0a3 (2024-04-12 03:14 UTC)
Build Info:
 Official https://julialang.org/ release
Platform Info:
 OS: Linux (x86_64-linux-gnu)
 Ubuntu 22.04.4 LTS
 uname: Linux 6.5.0-1017-azure #17~22.04.1-Ubuntu SMP Sat Mar 9 04:50:38 UTC 2024 x86_64 x86_64
 CPU: AMD EPYC 7763 64-Core Processor: 
 speed user nice sys idle irq
 #1 0 MHz 3520 s 0 s 382 s 9741 s 0 s
 #2 0 MHz 2872 s 0 s 360 s 10400 s 0 s
 #3 0 MHz 3337 s 0 s 390 s 9915 s 0 s
 #4 0 MHz 4441 s 0 s 374 s 8831 s 0 s
 Memory: 15.606498718261719 GB (13207.7734375 MB free)
 Uptime: 1371.81 sec
 Load Avg: 1.05 1.1 0.94
 WORD_SIZE: 64
 LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Target result

Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl

Job Properties

Time of benchmark: 12 Apr 2024 - 13:38
Package commit: 4fef66
Julia commit: 1ae41a
Julia command flags: None
Environment variables: None

Results

Below is a table of this job's results, obtained by running the benchmarks.
The values listed in the ID column have the structure [parent_group, child_group, ..., key], and can be used to
index into the BaseBenchmarks suite to retrieve the corresponding benchmarks.
The percentages accompanying time and memory values in the below table are noise tolerances. The "true"
time/memory value for a given benchmark is expected to fall within this percentage of the reported value.
An empty cell means that the value was zero.

ID	time	GC time	memory	allocations
`["circuitsim", "mctrajectories", "q1001_r1"]`	14.956 ms (5%)		500.77 KiB (1%)	18019
`["circuitsim", "mctrajectories", "q101_r1"]`	169.877 μs (5%)		50.53 KiB (1%)	1818
`["circuitsim", "mctrajectories_sumtype", "q1001_r1"]`	13.279 ms (5%)		496 bytes (1%)	9
`["circuitsim", "mctrajectories_sumtype", "q101_r1"]`	117.250 μs (5%)		256 bytes (1%)	8
`["circuitsim", "mctrajectories_union", "q1001_r1"]`	12.998 ms (5%)		496 bytes (1%)	9
`["circuitsim", "mctrajectories_union", "q101_r1"]`	116.057 μs (5%)		256 bytes (1%)	8
`["circuitsim", "pftrajectories", "q1001_r1"]`	62.076 μs (5%)		93.80 KiB (1%)	2001
`["circuitsim", "pftrajectories", "q1001_r100"]`	173.353 μs (5%)		93.80 KiB (1%)	2001
`["circuitsim", "pftrajectories", "q1001_r10000"]`	1.102 ms (5%)		93.80 KiB (1%)	2001
`["circuitsim", "pftrajectories", "q101_r1"]`	6.201 μs (5%)		9.42 KiB (1%)	201
`["circuitsim", "pftrajectories_sumtype", "q1001_r1"]`	44.543 μs (5%)		93.84 KiB (1%)	2002
`["circuitsim", "pftrajectories_sumtype", "q1001_r100"]`	157.774 μs (5%)		93.84 KiB (1%)	2002
`["circuitsim", "pftrajectories_sumtype", "q1001_r10000"]`	1.084 ms (5%)		93.84 KiB (1%)	2002
`["circuitsim", "pftrajectories_sumtype", "q1001_r10000_fastrow"]`	5.856 ms (5%)		93.84 KiB (1%)	2002
`["circuitsim", "pftrajectories_sumtype", "q101_r1"]`	4.518 μs (5%)		9.47 KiB (1%)	202
`["circuitsim", "pftrajectories_union", "q1001_r1"]`	22.622 μs (5%)		96 bytes (1%)	2
`["circuitsim", "pftrajectories_union", "q1001_r100"]`	132.046 μs (5%)		96 bytes (1%)	2
`["circuitsim", "pftrajectories_union", "q1001_r10000"]`	1.056 ms (5%)		96 bytes (1%)	2
`["circuitsim", "pftrajectories_union", "q101_r1"]`	2.334 μs (5%)		96 bytes (1%)	2
`["clifford", "dense", "cnot250_on_dense500_destab"]`	11.823 ms (5%)		512 bytes (1%)	8
`["clifford", "dense", "cnot250_on_dense500_stab"]`	5.788 ms (5%)		512 bytes (1%)	8
`["clifford", "dense", "cnot250_on_diag500_destab"]`	1.123 ms (5%)		512 bytes (1%)	8
`["clifford", "dense", "cnot250_on_diag500_stab"]`	560.627 μs (5%)		512 bytes (1%)	8
`["clifford", "dense", "cnot_on_dense500_destab"]`	50.795 μs (5%)		368 bytes (1%)	10
`["clifford", "dense", "cnot_on_dense500_stab"]`	24.326 μs (5%)		368 bytes (1%)	10
`["clifford", "dense", "cnot_on_diag500_destab"]`	26.519 μs (5%)		368 bytes (1%)	10
`["clifford", "dense", "cnot_on_diag500_stab"]`	13.676 μs (5%)		368 bytes (1%)	10
`["clifford", "dense", "dense500_on_dense500_destab"]`	11.866 ms (5%)		512 bytes (1%)	8
`["clifford", "dense", "dense500_on_dense500_stab"]`	5.783 ms (5%)		512 bytes (1%)	8
`["clifford", "dense", "dense500_on_diag500_destab"]`	979.950 μs (5%)		512 bytes (1%)	8
`["clifford", "dense", "dense500_on_diag500_stab"]`	560.638 μs (5%)		512 bytes (1%)	8
`["clifford", "symbolic", "cnot250_on_dense500_destab"]`	1.449 ms (5%)
`["clifford", "symbolic", "cnot250_on_dense500_stab"]`	688.406 μs (5%)
`["clifford", "symbolic", "cnot250_on_diag500_destab"]`	1.159 ms (5%)
`["clifford", "symbolic", "cnot250_on_diag500_stab"]`	569.814 μs (5%)
`["clifford", "symbolic", "cnot_on_dense500_destab"]`	4.729 μs (5%)
`["clifford", "symbolic", "cnot_on_dense500_stab"]`	2.304 μs (5%)
`["clifford", "symbolic", "cnot_on_diag500_destab"]`	4.659 μs (5%)
`["clifford", "symbolic", "cnot_on_diag500_stab"]`	2.295 μs (5%)
`["ecc", "evaluate_decoder", "shor_bp_comm"]`	1.826 ms (5%)		1.78 MiB (1%)	42707
`["ecc", "evaluate_decoder", "shor_bp_naivesyn"]`	4.093 ms (5%)		3.61 MiB (1%)	81494
`["ecc", "evaluate_decoder", "shor_bp_shorsyn"]`	4.535 ms (5%)		3.72 MiB (1%)	83014
`["ecc", "evaluate_decoder", "shor_pybp_comm"]`	58.789 ms (5%)		4.76 MiB (1%)	144207
`["ecc", "evaluate_decoder", "shor_pybp_naivesyn"]`	119.038 ms (5%)		9.57 MiB (1%)	284018
`["ecc", "evaluate_decoder", "shor_pybp_shorsyn"]`	124.628 ms (5%)		9.66 MiB (1%)	285104
`["ecc", "evaluate_decoder", "shor_table_comm"]`	257.010 μs (5%)		241.42 KiB (1%)	4593
`["ecc", "evaluate_decoder", "shor_table_naivesyn"]`	896.275 μs (5%)		334.20 KiB (1%)	4790
`["ecc", "evaluate_decoder", "shor_table_shorsyn"]`	1.296 ms (5%)		427.73 KiB (1%)	5876
`["ecc", "evaluate_decoder", "toric8_bp_comm"]`	706.062 ms (5%)	18.157 ms	174.51 MiB (1%)	1046123
`["ecc", "evaluate_decoder", "toric8_bp_naivesyn"]`	1.429 s (5%)	25.180 ms	357.36 MiB (1%)	2163476
`["ecc", "evaluate_decoder", "toric8_bp_shorsyn"]`	1.412 s (5%)	27.075 ms	350.81 MiB (1%)	2142613
`["ecc", "evaluate_decoder", "toric8_pybp_comm"]`	459.145 ms (5%)		8.11 MiB (1%)	163696
`["ecc", "evaluate_decoder", "toric8_pybp_naivesyn"]`	968.377 ms (5%)		18.04 MiB (1%)	360388
`["ecc", "evaluate_decoder", "toric8_pybp_shorsyn"]`	983.377 ms (5%)		20.56 MiB (1%)	392802
`["ecc", "evaluate_decoder", "toric8_pymatch_comm"]`	4.156 ms (5%)		1.95 MiB (1%)	23176
`["ecc", "evaluate_decoder", "toric8_pymatch_naivesyn"]`	12.443 ms (5%)		5.58 MiB (1%)	79355
`["ecc", "evaluate_decoder", "toric8_pymatch_shorsyn"]`	21.042 ms (5%)		8.09 MiB (1%)	111769
`["ecc", "evaluate_decoder", "toric8_table_comm"]`	2.613 ms (5%)		1.64 MiB (1%)	23082
`["ecc", "evaluate_decoder", "toric8_table_naivesyn"]`	10.148 ms (5%)		4.93 MiB (1%)	79160
`["ecc", "evaluate_decoder", "toric8_table_shorsyn"]`	18.707 ms (5%)		7.44 MiB (1%)	111574
`["pauli", "mul", "100"]`	14.496 ns (5%)
`["pauli", "mul", "1000"]`	17.628 ns (5%)
`["pauli", "mul", "100000"]`	730.320 ns (5%)
`["pauli", "mul", "20000000"]`	192.289 μs (5%)
`["stabilizer", "canon", "cano500"]`	3.074 ms (5%)
`["stabilizer", "canon", "diag_cano500"]`	709.174 μs (5%)
`["stabilizer", "canon", "diag_gott500"]`	4.078 ms (5%)		5.19 MiB (1%)	34521
`["stabilizer", "canon", "diag_rref500"]`	675.983 μs (5%)
`["stabilizer", "canon", "gott500"]`	6.480 ms (5%)		5.19 MiB (1%)	34531
`["stabilizer", "canon", "md_cano500"]`	1.538 ms (5%)
`["stabilizer", "canon", "md_rref500"]`	1.568 ms (5%)
`["stabilizer", "canon", "rref500"]`	3.148 ms (5%)
`["stabilizer", "project", "destabilizer"]`	18.485 μs (5%)		288 bytes (1%)	5
`["stabilizer", "project", "stabilizer"]`	8.906 μs (5%)		80 bytes (1%)	2
`["stabilizer", "tensor", "diag_pow5_20"]`	1.405 ms (5%)		23.97 MiB (1%)	34
`["stabilizer", "tensor", "pow5_20"]`	2.948 μs (5%)		6.44 KiB (1%)	31
`["stabilizer", "trace", "destabilizer"]`	30.918 μs (5%)		80 bytes (1%)	2
`["stabilizer", "trace", "stabilizer"]`	23.854 μs (5%)		112 bytes (1%)	3

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

["circuitsim", "mctrajectories"]
["circuitsim", "mctrajectories_sumtype"]
["circuitsim", "mctrajectories_union"]
["circuitsim", "pftrajectories"]
["circuitsim", "pftrajectories_sumtype"]
["circuitsim", "pftrajectories_union"]
["clifford", "dense"]
["clifford", "symbolic"]
["ecc", "evaluate_decoder"]
["pauli", "mul"]
["stabilizer", "canon"]
["stabilizer", "project"]
["stabilizer", "tensor"]
["stabilizer", "trace"]

Julia versioninfo

Julia Version 1.12.0-DEV.334
Commit 1ae41a2c0a3 (2024-04-12 03:14 UTC)
Build Info:
 Official https://julialang.org/ release
Platform Info:
 OS: Linux (x86_64-linux-gnu)
 Ubuntu 22.04.4 LTS
 uname: Linux 6.5.0-1017-azure #17~22.04.1-Ubuntu SMP Sat Mar 9 04:50:38 UTC 2024 x86_64 x86_64
 CPU: AMD EPYC 7763 64-Core Processor: 
 speed user nice sys idle irq
 #1 0 MHz 2544 s 0 s 289 s 6455 s 0 s
 #2 0 MHz 1822 s 0 s 288 s 7164 s 0 s
 #3 0 MHz 2181 s 0 s 292 s 6819 s 0 s
 #4 0 MHz 3122 s 0 s 272 s 5903 s 0 s
 Memory: 15.606498718261719 GB (13358.14453125 MB free)
 Uptime: 935.01 sec
 Load Avg: 1.0 1.09 0.8
 WORD_SIZE: 64
 LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Baseline result

Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl

Job Properties

Time of benchmark: 12 Apr 2024 - 13:45
Package commit: 64f695
Julia commit: 1ae41a
Julia command flags: None
Environment variables: None

Results

Below is a table of this job's results, obtained by running the benchmarks.
The values listed in the ID column have the structure [parent_group, child_group, ..., key], and can be used to
index into the BaseBenchmarks suite to retrieve the corresponding benchmarks.
The percentages accompanying time and memory values in the below table are noise tolerances. The "true"
time/memory value for a given benchmark is expected to fall within this percentage of the reported value.
An empty cell means that the value was zero.

ID	time	GC time	memory	allocations
`["circuitsim", "mctrajectories", "q1001_r1"]`	15.339 ms (5%)		500.77 KiB (1%)	18019
`["circuitsim", "mctrajectories", "q101_r1"]`	171.411 μs (5%)		50.53 KiB (1%)	1818
`["circuitsim", "mctrajectories_sumtype", "q1001_r1"]`	13.385 ms (5%)		496 bytes (1%)	9
`["circuitsim", "mctrajectories_sumtype", "q101_r1"]`	118.872 μs (5%)		256 bytes (1%)	8
`["circuitsim", "mctrajectories_union", "q1001_r1"]`	13.479 ms (5%)		496 bytes (1%)	9
`["circuitsim", "mctrajectories_union", "q101_r1"]`	117.109 μs (5%)		256 bytes (1%)	8
`["circuitsim", "pftrajectories", "q1001_r1"]`	59.482 μs (5%)		93.80 KiB (1%)	2001
`["circuitsim", "pftrajectories", "q1001_r100"]`	169.527 μs (5%)		93.80 KiB (1%)	2001
`["circuitsim", "pftrajectories", "q1001_r10000"]`	1.094 ms (5%)		93.80 KiB (1%)	2001
`["circuitsim", "pftrajectories", "q101_r1"]`	6.001 μs (5%)		9.42 KiB (1%)	201
`["circuitsim", "pftrajectories_sumtype", "q1001_r1"]`	44.292 μs (5%)		93.84 KiB (1%)	2002
`["circuitsim", "pftrajectories_sumtype", "q1001_r100"]`	155.120 μs (5%)		93.84 KiB (1%)	2002
`["circuitsim", "pftrajectories_sumtype", "q1001_r10000"]`	1.086 ms (5%)		93.84 KiB (1%)	2002
`["circuitsim", "pftrajectories_sumtype", "q1001_r10000_fastrow"]`	5.862 ms (5%)		93.84 KiB (1%)	2002
`["circuitsim", "pftrajectories_sumtype", "q101_r1"]`	4.489 μs (5%)		9.47 KiB (1%)	202
`["circuitsim", "pftrajectories_union", "q1001_r1"]`	22.612 μs (5%)		96 bytes (1%)	2
`["circuitsim", "pftrajectories_union", "q1001_r100"]`	131.897 μs (5%)		96 bytes (1%)	2
`["circuitsim", "pftrajectories_union", "q1001_r10000"]`	1.051 ms (5%)		96 bytes (1%)	2
`["circuitsim", "pftrajectories_union", "q101_r1"]`	2.334 μs (5%)		96 bytes (1%)	2
`["clifford", "dense", "cnot250_on_dense500_destab"]`	11.626 ms (5%)		512 bytes (1%)	8
`["clifford", "dense", "cnot250_on_dense500_stab"]`	5.710 ms (5%)		512 bytes (1%)	8
`["clifford", "dense", "cnot250_on_diag500_destab"]`	1.122 ms (5%)		512 bytes (1%)	8
`["clifford", "dense", "cnot250_on_diag500_stab"]`	560.507 μs (5%)		512 bytes (1%)	8
`["clifford", "dense", "cnot_on_dense500_destab"]`	50.956 μs (5%)		368 bytes (1%)	10
`["clifford", "dense", "cnot_on_dense500_stab"]`	24.385 μs (5%)		368 bytes (1%)	10
`["clifford", "dense", "cnot_on_diag500_destab"]`	26.259 μs (5%)		368 bytes (1%)	10
`["clifford", "dense", "cnot_on_diag500_stab"]`	13.785 μs (5%)		368 bytes (1%)	10
`["clifford", "dense", "dense500_on_dense500_destab"]`	11.632 ms (5%)		512 bytes (1%)	8
`["clifford", "dense", "dense500_on_dense500_stab"]`	5.705 ms (5%)		512 bytes (1%)	8
`["clifford", "dense", "dense500_on_diag500_destab"]`	980.362 μs (5%)		512 bytes (1%)	8
`["clifford", "dense", "dense500_on_diag500_stab"]`	560.507 μs (5%)		512 bytes (1%)	8
`["clifford", "symbolic", "cnot250_on_dense500_destab"]`	1.441 ms (5%)
`["clifford", "symbolic", "cnot250_on_dense500_stab"]`	685.972 μs (5%)
`["clifford", "symbolic", "cnot250_on_diag500_destab"]`	1.161 ms (5%)
`["clifford", "symbolic", "cnot250_on_diag500_stab"]`	569.925 μs (5%)
`["clifford", "symbolic", "cnot_on_dense500_destab"]`	4.759 μs (5%)
`["clifford", "symbolic", "cnot_on_dense500_stab"]`	2.294 μs (5%)
`["clifford", "symbolic", "cnot_on_diag500_destab"]`	4.668 μs (5%)
`["clifford", "symbolic", "cnot_on_diag500_stab"]`	2.294 μs (5%)
`["ecc", "evaluate_decoder", "shor_bp_comm"]`	1.810 ms (5%)		1.77 MiB (1%)	42567
`["ecc", "evaluate_decoder", "shor_bp_naivesyn"]`	4.089 ms (5%)		3.61 MiB (1%)	81480
`["ecc", "evaluate_decoder", "shor_bp_shorsyn"]`	4.890 ms (5%)		3.80 MiB (1%)	84890
`["ecc", "evaluate_decoder", "shor_pybp_comm"]`	53.401 ms (5%)		4.76 MiB (1%)	144207
`["ecc", "evaluate_decoder", "shor_pybp_naivesyn"]`	159.039 ms (5%)		9.57 MiB (1%)	284018
`["ecc", "evaluate_decoder", "shor_pybp_shorsyn"]`	150.257 ms (5%)		9.66 MiB (1%)	285104
`["ecc", "evaluate_decoder", "shor_table_comm"]`	255.052 μs (5%)		241.42 KiB (1%)	4593
`["ecc", "evaluate_decoder", "shor_table_naivesyn"]`	865.618 μs (5%)		334.20 KiB (1%)	4790
`["ecc", "evaluate_decoder", "shor_table_shorsyn"]`	1.264 ms (5%)		427.73 KiB (1%)	5876
`["ecc", "evaluate_decoder", "toric8_bp_comm"]`	684.192 ms (5%)	13.009 ms	168.33 MiB (1%)	1009765
`["ecc", "evaluate_decoder", "toric8_bp_naivesyn"]`	1.387 s (5%)	23.748 ms	347.70 MiB (1%)	2106685
`["ecc", "evaluate_decoder", "toric8_bp_shorsyn"]`	1.406 s (5%)	32.426 ms	352.19 MiB (1%)	2150726
`["ecc", "evaluate_decoder", "toric8_pybp_comm"]`	479.316 ms (5%)		8.11 MiB (1%)	163696
`["ecc", "evaluate_decoder", "toric8_pybp_naivesyn"]`	1.000 s (5%)		18.04 MiB (1%)	360388
`["ecc", "evaluate_decoder", "toric8_pybp_shorsyn"]`	996.243 ms (5%)		20.56 MiB (1%)	392802
`["ecc", "evaluate_decoder", "toric8_pymatch_comm"]`	4.263 ms (5%)		1.95 MiB (1%)	23176
`["ecc", "evaluate_decoder", "toric8_pymatch_naivesyn"]`	12.491 ms (5%)		5.58 MiB (1%)	79355
`["ecc", "evaluate_decoder", "toric8_pymatch_shorsyn"]`	20.958 ms (5%)		8.09 MiB (1%)	111769
`["ecc", "evaluate_decoder", "toric8_table_comm"]`	2.672 ms (5%)		1.64 MiB (1%)	23082
`["ecc", "evaluate_decoder", "toric8_table_naivesyn"]`	10.177 ms (5%)		4.93 MiB (1%)	79160
`["ecc", "evaluate_decoder", "toric8_table_shorsyn"]`	18.742 ms (5%)		7.44 MiB (1%)	111574
`["pauli", "mul", "100"]`	14.597 ns (5%)
`["pauli", "mul", "1000"]`	17.899 ns (5%)
`["pauli", "mul", "100000"]`	731.848 ns (5%)
`["pauli", "mul", "20000000"]`	202.258 μs (5%)
`["stabilizer", "canon", "cano500"]`	3.083 ms (5%)
`["stabilizer", "canon", "diag_cano500"]`	676.415 μs (5%)
`["stabilizer", "canon", "diag_gott500"]`	4.155 ms (5%)		5.19 MiB (1%)	34521
`["stabilizer", "canon", "diag_rref500"]`	639.054 μs (5%)
`["stabilizer", "canon", "gott500"]`	6.540 ms (5%)		5.19 MiB (1%)	34531
`["stabilizer", "canon", "md_cano500"]`	1.583 ms (5%)
`["stabilizer", "canon", "md_rref500"]`	1.625 ms (5%)
`["stabilizer", "canon", "rref500"]`	3.137 ms (5%)
`["stabilizer", "project", "destabilizer"]`	18.534 μs (5%)		288 bytes (1%)	5
`["stabilizer", "project", "stabilizer"]`	9.027 μs (5%)		80 bytes (1%)	2
`["stabilizer", "tensor", "diag_pow5_20"]`	1.436 ms (5%)		23.97 MiB (1%)	34
`["stabilizer", "tensor", "pow5_20"]`	2.829 μs (5%)		6.44 KiB (1%)	31
`["stabilizer", "trace", "destabilizer"]`	34.014 μs (5%)		80 bytes (1%)	2
`["stabilizer", "trace", "stabilizer"]`	24.235 μs (5%)		112 bytes (1%)	3

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

["circuitsim", "mctrajectories"]
["circuitsim", "mctrajectories_sumtype"]
["circuitsim", "mctrajectories_union"]
["circuitsim", "pftrajectories"]
["circuitsim", "pftrajectories_sumtype"]
["circuitsim", "pftrajectories_union"]
["clifford", "dense"]
["clifford", "symbolic"]
["ecc", "evaluate_decoder"]
["pauli", "mul"]
["stabilizer", "canon"]
["stabilizer", "project"]
["stabilizer", "tensor"]
["stabilizer", "trace"]

Julia versioninfo

Julia Version 1.12.0-DEV.334
Commit 1ae41a2c0a3 (2024-04-12 03:14 UTC)
Build Info:
 Official https://julialang.org/ release
Platform Info:
 OS: Linux (x86_64-linux-gnu)
 Ubuntu 22.04.4 LTS
 uname: Linux 6.5.0-1017-azure #17~22.04.1-Ubuntu SMP Sat Mar 9 04:50:38 UTC 2024 x86_64 x86_64
 CPU: AMD EPYC 7763 64-Core Processor: 
 speed user nice sys idle irq
 #1 0 MHz 3520 s 0 s 382 s 9741 s 0 s
 #2 0 MHz 2872 s 0 s 360 s 10400 s 0 s
 #3 0 MHz 3337 s 0 s 390 s 9915 s 0 s
 #4 0 MHz 4441 s 0 s 374 s 8831 s 0 s
 Memory: 15.606498718261719 GB (13207.7734375 MB free)
 Uptime: 1371.81 sec
 Load Avg: 1.05 1.1 0.94
 WORD_SIZE: 64
 LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Runtime information

Runtime Info
BLAS #threads	2
`BLAS.vendor()`	`lbt`
`Sys.CPU_THREADS`	4

lscpu output:

Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Address sizes: 48 bits physical, 48 bits virtual
Byte Order: Little Endian
CPU(s): 4
On-line CPU(s) list: 0-3
Vendor ID: AuthenticAMD
Model name: AMD EPYC 7763 64-Core Processor
CPU family: 25
Model: 1
Thread(s) per core: 2
Core(s) per socket: 2
Socket(s): 1
Stepping: 1
BogoMIPS: 4890.85
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl tsc_reliable nonstop_tsc cpuid extd_apicid aperfmperf pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy svm cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw topoext invpcid_single vmmcall fsgsbase bmi1 avx2 smep bmi2 erms invpcid rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves clzero xsaveerptr rdpru arat npt nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold v_vmsave_vmload umip vaes vpclmulqdq rdpid fsrm
Virtualization: AMD-V
Hypervisor vendor: Microsoft
Virtualization type: full
L1d cache: 64 KiB (2 instances)
L1i cache: 64 KiB (2 instances)
L2 cache: 1 MiB (2 instances)
L3 cache: 32 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-3
Vulnerability Gather data sampling: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Vulnerable: Safe RET, no microcode
Vulnerability Spec store bypass: Vulnerable
Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2: Mitigation; Retpolines, STIBP disabled, RSB filling, PBRSB-eIBRS Not affected
Vulnerability Srbds: Not affected
Vulnerability Tsx async abort: Not affected

Cpu Property	Value
Brand	AMD EPYC 7763 64-Core Processor
Vendor	:AMD
Architecture	:Unknown
Model	Family: 0xaf, Model: 0x01, Stepping: 0x01, Type: 0x00
Cores	16 physical cores, 16 logical cores (on executing CPU)
	No Hyperthreading hardware capability detected
Clock Frequencies	Not supported by CPU
Data Cache	Level 1:3 : (32, 512, 32768) kbytes
	64 byte cache line size
Address Size	48 bits virtual, 48 bits physical
SIMD	256 bit = 32 byte max. SIMD vector size
Time Stamp Counter	TSC is accessible via `rdtsc`
	TSC runs at constant rate (invariant from clock frequency)
Perf. Monitoring	Performance Monitoring Counters (PMC) are not supported
Hypervisor	Yes, Microsoft

Benchmark Result

Judge result

Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl

Job Properties

Time of benchmarks:

Target: 12 Apr 2024 - 13:48
Baseline: 12 Apr 2024 - 13:55

Package commits:

Target: cb53f3
Baseline: 64f695

Julia commits:

Target: 1ae41a
Baseline: 1ae41a

Julia command flags:

Target: None
Baseline: None

Environment variables:

Target: None
Baseline: None

Results

A ratio greater than 1.0 denotes a possible regression (marked with ❌), while a ratio less
than 1.0 denotes a possible improvement (marked with ✅). Only significant results - results
that indicate possible regressions or improvements - are shown below (thus, an empty table means that all
benchmark results remained invariant between builds).

ID	time ratio	memory ratio
`["circuitsim", "mctrajectories_sumtype", "q1001_r1"]`	0.93 (5%) ✅	1.00 (1%)
`["ecc", "evaluate_decoder", "shor_bp_shorsyn"]`	0.90 (5%) ✅	0.98 (1%) ✅
`["ecc", "evaluate_decoder", "shor_pybp_naivesyn"]`	1.08 (5%) ❌	1.00 (1%)
`["ecc", "evaluate_decoder", "shor_pybp_shorsyn"]`	0.88 (5%) ✅	1.00 (1%)
`["ecc", "evaluate_decoder", "toric8_bp_comm"]`	0.99 (5%)	1.01 (1%) ❌
`["ecc", "evaluate_decoder", "toric8_bp_shorsyn"]`	0.97 (5%)	0.98 (1%) ✅
`["stabilizer", "canon", "diag_cano500"]`	1.11 (5%) ❌	1.00 (1%)
`["stabilizer", "canon", "diag_rref500"]`	0.87 (5%) ✅	1.00 (1%)
`["stabilizer", "tensor", "diag_pow5_20"]`	1.16 (5%) ❌	1.00 (1%)

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

["circuitsim", "mctrajectories"]
["circuitsim", "mctrajectories_sumtype"]
["circuitsim", "mctrajectories_union"]
["circuitsim", "pftrajectories"]
["circuitsim", "pftrajectories_sumtype"]
["circuitsim", "pftrajectories_union"]
["clifford", "dense"]
["clifford", "symbolic"]
["ecc", "evaluate_decoder"]
["pauli", "mul"]
["stabilizer", "canon"]
["stabilizer", "project"]
["stabilizer", "tensor"]
["stabilizer", "trace"]

Julia versioninfo

Target

Julia Version 1.12.0-DEV.334
Commit 1ae41a2c0a3 (2024-04-12 03:14 UTC)
Build Info:
 Official https://julialang.org/ release
Platform Info:
 OS: Linux (x86_64-linux-gnu)
 Ubuntu 22.04.4 LTS
 uname: Linux 6.5.0-1017-azure #17~22.04.1-Ubuntu SMP Sat Mar 9 04:50:38 UTC 2024 x86_64 x86_64
 CPU: AMD EPYC 7763 64-Core Processor: 
 speed user nice sys idle irq
 #1 0 MHz 2166 s 0 s 308 s 6606 s 0 s
 #2 0 MHz 2714 s 0 s 285 s 6107 s 0 s
 #3 0 MHz 2285 s 0 s 284 s 6498 s 0 s
 #4 0 MHz 2540 s 0 s 296 s 6242 s 0 s
 Memory: 15.606494903564453 GB (13380.4765625 MB free)
 Uptime: 915.57 sec
 Load Avg: 1.1 1.14 0.82
 WORD_SIZE: 64
 LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Baseline

Julia Version 1.12.0-DEV.334
Commit 1ae41a2c0a3 (2024-04-12 03:14 UTC)
Build Info:
 Official https://julialang.org/ release
Platform Info:
 OS: Linux (x86_64-linux-gnu)
 Ubuntu 22.04.4 LTS
 uname: Linux 6.5.0-1017-azure #17~22.04.1-Ubuntu SMP Sat Mar 9 04:50:38 UTC 2024 x86_64 x86_64
 CPU: AMD EPYC 7763 64-Core Processor: 
 speed user nice sys idle irq
 #1 0 MHz 2989 s 0 s 415 s 9989 s 0 s
 #2 0 MHz 3811 s 0 s 387 s 9224 s 0 s
 #3 0 MHz 3449 s 0 s 368 s 9567 s 0 s
 #4 0 MHz 3914 s 0 s 393 s 9088 s 0 s
 Memory: 15.606494903564453 GB (13298.3828125 MB free)
 Uptime: 1348.76 sec
 Load Avg: 1.0 1.06 0.93
 WORD_SIZE: 64
 LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Target result

Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl

Job Properties

Time of benchmark: 12 Apr 2024 - 13:48
Package commit: cb53f3
Julia commit: 1ae41a
Julia command flags: None
Environment variables: None

Results

Below is a table of this job's results, obtained by running the benchmarks.
The values listed in the ID column have the structure [parent_group, child_group, ..., key], and can be used to
index into the BaseBenchmarks suite to retrieve the corresponding benchmarks.
The percentages accompanying time and memory values in the below table are noise tolerances. The "true"
time/memory value for a given benchmark is expected to fall within this percentage of the reported value.
An empty cell means that the value was zero.

ID	time	GC time	memory	allocations
`["circuitsim", "mctrajectories", "q1001_r1"]`	14.997 ms (5%)		500.77 KiB (1%)	18019
`["circuitsim", "mctrajectories", "q101_r1"]`	170.285 μs (5%)		50.53 KiB (1%)	1818
`["circuitsim", "mctrajectories_sumtype", "q1001_r1"]`	13.349 ms (5%)		496 bytes (1%)	9
`["circuitsim", "mctrajectories_sumtype", "q101_r1"]`	117.488 μs (5%)		256 bytes (1%)	8
`["circuitsim", "mctrajectories_union", "q1001_r1"]`	13.203 ms (5%)		496 bytes (1%)	9
`["circuitsim", "mctrajectories_union", "q101_r1"]`	116.325 μs (5%)		256 bytes (1%)	8
`["circuitsim", "pftrajectories", "q1001_r1"]`	59.601 μs (5%)		93.80 KiB (1%)	2001
`["circuitsim", "pftrajectories", "q1001_r100"]`	169.855 μs (5%)		93.80 KiB (1%)	2001
`["circuitsim", "pftrajectories", "q1001_r10000"]`	1.100 ms (5%)		93.80 KiB (1%)	2001
`["circuitsim", "pftrajectories", "q101_r1"]`	6.001 μs (5%)		9.42 KiB (1%)	201
`["circuitsim", "pftrajectories_sumtype", "q1001_r1"]`	44.612 μs (5%)		93.84 KiB (1%)	2002
`["circuitsim", "pftrajectories_sumtype", "q1001_r100"]`	154.937 μs (5%)		93.84 KiB (1%)	2002
`["circuitsim", "pftrajectories_sumtype", "q1001_r10000"]`	1.085 ms (5%)		93.84 KiB (1%)	2002
`["circuitsim", "pftrajectories_sumtype", "q1001_r10000_fastrow"]`	5.871 ms (5%)		93.84 KiB (1%)	2002
`["circuitsim", "pftrajectories_sumtype", "q101_r1"]`	4.538 μs (5%)		9.47 KiB (1%)	202
`["circuitsim", "pftrajectories_union", "q1001_r1"]`	22.612 μs (5%)		96 bytes (1%)	2
`["circuitsim", "pftrajectories_union", "q1001_r100"]`	132.617 μs (5%)		96 bytes (1%)	2
`["circuitsim", "pftrajectories_union", "q1001_r10000"]`	1.054 ms (5%)		96 bytes (1%)	2
`["circuitsim", "pftrajectories_union", "q101_r1"]`	2.334 μs (5%)		96 bytes (1%)	2
`["clifford", "dense", "cnot250_on_dense500_destab"]`	11.815 ms (5%)		512 bytes (1%)	8
`["clifford", "dense", "cnot250_on_dense500_stab"]`	5.789 ms (5%)		512 bytes (1%)	8
`["clifford", "dense", "cnot250_on_diag500_destab"]`	1.122 ms (5%)		512 bytes (1%)	8
`["clifford", "dense", "cnot250_on_diag500_stab"]`	560.616 μs (5%)		512 bytes (1%)	8
`["clifford", "dense", "cnot_on_dense500_destab"]`	52.897 μs (5%)		368 bytes (1%)	10
`["clifford", "dense", "cnot_on_dense500_stab"]`	23.132 μs (5%)		368 bytes (1%)	10
`["clifford", "dense", "cnot_on_diag500_destab"]`	26.309 μs (5%)		368 bytes (1%)	10
`["clifford", "dense", "cnot_on_diag500_stab"]`	13.294 μs (5%)		368 bytes (1%)	10
`["clifford", "dense", "dense500_on_dense500_destab"]`	11.816 ms (5%)		512 bytes (1%)	8
`["clifford", "dense", "dense500_on_dense500_stab"]`	5.794 ms (5%)		512 bytes (1%)	8
`["clifford", "dense", "dense500_on_diag500_destab"]`	980.032 μs (5%)		512 bytes (1%)	8
`["clifford", "dense", "dense500_on_diag500_stab"]`	560.626 μs (5%)		512 bytes (1%)	8
`["clifford", "symbolic", "cnot250_on_dense500_destab"]`	1.434 ms (5%)
`["clifford", "symbolic", "cnot250_on_dense500_stab"]`	687.821 μs (5%)
`["clifford", "symbolic", "cnot250_on_diag500_destab"]`	1.162 ms (5%)
`["clifford", "symbolic", "cnot250_on_diag500_stab"]`	569.535 μs (5%)
`["clifford", "symbolic", "cnot_on_dense500_destab"]`	4.749 μs (5%)
`["clifford", "symbolic", "cnot_on_dense500_stab"]`	2.294 μs (5%)
`["clifford", "symbolic", "cnot_on_diag500_destab"]`	4.669 μs (5%)
`["clifford", "symbolic", "cnot_on_diag500_stab"]`	2.294 μs (5%)
`["ecc", "evaluate_decoder", "shor_bp_comm"]`	1.838 ms (5%)		1.76 MiB (1%)	42455
`["ecc", "evaluate_decoder", "shor_bp_naivesyn"]`	4.042 ms (5%)		3.62 MiB (1%)	81718
`["ecc", "evaluate_decoder", "shor_bp_shorsyn"]`	4.486 ms (5%)		3.71 MiB (1%)	82944
`["ecc", "evaluate_decoder", "shor_pybp_comm"]`	60.832 ms (5%)		4.76 MiB (1%)	144207
`["ecc", "evaluate_decoder", "shor_pybp_naivesyn"]`	146.900 ms (5%)		9.57 MiB (1%)	284018
`["ecc", "evaluate_decoder", "shor_pybp_shorsyn"]`	128.963 ms (5%)		9.66 MiB (1%)	285104
`["ecc", "evaluate_decoder", "shor_table_comm"]`	257.289 μs (5%)		241.42 KiB (1%)	4593
`["ecc", "evaluate_decoder", "shor_table_naivesyn"]`	869.387 μs (5%)		334.20 KiB (1%)	4790
`["ecc", "evaluate_decoder", "shor_table_shorsyn"]`	1.265 ms (5%)		427.73 KiB (1%)	5876
`["ecc", "evaluate_decoder", "toric8_bp_comm"]`	676.413 ms (5%)	10.161 ms	173.16 MiB (1%)	1038164
`["ecc", "evaluate_decoder", "toric8_bp_naivesyn"]`	1.421 s (5%)	24.163 ms	352.70 MiB (1%)	2136078
`["ecc", "evaluate_decoder", "toric8_bp_shorsyn"]`	1.377 s (5%)	28.011 ms	347.48 MiB (1%)	2123041
`["ecc", "evaluate_decoder", "toric8_pybp_comm"]`	474.298 ms (5%)		8.11 MiB (1%)	163696
`["ecc", "evaluate_decoder", "toric8_pybp_naivesyn"]`	1.001 s (5%)		18.04 MiB (1%)	360388
`["ecc", "evaluate_decoder", "toric8_pybp_shorsyn"]`	1.010 s (5%)		20.56 MiB (1%)	392802
`["ecc", "evaluate_decoder", "toric8_pymatch_comm"]`	4.160 ms (5%)		1.95 MiB (1%)	23176
`["ecc", "evaluate_decoder", "toric8_pymatch_naivesyn"]`	12.361 ms (5%)		5.58 MiB (1%)	79355
`["ecc", "evaluate_decoder", "toric8_pymatch_shorsyn"]`	20.775 ms (5%)		8.09 MiB (1%)	111769
`["ecc", "evaluate_decoder", "toric8_table_comm"]`	2.604 ms (5%)		1.64 MiB (1%)	23082
`["ecc", "evaluate_decoder", "toric8_table_naivesyn"]`	10.088 ms (5%)		4.93 MiB (1%)	79160
`["ecc", "evaluate_decoder", "toric8_table_shorsyn"]`	18.614 ms (5%)		7.44 MiB (1%)	111574
`["pauli", "mul", "100"]`	14.204 ns (5%)
`["pauli", "mul", "1000"]`	17.909 ns (5%)
`["pauli", "mul", "100000"]`	712.881 ns (5%)
`["pauli", "mul", "20000000"]`	196.774 μs (5%)
`["stabilizer", "canon", "cano500"]`	3.095 ms (5%)
`["stabilizer", "canon", "diag_cano500"]`	752.408 μs (5%)
`["stabilizer", "canon", "diag_gott500"]`	4.069 ms (5%)		5.19 MiB (1%)	34521
`["stabilizer", "canon", "diag_rref500"]`	626.931 μs (5%)
`["stabilizer", "canon", "gott500"]`	6.415 ms (5%)		5.19 MiB (1%)	34531
`["stabilizer", "canon", "md_cano500"]`	1.548 ms (5%)
`["stabilizer", "canon", "md_rref500"]`	1.575 ms (5%)
`["stabilizer", "canon", "rref500"]`	3.100 ms (5%)
`["stabilizer", "project", "destabilizer"]`	18.525 μs (5%)		288 bytes (1%)	5
`["stabilizer", "project", "stabilizer"]`	9.377 μs (5%)		80 bytes (1%)	2
`["stabilizer", "tensor", "diag_pow5_20"]`	1.355 ms (5%)		23.97 MiB (1%)	34
`["stabilizer", "tensor", "pow5_20"]`	2.899 μs (5%)		6.44 KiB (1%)	31
`["stabilizer", "trace", "destabilizer"]`	31.548 μs (5%)		80 bytes (1%)	2
`["stabilizer", "trace", "stabilizer"]`	24.094 μs (5%)		112 bytes (1%)	3

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

["circuitsim", "mctrajectories"]
["circuitsim", "mctrajectories_sumtype"]
["circuitsim", "mctrajectories_union"]
["circuitsim", "pftrajectories"]
["circuitsim", "pftrajectories_sumtype"]
["circuitsim", "pftrajectories_union"]
["clifford", "dense"]
["clifford", "symbolic"]
["ecc", "evaluate_decoder"]
["pauli", "mul"]
["stabilizer", "canon"]
["stabilizer", "project"]
["stabilizer", "tensor"]
["stabilizer", "trace"]

Julia versioninfo

Julia Version 1.12.0-DEV.334
Commit 1ae41a2c0a3 (2024-04-12 03:14 UTC)
Build Info:
 Official https://julialang.org/ release
Platform Info:
 OS: Linux (x86_64-linux-gnu)
 Ubuntu 22.04.4 LTS
 uname: Linux 6.5.0-1017-azure #17~22.04.1-Ubuntu SMP Sat Mar 9 04:50:38 UTC 2024 x86_64 x86_64
 CPU: AMD EPYC 7763 64-Core Processor: 
 speed user nice sys idle irq
 #1 0 MHz 2166 s 0 s 308 s 6606 s 0 s
 #2 0 MHz 2714 s 0 s 285 s 6107 s 0 s
 #3 0 MHz 2285 s 0 s 284 s 6498 s 0 s
 #4 0 MHz 2540 s 0 s 296 s 6242 s 0 s
 Memory: 15.606494903564453 GB (13380.4765625 MB free)
 Uptime: 915.57 sec
 Load Avg: 1.1 1.14 0.82
 WORD_SIZE: 64
 LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Baseline result

Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl

Job Properties

Time of benchmark: 12 Apr 2024 - 13:55
Package commit: 64f695
Julia commit: 1ae41a
Julia command flags: None
Environment variables: None

Results

Below is a table of this job's results, obtained by running the benchmarks.
The values listed in the ID column have the structure [parent_group, child_group, ..., key], and can be used to
index into the BaseBenchmarks suite to retrieve the corresponding benchmarks.
The percentages accompanying time and memory values in the below table are noise tolerances. The "true"
time/memory value for a given benchmark is expected to fall within this percentage of the reported value.
An empty cell means that the value was zero.

ID	time	GC time	memory	allocations
`["circuitsim", "mctrajectories", "q1001_r1"]`	15.324 ms (5%)		500.77 KiB (1%)	18019
`["circuitsim", "mctrajectories", "q101_r1"]`	171.759 μs (5%)		50.53 KiB (1%)	1818
`["circuitsim", "mctrajectories_sumtype", "q1001_r1"]`	14.359 ms (5%)		496 bytes (1%)	9
`["circuitsim", "mctrajectories_sumtype", "q101_r1"]`	119.102 μs (5%)		256 bytes (1%)	8
`["circuitsim", "mctrajectories_union", "q1001_r1"]`	13.265 ms (5%)		496 bytes (1%)	9
`["circuitsim", "mctrajectories_union", "q101_r1"]`	118.039 μs (5%)		256 bytes (1%)	8
`["circuitsim", "pftrajectories", "q1001_r1"]`	59.681 μs (5%)		93.80 KiB (1%)	2001
`["circuitsim", "pftrajectories", "q1001_r100"]`	170.397 μs (5%)		93.80 KiB (1%)	2001
`["circuitsim", "pftrajectories", "q1001_r10000"]`	1.096 ms (5%)		93.80 KiB (1%)	2001
`["circuitsim", "pftrajectories", "q101_r1"]`	5.991 μs (5%)		9.42 KiB (1%)	201
`["circuitsim", "pftrajectories_sumtype", "q1001_r1"]`	44.813 μs (5%)		93.84 KiB (1%)	2002
`["circuitsim", "pftrajectories_sumtype", "q1001_r100"]`	156.360 μs (5%)		93.84 KiB (1%)	2002
`["circuitsim", "pftrajectories_sumtype", "q1001_r10000"]`	1.091 ms (5%)		93.84 KiB (1%)	2002
`["circuitsim", "pftrajectories_sumtype", "q1001_r10000_fastrow"]`	5.822 ms (5%)		93.84 KiB (1%)	2002
`["circuitsim", "pftrajectories_sumtype", "q101_r1"]`	4.458 μs (5%)		9.47 KiB (1%)	202
`["circuitsim", "pftrajectories_union", "q1001_r1"]`	23.223 μs (5%)		96 bytes (1%)	2
`["circuitsim", "pftrajectories_union", "q1001_r100"]`	133.188 μs (5%)		96 bytes (1%)	2
`["circuitsim", "pftrajectories_union", "q1001_r10000"]`	1.056 ms (5%)		96 bytes (1%)	2
`["circuitsim", "pftrajectories_union", "q101_r1"]`	2.394 μs (5%)		96 bytes (1%)	2
`["clifford", "dense", "cnot250_on_dense500_destab"]`	11.747 ms (5%)		512 bytes (1%)	8
`["clifford", "dense", "cnot250_on_dense500_stab"]`	5.819 ms (5%)		512 bytes (1%)	8
`["clifford", "dense", "cnot250_on_diag500_destab"]`	1.122 ms (5%)		512 bytes (1%)	8
`["clifford", "dense", "cnot250_on_diag500_stab"]`	561.704 μs (5%)		512 bytes (1%)	8
`["clifford", "dense", "cnot_on_dense500_destab"]`	50.523 μs (5%)		368 bytes (1%)	10
`["clifford", "dense", "cnot_on_dense500_stab"]`	23.363 μs (5%)		368 bytes (1%)	10
`["clifford", "dense", "cnot_on_diag500_destab"]`	26.309 μs (5%)		368 bytes (1%)	10
`["clifford", "dense", "cnot_on_diag500_stab"]`	13.314 μs (5%)		368 bytes (1%)	10
`["clifford", "dense", "dense500_on_dense500_destab"]`	11.806 ms (5%)		512 bytes (1%)	8
`["clifford", "dense", "dense500_on_dense500_stab"]`	5.843 ms (5%)		512 bytes (1%)	8
`["clifford", "dense", "dense500_on_diag500_destab"]`	980.380 μs (5%)		512 bytes (1%)	8
`["clifford", "dense", "dense500_on_diag500_stab"]`	561.704 μs (5%)		512 bytes (1%)	8
`["clifford", "symbolic", "cnot250_on_dense500_destab"]`	1.434 ms (5%)
`["clifford", "symbolic", "cnot250_on_dense500_stab"]`	687.427 μs (5%)
`["clifford", "symbolic", "cnot250_on_diag500_destab"]`	1.161 ms (5%)
`["clifford", "symbolic", "cnot250_on_diag500_stab"]`	569.809 μs (5%)
`["clifford", "symbolic", "cnot_on_dense500_destab"]`	4.769 μs (5%)
`["clifford", "symbolic", "cnot_on_dense500_stab"]`	2.294 μs (5%)
`["clifford", "symbolic", "cnot_on_diag500_destab"]`	4.688 μs (5%)
`["clifford", "symbolic", "cnot_on_diag500_stab"]`	2.294 μs (5%)
`["ecc", "evaluate_decoder", "shor_bp_comm"]`	1.826 ms (5%)		1.77 MiB (1%)	42721
`["ecc", "evaluate_decoder", "shor_bp_naivesyn"]`	4.086 ms (5%)		3.61 MiB (1%)	81690
`["ecc", "evaluate_decoder", "shor_bp_shorsyn"]`	5.003 ms (5%)		3.79 MiB (1%)	84792
`["ecc", "evaluate_decoder", "shor_pybp_comm"]`	59.083 ms (5%)		4.76 MiB (1%)	144207
`["ecc", "evaluate_decoder", "shor_pybp_naivesyn"]`	136.484 ms (5%)		9.57 MiB (1%)	284018
`["ecc", "evaluate_decoder", "shor_pybp_shorsyn"]`	145.730 ms (5%)		9.66 MiB (1%)	285104
`["ecc", "evaluate_decoder", "shor_table_comm"]`	255.874 μs (5%)		241.42 KiB (1%)	4593
`["ecc", "evaluate_decoder", "shor_table_naivesyn"]`	887.918 μs (5%)		334.20 KiB (1%)	4790
`["ecc", "evaluate_decoder", "shor_table_shorsyn"]`	1.275 ms (5%)		427.73 KiB (1%)	5876
`["ecc", "evaluate_decoder", "toric8_bp_comm"]`	684.094 ms (5%)	15.175 ms	170.60 MiB (1%)	1023114
`["ecc", "evaluate_decoder", "toric8_bp_naivesyn"]`	1.390 s (5%)	19.896 ms	350.72 MiB (1%)	2124437
`["ecc", "evaluate_decoder", "toric8_bp_shorsyn"]`	1.427 s (5%)	24.819 ms	356.12 MiB (1%)	2173840
`["ecc", "evaluate_decoder", "toric8_pybp_comm"]`	488.487 ms (5%)		8.11 MiB (1%)	163696
`["ecc", "evaluate_decoder", "toric8_pybp_naivesyn"]`	1.006 s (5%)		18.04 MiB (1%)	360388
`["ecc", "evaluate_decoder", "toric8_pybp_shorsyn"]`	1.002 s (5%)		20.56 MiB (1%)	392802
`["ecc", "evaluate_decoder", "toric8_pymatch_comm"]`	4.279 ms (5%)		1.95 MiB (1%)	23176
`["ecc", "evaluate_decoder", "toric8_pymatch_naivesyn"]`	12.430 ms (5%)		5.58 MiB (1%)	79355
`["ecc", "evaluate_decoder", "toric8_pymatch_shorsyn"]`	20.840 ms (5%)		8.09 MiB (1%)	111769
`["ecc", "evaluate_decoder", "toric8_table_comm"]`	2.643 ms (5%)		1.64 MiB (1%)	23082
`["ecc", "evaluate_decoder", "toric8_table_naivesyn"]`	10.061 ms (5%)		4.93 MiB (1%)	79160
`["ecc", "evaluate_decoder", "toric8_table_shorsyn"]`	18.486 ms (5%)		7.44 MiB (1%)	111574
`["pauli", "mul", "100"]`	14.234 ns (5%)
`["pauli", "mul", "1000"]`	17.748 ns (5%)
`["pauli", "mul", "100000"]`	721.866 ns (5%)
`["pauli", "mul", "20000000"]`	203.167 μs (5%)
`["stabilizer", "canon", "cano500"]`	3.106 ms (5%)
`["stabilizer", "canon", "diag_cano500"]`	676.426 μs (5%)
`["stabilizer", "canon", "diag_gott500"]`	4.039 ms (5%)		5.19 MiB (1%)	34521
`["stabilizer", "canon", "diag_rref500"]`	721.900 μs (5%)
`["stabilizer", "canon", "gott500"]`	6.428 ms (5%)		5.19 MiB (1%)	34531
`["stabilizer", "canon", "md_cano500"]`	1.550 ms (5%)
`["stabilizer", "canon", "md_rref500"]`	1.564 ms (5%)
`["stabilizer", "canon", "rref500"]`	3.144 ms (5%)
`["stabilizer", "project", "destabilizer"]`	18.655 μs (5%)		288 bytes (1%)	5
`["stabilizer", "project", "stabilizer"]`	9.257 μs (5%)		80 bytes (1%)	2
`["stabilizer", "tensor", "diag_pow5_20"]`	1.171 ms (5%)		23.97 MiB (1%)	34
`["stabilizer", "tensor", "pow5_20"]`	2.817 μs (5%)		6.44 KiB (1%)	31
`["stabilizer", "trace", "destabilizer"]`	31.288 μs (5%)		80 bytes (1%)	2
`["stabilizer", "trace", "stabilizer"]`	24.216 μs (5%)		112 bytes (1%)	3

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

["circuitsim", "mctrajectories"]
["circuitsim", "mctrajectories_sumtype"]
["circuitsim", "mctrajectories_union"]
["circuitsim", "pftrajectories"]
["circuitsim", "pftrajectories_sumtype"]
["circuitsim", "pftrajectories_union"]
["clifford", "dense"]
["clifford", "symbolic"]
["ecc", "evaluate_decoder"]
["pauli", "mul"]
["stabilizer", "canon"]
["stabilizer", "project"]
["stabilizer", "tensor"]
["stabilizer", "trace"]

Julia versioninfo

Julia Version 1.12.0-DEV.334
Commit 1ae41a2c0a3 (2024-04-12 03:14 UTC)
Build Info:
 Official https://julialang.org/ release
Platform Info:
 OS: Linux (x86_64-linux-gnu)
 Ubuntu 22.04.4 LTS
 uname: Linux 6.5.0-1017-azure #17~22.04.1-Ubuntu SMP Sat Mar 9 04:50:38 UTC 2024 x86_64 x86_64
 CPU: AMD EPYC 7763 64-Core Processor: 
 speed user nice sys idle irq
 #1 0 MHz 2989 s 0 s 415 s 9989 s 0 s
 #2 0 MHz 3811 s 0 s 387 s 9224 s 0 s
 #3 0 MHz 3449 s 0 s 368 s 9567 s 0 s
 #4 0 MHz 3914 s 0 s 393 s 9088 s 0 s
 Memory: 15.606494903564453 GB (13298.3828125 MB free)
 Uptime: 1348.76 sec
 Load Avg: 1.0 1.06 0.93
 WORD_SIZE: 64
 LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Runtime information

Runtime Info
BLAS #threads	2
`BLAS.vendor()`	`lbt`
`Sys.CPU_THREADS`	4

lscpu output:

Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Address sizes: 48 bits physical, 48 bits virtual
Byte Order: Little Endian
CPU(s): 4
On-line CPU(s) list: 0-3
Vendor ID: AuthenticAMD
Model name: AMD EPYC 7763 64-Core Processor
CPU family: 25
Model: 1
Thread(s) per core: 2
Core(s) per socket: 2
Socket(s): 1
Stepping: 1
BogoMIPS: 4890.86
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl tsc_reliable nonstop_tsc cpuid extd_apicid aperfmperf pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy svm cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw topoext invpcid_single vmmcall fsgsbase bmi1 avx2 smep bmi2 erms invpcid rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves clzero xsaveerptr rdpru arat npt nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold v_vmsave_vmload umip vaes vpclmulqdq rdpid fsrm
Virtualization: AMD-V
Hypervisor vendor: Microsoft
Virtualization type: full
L1d cache: 64 KiB (2 instances)
L1i cache: 64 KiB (2 instances)
L2 cache: 1 MiB (2 instances)
L3 cache: 32 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-3
Vulnerability Gather data sampling: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Vulnerable: Safe RET, no microcode
Vulnerability Spec store bypass: Vulnerable
Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2: Mitigation; Retpolines, STIBP disabled, RSB filling, PBRSB-eIBRS Not affected
Vulnerability Srbds: Not affected
Vulnerability Tsx async abort: Not affected

Cpu Property	Value
Brand	AMD EPYC 7763 64-Core Processor
Vendor	:AMD
Architecture	:Unknown
Model	Family: 0xaf, Model: 0x01, Stepping: 0x01, Type: 0x00
Cores	16 physical cores, 16 logical cores (on executing CPU)
	No Hyperthreading hardware capability detected
Clock Frequencies	Not supported by CPU
Data Cache	Level 1:3 : (32, 512, 32768) kbytes
	64 byte cache line size
Address Size	48 bits virtual, 48 bits physical
SIMD	256 bit = 32 byte max. SIMD vector size
Time Stamp Counter	TSC is accessible via `rdtsc`
	TSC runs at constant rate (invariant from clock frequency)
Perf. Monitoring	Performance Monitoring Counters (PMC) are not supported
Hypervisor	Yes, Microsoft

Benchmark Result

Judge result

Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl

Job Properties

Time of benchmarks:

Target: 12 Apr 2024 - 13:56
Baseline: 12 Apr 2024 - 14:04

Package commits:

Target: 3d58e9
Baseline: 64f695

Julia commits:

Target: 1ae41a
Baseline: 1ae41a

Julia command flags:

Target: None
Baseline: None

Environment variables:

Target: None
Baseline: None

Results

A ratio greater than 1.0 denotes a possible regression (marked with ❌), while a ratio less
than 1.0 denotes a possible improvement (marked with ✅). Only significant results - results
that indicate possible regressions or improvements - are shown below (thus, an empty table means that all
benchmark results remained invariant between builds).

ID	time ratio	memory ratio
`["circuitsim", "mctrajectories_sumtype", "q1001_r1"]`	0.90 (5%) ✅	1.00 (1%)
`["circuitsim", "mctrajectories_union", "q1001_r1"]`	0.92 (5%) ✅	1.00 (1%)
`["circuitsim", "pftrajectories_union", "q101_r1"]`	0.92 (5%) ✅	1.00 (1%)
`["clifford", "dense", "cnot250_on_dense500_destab"]`	1.16 (5%) ❌	1.00 (1%)
`["clifford", "dense", "cnot250_on_diag500_stab"]`	0.88 (5%) ✅	1.00 (1%)
`["clifford", "dense", "dense500_on_diag500_stab"]`	0.87 (5%) ✅	1.00 (1%)
`["clifford", "symbolic", "cnot_on_dense500_stab"]`	0.93 (5%) ✅	1.00 (1%)
`["ecc", "evaluate_decoder", "shor_bp_shorsyn"]`	0.90 (5%) ✅	0.98 (1%) ✅
`["ecc", "evaluate_decoder", "shor_pybp_comm"]`	1.12 (5%) ❌	1.00 (1%)
`["ecc", "evaluate_decoder", "shor_pybp_naivesyn"]`	0.86 (5%) ✅	1.00 (1%)
`["ecc", "evaluate_decoder", "toric8_bp_comm"]`	0.94 (5%) ✅	0.96 (1%) ✅
`["ecc", "evaluate_decoder", "toric8_bp_naivesyn"]`	1.00 (5%)	1.01 (1%) ❌
`["pauli", "mul", "100000"]`	0.92 (5%) ✅	1.00 (1%)
`["stabilizer", "canon", "diag_cano500"]`	1.06 (5%) ❌	1.00 (1%)
`["stabilizer", "canon", "diag_rref500"]`	1.15 (5%) ❌	1.00 (1%)
`["stabilizer", "canon", "md_cano500"]`	1.06 (5%) ❌	1.00 (1%)
`["stabilizer", "project", "destabilizer"]`	0.93 (5%) ✅	1.00 (1%)
`["stabilizer", "project", "stabilizer"]`	1.08 (5%) ❌	1.00 (1%)
`["stabilizer", "trace", "destabilizer"]`	1.08 (5%) ❌	1.00 (1%)

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

["circuitsim", "mctrajectories"]
["circuitsim", "mctrajectories_sumtype"]
["circuitsim", "mctrajectories_union"]
["circuitsim", "pftrajectories"]
["circuitsim", "pftrajectories_sumtype"]
["circuitsim", "pftrajectories_union"]
["clifford", "dense"]
["clifford", "symbolic"]
["ecc", "evaluate_decoder"]
["pauli", "mul"]
["stabilizer", "canon"]
["stabilizer", "project"]
["stabilizer", "tensor"]
["stabilizer", "trace"]

Julia versioninfo

Target

Julia Version 1.12.0-DEV.334
Commit 1ae41a2c0a3 (2024-04-12 03:14 UTC)
Build Info:
 Official https://julialang.org/ release
Platform Info:
 OS: Linux (x86_64-linux-gnu)
 Ubuntu 22.04.4 LTS
 uname: Linux 6.5.0-1017-azure #17~22.04.1-Ubuntu SMP Sat Mar 9 04:50:38 UTC 2024 x86_64 x86_64
 CPU: AMD EPYC 7763 64-Core Processor: 
 speed user nice sys idle irq
 #1 0 MHz 2543 s 0 s 272 s 5972 s 0 s
 #2 0 MHz 2933 s 0 s 284 s 5595 s 0 s
 #3 0 MHz 2016 s 0 s 298 s 6477 s 0 s
 #4 0 MHz 1869 s 0 s 293 s 6632 s 0 s
 Memory: 15.606498718261719 GB (13355.90625 MB free)
 Uptime: 885.05 sec
 Load Avg: 1.08 1.09 0.76
 WORD_SIZE: 64
 LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Baseline

Julia Version 1.12.0-DEV.334
Commit 1ae41a2c0a3 (2024-04-12 03:14 UTC)
Build Info:
 Official https://julialang.org/ release
Platform Info:
 OS: Linux (x86_64-linux-gnu)
 Ubuntu 22.04.4 LTS
 uname: Linux 6.5.0-1017-azure #17~22.04.1-Ubuntu SMP Sat Mar 9 04:50:38 UTC 2024 x86_64 x86_64
 CPU: AMD EPYC 7763 64-Core Processor: 
 speed user nice sys idle irq
 #1 0 MHz 3595 s 0 s 356 s 9082 s 0 s
 #2 0 MHz 3962 s 0 s 394 s 8704 s 0 s
 #3 0 MHz 3468 s 0 s 391 s 9179 s 0 s
 #4 0 MHz 2731 s 0 s 387 s 9923 s 0 s
 Memory: 15.606498718261719 GB (13284.78515625 MB free)
 Uptime: 1311.18 sec
 Load Avg: 1.09 1.13 0.93
 WORD_SIZE: 64
 LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Target result

Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl

Job Properties

Time of benchmark: 12 Apr 2024 - 13:56
Package commit: 3d58e9
Julia commit: 1ae41a
Julia command flags: None
Environment variables: None

Results

Below is a table of this job's results, obtained by running the benchmarks.
The values listed in the ID column have the structure [parent_group, child_group, ..., key], and can be used to
index into the BaseBenchmarks suite to retrieve the corresponding benchmarks.
The percentages accompanying time and memory values in the below table are noise tolerances. The "true"
time/memory value for a given benchmark is expected to fall within this percentage of the reported value.
An empty cell means that the value was zero.

ID	time	GC time	memory	allocations
`["circuitsim", "mctrajectories", "q1001_r1"]`	14.300 ms (5%)		500.77 KiB (1%)	18019
`["circuitsim", "mctrajectories", "q101_r1"]`	159.397 μs (5%)		50.53 KiB (1%)	1818
`["circuitsim", "mctrajectories_sumtype", "q1001_r1"]`	12.642 ms (5%)		496 bytes (1%)	9
`["circuitsim", "mctrajectories_sumtype", "q101_r1"]`	109.774 μs (5%)		256 bytes (1%)	8
`["circuitsim", "mctrajectories_union", "q1001_r1"]`	12.614 ms (5%)		496 bytes (1%)	9
`["circuitsim", "mctrajectories_union", "q101_r1"]`	109.143 μs (5%)		256 bytes (1%)	8
`["circuitsim", "pftrajectories", "q1001_r1"]`	58.399 μs (5%)		93.80 KiB (1%)	2001
`["circuitsim", "pftrajectories", "q1001_r100"]`	161.982 μs (5%)		93.80 KiB (1%)	2001
`["circuitsim", "pftrajectories", "q1001_r10000"]`	1.017 ms (5%)		93.80 KiB (1%)	2001
`["circuitsim", "pftrajectories", "q101_r1"]`	5.861 μs (5%)		9.42 KiB (1%)	201
`["circuitsim", "pftrajectories_sumtype", "q1001_r1"]`	40.645 μs (5%)		93.84 KiB (1%)	2002
`["circuitsim", "pftrajectories_sumtype", "q1001_r100"]`	145.422 μs (5%)		93.84 KiB (1%)	2002
`["circuitsim", "pftrajectories_sumtype", "q1001_r10000"]`	1.003 ms (5%)		93.84 KiB (1%)	2002
`["circuitsim", "pftrajectories_sumtype", "q1001_r10000_fastrow"]`	5.391 ms (5%)		93.84 KiB (1%)	2002
`["circuitsim", "pftrajectories_sumtype", "q101_r1"]`	4.117 μs (5%)		9.47 KiB (1%)	202
`["circuitsim", "pftrajectories_union", "q1001_r1"]`	20.858 μs (5%)		96 bytes (1%)	2
`["circuitsim", "pftrajectories_union", "q1001_r100"]`	123.748 μs (5%)		96 bytes (1%)	2
`["circuitsim", "pftrajectories_union", "q1001_r10000"]`	973.621 μs (5%)		96 bytes (1%)	2
`["circuitsim", "pftrajectories_union", "q101_r1"]`	2.164 μs (5%)		96 bytes (1%)	2
`["clifford", "dense", "cnot250_on_dense500_destab"]`	12.768 ms (5%)		512 bytes (1%)	8
`["clifford", "dense", "cnot250_on_dense500_stab"]`	5.441 ms (5%)		512 bytes (1%)	8
`["clifford", "dense", "cnot250_on_diag500_destab"]`	1.038 ms (5%)		512 bytes (1%)	8
`["clifford", "dense", "cnot250_on_diag500_stab"]`	455.187 μs (5%)		512 bytes (1%)	8
`["clifford", "dense", "cnot_on_dense500_destab"]`	46.927 μs (5%)		368 bytes (1%)	10
`["clifford", "dense", "cnot_on_dense500_stab"]`	21.681 μs (5%)		368 bytes (1%)	10
`["clifford", "dense", "cnot_on_diag500_destab"]`	24.225 μs (5%)		368 bytes (1%)	10
`["clifford", "dense", "cnot_on_diag500_stab"]`	12.133 μs (5%)		368 bytes (1%)	10
`["clifford", "dense", "dense500_on_dense500_destab"]`	10.692 ms (5%)		512 bytes (1%)	8
`["clifford", "dense", "dense500_on_dense500_stab"]`	5.310 ms (5%)		512 bytes (1%)	8
`["clifford", "dense", "dense500_on_diag500_destab"]`	903.782 μs (5%)		512 bytes (1%)	8
`["clifford", "dense", "dense500_on_diag500_stab"]`	451.490 μs (5%)		512 bytes (1%)	8
`["clifford", "symbolic", "cnot250_on_dense500_destab"]`	1.323 ms (5%)
`["clifford", "symbolic", "cnot250_on_dense500_stab"]`	634.651 μs (5%)
`["clifford", "symbolic", "cnot250_on_diag500_destab"]`	1.076 ms (5%)
`["clifford", "symbolic", "cnot250_on_diag500_stab"]`	525.618 μs (5%)
`["clifford", "symbolic", "cnot_on_dense500_destab"]`	4.388 μs (5%)
`["clifford", "symbolic", "cnot_on_dense500_stab"]`	2.123 μs (5%)
`["clifford", "symbolic", "cnot_on_diag500_destab"]`	4.328 μs (5%)
`["clifford", "symbolic", "cnot_on_diag500_stab"]`	2.113 μs (5%)
`["ecc", "evaluate_decoder", "shor_bp_comm"]`	1.705 ms (5%)		1.77 MiB (1%)	42553
`["ecc", "evaluate_decoder", "shor_bp_naivesyn"]`	3.787 ms (5%)		3.61 MiB (1%)	81634
`["ecc", "evaluate_decoder", "shor_bp_shorsyn"]`	4.164 ms (5%)		3.70 MiB (1%)	82636
`["ecc", "evaluate_decoder", "shor_pybp_comm"]`	64.280 ms (5%)		4.76 MiB (1%)	144207
`["ecc", "evaluate_decoder", "shor_pybp_naivesyn"]`	133.041 ms (5%)		9.57 MiB (1%)	284018
`["ecc", "evaluate_decoder", "shor_pybp_shorsyn"]`	129.201 ms (5%)		9.66 MiB (1%)	285104
`["ecc", "evaluate_decoder", "shor_table_comm"]`	241.610 μs (5%)		241.42 KiB (1%)	4593
`["ecc", "evaluate_decoder", "shor_table_naivesyn"]`	798.937 μs (5%)		334.20 KiB (1%)	4790
`["ecc", "evaluate_decoder", "shor_table_shorsyn"]`	1.172 ms (5%)		427.73 KiB (1%)	5876
`["ecc", "evaluate_decoder", "toric8_bp_comm"]`	640.512 ms (5%)	10.718 ms	166.49 MiB (1%)	998936
`["ecc", "evaluate_decoder", "toric8_bp_naivesyn"]`	1.387 s (5%)	24.453 ms	351.19 MiB (1%)	2127202
`["ecc", "evaluate_decoder", "toric8_bp_shorsyn"]`	1.386 s (5%)	25.639 ms	354.44 MiB (1%)	2163963
`["ecc", "evaluate_decoder", "toric8_pybp_comm"]`	457.341 ms (5%)		8.11 MiB (1%)	163696
`["ecc", "evaluate_decoder", "toric8_pybp_naivesyn"]`	927.357 ms (5%)		18.04 MiB (1%)	360388
`["ecc", "evaluate_decoder", "toric8_pybp_shorsyn"]`	955.391 ms (5%)		20.56 MiB (1%)	392802
`["ecc", "evaluate_decoder", "toric8_pymatch_comm"]`	3.883 ms (5%)		1.95 MiB (1%)	23176
`["ecc", "evaluate_decoder", "toric8_pymatch_naivesyn"]`	11.745 ms (5%)		5.58 MiB (1%)	79355
`["ecc", "evaluate_decoder", "toric8_pymatch_shorsyn"]`	19.647 ms (5%)		8.09 MiB (1%)	111769
`["ecc", "evaluate_decoder", "toric8_table_comm"]`	2.454 ms (5%)		1.64 MiB (1%)	23082
`["ecc", "evaluate_decoder", "toric8_table_naivesyn"]`	9.399 ms (5%)		4.93 MiB (1%)	79160
`["ecc", "evaluate_decoder", "toric8_table_shorsyn"]`	17.538 ms (5%)		7.44 MiB (1%)	111574
`["pauli", "mul", "100"]`	13.562 ns (5%)
`["pauli", "mul", "1000"]`	17.344 ns (5%)
`["pauli", "mul", "100000"]`	681.267 ns (5%)
`["pauli", "mul", "20000000"]`	163.234 μs (5%)
`["stabilizer", "canon", "cano500"]`	2.887 ms (5%)
`["stabilizer", "canon", "diag_cano500"]`	694.102 μs (5%)
`["stabilizer", "canon", "diag_gott500"]`	3.778 ms (5%)		5.19 MiB (1%)	34521
`["stabilizer", "canon", "diag_rref500"]`	665.559 μs (5%)
`["stabilizer", "canon", "gott500"]`	6.071 ms (5%)		5.19 MiB (1%)	34531
`["stabilizer", "canon", "md_cano500"]`	1.504 ms (5%)
`["stabilizer", "canon", "md_rref500"]`	1.493 ms (5%)
`["stabilizer", "canon", "rref500"]`	2.939 ms (5%)
`["stabilizer", "project", "destabilizer"]`	16.992 μs (5%)		288 bytes (1%)	5
`["stabilizer", "project", "stabilizer"]`	8.897 μs (5%)		80 bytes (1%)	2
`["stabilizer", "tensor", "diag_pow5_20"]`	1.080 ms (5%)		23.97 MiB (1%)	34
`["stabilizer", "tensor", "pow5_20"]`	2.679 μs (5%)		6.44 KiB (1%)	31
`["stabilizer", "trace", "destabilizer"]`	30.887 μs (5%)		80 bytes (1%)	2
`["stabilizer", "trace", "stabilizer"]`	22.552 μs (5%)		112 bytes (1%)	3

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

["circuitsim", "mctrajectories"]
["circuitsim", "mctrajectories_sumtype"]
["circuitsim", "mctrajectories_union"]
["circuitsim", "pftrajectories"]
["circuitsim", "pftrajectories_sumtype"]
["circuitsim", "pftrajectories_union"]
["clifford", "dense"]
["clifford", "symbolic"]
["ecc", "evaluate_decoder"]
["pauli", "mul"]
["stabilizer", "canon"]
["stabilizer", "project"]
["stabilizer", "tensor"]
["stabilizer", "trace"]

Julia versioninfo

Julia Version 1.12.0-DEV.334
Commit 1ae41a2c0a3 (2024-04-12 03:14 UTC)
Build Info:
 Official https://julialang.org/ release
Platform Info:
 OS: Linux (x86_64-linux-gnu)
 Ubuntu 22.04.4 LTS
 uname: Linux 6.5.0-1017-azure #17~22.04.1-Ubuntu SMP Sat Mar 9 04:50:38 UTC 2024 x86_64 x86_64
 CPU: AMD EPYC 7763 64-Core Processor: 
 speed user nice sys idle irq
 #1 0 MHz 2543 s 0 s 272 s 5972 s 0 s
 #2 0 MHz 2933 s 0 s 284 s 5595 s 0 s
 #3 0 MHz 2016 s 0 s 298 s 6477 s 0 s
 #4 0 MHz 1869 s 0 s 293 s 6632 s 0 s
 Memory: 15.606498718261719 GB (13355.90625 MB free)
 Uptime: 885.05 sec
 Load Avg: 1.08 1.09 0.76
 WORD_SIZE: 64
 LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtu...*[Comment body truncated]*

Krastanov · 2024-04-12T13:52:20Z

@hongyehu , there are some failures related to the expectation value (the tests in test_nonclifford_quantumoptics.jl). I will not have the time to investigate today, but it seems I have messed up a convention somewhere and we are getting expectation values that are occasionally imaginary. Hopefully I will get to it early next week, but let me know if you would like to discuss it together if you have time to investigate it.

hongyehu · 2024-04-12T17:18:20Z

@Krastanov Hi Stefan, I can take a look into the expect! function and see whether we missed anything!

Krastanov · 2024-04-18T18:45:25Z

here is a small reproducers:

using QuantumClifford
using QuantumOpticsBase # which provides Operator and Ket for conversion (kinda like qutip)
using LinearAlgebra

qo_basis = SpinBasis(1//2)
qo_tgate = sparse(identityoperator(qo_basis))
qo_tgate.data[2,2] = exp(im*pi/4)

for s in [S"X", S"Y", S"Z", S"-X", S"-Y", S"-Z"]
    for p in [P"X", P"Y", P"Z", P"-X", P"-Y", P"-Z"]
        gs = GeneralizedStabilizer(s)
        apply!(gs, pcT)
        ρ = dm(qo_tgate*Ket(s))
        @test Operator(gs) ≈ ρ
        if isapprox(expect(p, gs), expect(Operator(p),ρ); atol=1e-5)
        else
            println("failure of expectation calculation for state = T*(", s, ") and observable = ", p)
        end
    end
end

It prints the following failing cases:

failure of expect for state = T*(+ X) and observable = + Y
failure of expect for state = T*(+ X) and observable = - Y
failure of expect for state = T*(+ Y) and observable = + X
failure of expect for state = T*(+ Y) and observable = - X
failure of expect for state = T*(- X) and observable = + Y
failure of expect for state = T*(- X) and observable = - Y
failure of expect for state = T*(- Y) and observable = + X
failure of expect for state = T*(- Y) and observable = - X

Krastanov mentioned this pull request Apr 12, 2024

improvements to GeneralizedStabilizer API #260

Open

9 tasks

Krastanov force-pushed the nonclif branch from c8de2f2 to 052a503 Compare April 12, 2024 13:44

Krastanov added 6 commits April 18, 2024 14:10

Destabilizer now preserves input Stabilizer tableaux

0680562

StabMixture renamed to GeneralizedStabilizer

e395e26

rowdecompose was not accounting for input phase

feb7363

expect for GeneralizedStabilizer

1ad4b36

scaffolding for project! for GeneralizedStabilizer

21d7983

bump project version and test fixups

bcf816c

add smaller tests for expect on nonstabilizer states

9ecb7bf

Krastanov force-pushed the nonclif branch from 052a503 to 9ecb7bf Compare April 18, 2024 18:46

improvements to the non-clifford functionality #259

Are you sure you want to change the base?

improvements to the non-clifford functionality #259

Conversation

Krastanov commented Apr 12, 2024

Krastanov commented Apr 12, 2024

github-actions bot commented Apr 12, 2024 • edited

Judge result

Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl

Job Properties

Results

Benchmark Group List

Julia versioninfo

Target

Baseline

Target result

Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl

Job Properties

Results

Benchmark Group List

Julia versioninfo

Baseline result

Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl

Job Properties

Results

Benchmark Group List

Julia versioninfo

Runtime information

Judge result

Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl

Job Properties

Results

Benchmark Group List

Julia versioninfo

Target

Baseline

Target result

Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl

Job Properties

Results

Benchmark Group List

Julia versioninfo

Baseline result

Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl

Job Properties

Results

Benchmark Group List

Julia versioninfo

Runtime information

Judge result

Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl

Job Properties

Results

Benchmark Group List

Julia versioninfo

Target

Baseline

Target result

Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl

Job Properties

Results

Benchmark Group List

Julia versioninfo

Krastanov commented Apr 12, 2024

hongyehu commented Apr 12, 2024

Krastanov commented Apr 18, 2024

github-actions bot commented Apr 12, 2024 •

edited