-
Notifications
You must be signed in to change notification settings - Fork 379
/
phoronix-test-suite.html
1658 lines (1633 loc) · 179 KB
/
phoronix-test-suite.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" "http://www.w3.org/TR/REC-html40/loose.dtd">
<html>
<!--
Phoronix Test Suite
URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
Copyright (C) 2008 - 2021, Phoronix Media
Copyright (C) 2008 - 2021, Michael Larabel
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
-->
<head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<title>Phoronix Test Suite</title>
<style>
body
{
margin: 0;
padding: 0;
font: 14px ;
background: #fff;
color: #000;
letter-spacing: 1px;
}
ol, ul
{
list-style-type: none;
}
li
{
font-size: 20px;
font-weight: bold;
color: #337ca6;
}
a
{
color: inherit;
text-decoration: none;
}
a:hover
{
text-decoration: underline;
}
div#pts_container
{
text-align: left;
background: #FFF;
width: auto;
margin-top:0px !important;
margin-bottom:0px !important;
margin: 60px;
overflow: hidden;
}
a#pts_top_logo
{
display: block;
opacity: 1.0;
filter: alpha(opacity=100);
}
a#pts_top_logo svg
{
width: 160px;
height: auto;
}
a#pts_top_logo:hover
{
opacity: 0.48;
filter: alpha(opacity=48);
}
h1,h2,h3,h4,h5,h6
{
margin-left: 10px;
margin-top: 10px;
font-size: 21px;
color: #098BEF ;
font-weight: bold !important;
}
h3
{
color: #757575;
font-size: 16px;
}
p
{
margin-left: 10px;
}
div#pts_benchmark_area
{
background-color: #FFF;
margin: 30px auto;
color: #000;
overflow: hidden;
}
div.pts_benchmark_bar
{
overflow: hidden;
padding: 4px;
margin: 0 10px;
clear: both;
}
div.pts_benchmark_img_area
{
margin: 4px auto;
padding: 20px 10px;
overflow: hidden;
text-align: center;
}
div#pts_banner_nav
{
height: 28px;
background: #000 repeat-x;
border: solid #949494 1px;
text-align: center;
text-shadow: #000 0.1em 0.1em 0.2em;
color: #FFF;
font-weight: bold;
font-size: 14px;
padding-right: 1px;
margin: 4px 10px;
}
div#pts_banner_nav a
{
width: 25%;
height: 100%;
display: block;
float: left;
line-height: 28px;
margin-right: -1px;
border: solid #333;
border-width: 0 1px 0 0;
}
div#pts_banner_nav a:hover
{
text-decoration: none;
background: #c8d905 repeat-x;
}
div#pts_footer
{
clear: both;
top: 20px;
width: 100%;
border: 0 solid #D4D4D4;
border-width: 1px 0 0;
padding: 15px 0 10px;
color: #757575;
font-size: 10px;
}
div#pts_footer a
{
text-decoration: none;
}
div#pts_footer a:hover, div#oborg_main_area blockquote a
{
text-decoration: underline;
}
div#pts_footer p
{
margin: 1px;
text-indent: 20px;
}
div#pts_footer_logo
{
float: right;
width: 96px;
height: 50px;
margin: 0 5px 10px;
}
div#pts_footer_logo svg
{
width: 100px;
height: auto;
}
div#pts_footer_logo a
{
display: block;
opacity: 0.48;
filter: alpha(opacity=48);
}
div#pts_footer_logo a:hover
{
opacity: 1.0;
filter: alpha(opacity=100);
}
</style>
</head>
<body>
<div id="pts_container">
<div style="overflow: hidden;">
<div style="text-align: right; float: right; clear: right; margin: 12px 10px 6px 8px;"><a href="http://www.phoronix-test-suite.com/" id="pts_top_logo"><svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" version="1.1" viewbox="0 0 76 41" width="76" height="41" preserveaspectratio="xMinYMin meet">
<path d="m74 22v9m-5-16v16m-5-28v28m-23-2h12.5c2.485281 0 4.5-2.014719 4.5-4.5s-2.014719-4.5-4.5-4.5h-8c-2.485281 0-4.5-2.014719-4.5-4.5s2.014719-4.5 4.5-4.5h12.5m-21 5h-11m11 13h-2c-4.970563 0-9-4.029437-9-9v-20m-24 40v-20c0-4.970563 4.0294373-9 9-9 4.970563 0 9 4.029437 9 9s-4.029437 9-9 9h-9" stroke="#222" stroke-width="4" fill="none"></path>
</svg></a></div>
<h1>Phoronix Test Suite</h1>
</div>
<div id="pts_banner_nav"></div>
<div id="pts_template_body"><ol><li><a href="#GettingStarted">Getting Started</a></li><li><a href="#UserOptions">User Options</a></li><li><a href="#ModuleOptions">Module Options</a></li><li><a href="#InstallationInstructions">Installation Instructions</a></li><li><a href="#PhoronixTestSuiteOnWindows">Phoronix Test Suite On Windows</a></li><li><a href="#ExternalDependencies">External Dependencies</a></li><li><a href="#Configuration">Configuration</a></li><li><a href="#MainConfigurationFile">Main Configuration File</a></li><li><a href="#EnvironmentVariables">Environment Variables</a></li><li><a href="#GeneralInformation">General Information</a></li><li><a href="#VirtualTestSuites">Virtual Test Suites</a></li><li><a href="#ComponentTesting">Component Testing</a></li><li><a href="#PhoronixTestSuitePhoromatic">Phoronix Test Suite Phoromatic</a></li><li><a href="#OfflineTesting">Offline Testing</a></li><li><a href="#ConfidentialTesting/AvoidingAccidentalResultUploads">Confidential Testing / Avoiding Accidental Result Uploads</a></li><li><a href="#DevelopmentCredits">Development Credits</a></li></ol><hr style="height: 50px; border: 0;"><a name="GettingStarted"></a><h1>Getting Started</h1>
<h1>Overview</h1>
<p>The Phoronix Test Suite is the most comprehensive testing and benchmarking platform available for Linux, Solaris, macOS, Windows, and BSD operating systems. The Phoronix Test Suite allows for carrying out tests in a fully automated manner from test installation to execution and reporting. All tests are meant to be easily reproducible, easy-to-use, and support fully automated execution. The Phoronix Test Suite is open-source under the GNU GPLv3 license and is developed by Phoronix Media in cooperation with partners. Version 1.0 of the Phoronix Test Suite was publicly released in 2008.</p>
<p>The Phoronix Test Suite client itself is an automated test framework for providing seamless execution of test profiles and test suites. There are more than 650 tests available by default, which are transparently available via <a href="https://openbenchmarking.org/">OpenBenchmarking.org</a> integration. Of these default test profiles there is a range of sub-systems that can be tested and a range of hardware from mobile devices to desktops and worksrtations/servers. New tests can be easily introduced via the Phoronix Test Suite's extensible test architecture, with test profiles consisting of XML files and shell scripts. Test profiles can produce a quantitative result or other qualitative/abstract results like image quality comparisons and pass/fail. Using Phoronix Test Suite modules, other data can also be automatically collected at run-time such as the system power consumption, disk usage, and other software/hardware sensors. Test suites contain references to test profiles to execute as part of a set or can also reference other test suites. Test suites are defined via an XML schema.</p>
<p>Running the Phoronix Test Suite for the first time can be as simple as issuing a command such as <em>phoronix-test-suite benchmark c-ray</em>, which would proceed to install a simple CPU test, execute the test, and report the results. Along with the results, the system's hardware/software information is collected in a detailed manner, relevant system logs, and other important system attributes such as compiler flags and system state. Users can optionally upload their results to OpenBenchmarking.org for sharing results with others, comparing results against other systems, and to carry out further analysis.</p>
<h1>OpenBenchmarking.org</h1>
<p>OpenBenchmarking.org is an open, collaborative testing platform that makes the Phoronix Test Suite an even more extensible platform for conducting automated tests with complete integration into Phoronix Test Suite test client. OpenBenchmarking.org serves as a repository for storing test profiles, test suites, and result data. Test profiles and suites are stored in the OpenBenchmarking.org cloud to allow for new/updated tests to be seamlessly obtained via the Phoronix Test Suite without needing to manually update the Phoronix Test Suite client. OpenBenchmarking.org also makes it easy to facilitate side-by-side comparisons with any other results stored in the OpenBenchmarking.org cloud. Any Phoronix Test Suite user is permitted to upload their test results, test profiles, and suites to OpenBenchmarking.org.</p>
<p>When finding a set of results on OpenBenchmarking.org, it's as easy as running the Phoronix Test Suite with that OpenBenchmarking.org ID to perform an automated side-by-side comparison (e.g. <em>phoronix-test-suite benchmark 1203160-BY-NVTEGRA3785</em>).</p>
<p>Thanks to the wealth of test data (results, system logs, etc) from crowd-sourced benchmarking via the Phoronix Test Suite, a plethora of analytical features are also available from OpenBenchmarking.org.</p>
<h1>Phoromatic</h1>
<p>Phoromatic is a remote management system for the Phoronix Test Suite that allows the automatic scheduling of tests, remote installation of new tests, and the management of multiple test systems all through an intuitive, easy-to-use web interface. Tests can be scheduled to automatically run on a routine basis across multiple test systems. Phoromatic can also interface with revision control systems to offer support for issuing new tests on a context-basis, such as whenever a Git commit has been pushed or new daily image available. The test results are then available from this central, secure location.</p>
<p>Phoromatic is an add-on to the Phoronix Test Suite that's primarily intended for enterprise users when facilitating tests across a wide-spectrum of hardware within a test lab or when needing to carry out tests on a routine basis.</p>
<p>A Phoromatic server can be started using <em>phoronix-test-suite start-phoromatic-server</em> (or the included systemd phoromatic-server service file). Clients can connect to the server using the <em>phoronix-test-suite phoromatic.connect</em> command as well as a phoromatic-client systemd service. See the Phoromatic section of the documentation for more information on setting up Phoromatic.</p>
<hr style="height: 50px; border: 0;"><a name="UserOptions"></a><h1>User Options</h1>
<p>The following options are currently supported by the Phoronix Test Suite client. A list of available options can also be found by running <em>phoronix-test-suite help.</em></p><hr>
<h1>System</h1>
<h3>interactive</h3>
<p>A simple text-driven interactive interface to the Phoronix Test Suite.</p>
<h3>php-conf</h3>
<p>This option will print information that is useful to developers when debugging problems with the Phoronix Test Suite and/or test profiles and test suites.</p>
<h3>shell</h3>
<p>A simple text-driven shell interface / helper to the Phoronix Test Suite. Ideal for those that may be new to the Phoronix Test Suite</p>
<h3>system-info</h3>
<p>Display the installed system hardware and software information as detected by the Phoronix Test Suite Phodevi Library.</p>
<h3>system-properties</h3>
<p>Display various hardware/software system properties detected by the Phoronix Device Interface (Phodevi) library.</p>
<h3>system-sensors</h3>
<p>Display the installed system hardware and software sensors in real-time as detected by the Phoronix Test Suite Phodevi Library.</p>
<h1>Test Installation</h1>
<h3>force-install<em> [Test | Suite | OpenBenchmarking ID | Test Result] ...</em>
</h3>
<p>This option will force the installation (or re-installation) of a test or suite. The arguments and process is similar to the install option but even if the test is installed, the entire installation process will automatically be executed. This option is generally used when debugging a test installation problem or wishing to re-install test(s) due to compiler or other environmental changes.</p>
<h3>install<em> [Test | Suite | OpenBenchmarking ID | Test Result] ...</em>
</h3>
<p>This option will install the selected test(s) inside the testing environment directory. The install process from downloading of the test files to the installation is fully automated. The install option needs to be supplied with the test name or suite as an argument. Optionally, a OpenBenchmarking.org ID or the name of a saved results file can be supplied as well and the test(s) to install will automatically be extracted from that information. If the test is already installed and was run by the latest version of the installation process, no action will be taken. Multiple arguments can be supplied to install additional tests at the same time.</p>
<h3>install-dependencies<em> [Test | Suite | OpenBenchmarking ID | Test Result] ...</em>
</h3>
<p>This option will install the external dependencies needed by the selected test(s) using the distribution's package management system. For example, some tests depend upon GCC for compiling code. If GCC is not detected on the system, the Phoronix Test Suite will attempt to install GCC using the distribution's package management system. If you are running this command as a local user, you may be prompted for the root password while the process is running. For unsupported distributions, the dependency names will be displayed along with common names for the package. The install-dependencies option needs to be supplied with the test name or suite as an argument. When using the install option, the external dependencies are automatically checked.</p>
<h3>make-download-cache</h3>
<p>This option will create a download cache for use by the Phoronix Test Suite. The download cache is created of test files already downloaded to the local system. If passing any test/suite names to make-download-cache, the needed files for those test profiles will first be automatically downloaded before creating the cache.</p>
<h3>remove-installed-test<em> [Test]</em>
</h3>
<p>This option will permanently remove a installed test by the Phoronix Test Suite.</p>
<h1>Testing</h1>
<h3>benchmark<em> [Test | Suite | OpenBenchmarking ID | Test Result] ...</em>
</h3>
<p>This option will install the selected test(s) (if needed) and will proceed to run the test(s). This option is equivalent to running phoronix-test-suite with the install option followed by the run option. Multiple arguments can be supplied to run additional tests at the same time and save the results into one file.</p>
<h3>estimate-install-time<em> [Test | Suite | OpenBenchmarking ID | Test Result]</em>
</h3>
<p>This option will provide estimates for test install/setup time length.</p>
<h3>estimate-run-time<em> [Test | Suite | OpenBenchmarking ID | Test Result]</em>
</h3>
<p>This option will provide estimates for test run-time / length.</p>
<h3>finish-run<em> [Test Result]</em>
</h3>
<p>This option can be used if a test run had not properly finished running all tests within a saved results file. Using this option when specifying a saved results file where all tests had not completed will attempt to finish / resume testing on the remaining tests where there are missing results to be completed.</p>
<h3>run<em> [Test | Suite | OpenBenchmarking ID | Test Result] ...</em>
</h3>
<p>This option will run the selected test(s).</p>
<h3>run-random-tests</h3>
<p>This option will query OpenBenchmarking.org to run random benchmarks and result comparisons on the system. This test can be used for simply supplying interesting results from your system onto OpenBenchmarking.org, stressing your system with random workloads, seeding new OpenBenchmarking.org results, etc. Basic options are provided at start-up for tuning the randomness of the testing when running this command.</p>
<h3>run-subset<em> [Test | Suite | OpenBenchmarking ID | Test Result] ...</em>
</h3>
<p>This option will run the selected test(s) but prompt the user when passing any test suites or result files what subset / test(s) contained within there to run rather than running all passed tests/suites/results.</p>
<h3>run-tests-in-suite<em> [Suite]</em>
</h3>
<p>This option can be used if you wish to run all of the tests found in a supplied suite, but you wish to re-configure each of the test options rather than using the defaults supplied by the suite.</p>
<h3>stress-batch-run<em> [Test | Suite | OpenBenchmarking ID | Test Result] ...</em>
</h3>
<p>This option will run the passed tests/suites in the multi-process stress-testing mode while behaving by the Phoronix Test Suite batch testing characteristics. The stress-batch-run mode is similar to the stress-run command except that for any tests passed to it will run all combinations of the options rather than prompting the user for the values to be selected.</p>
<h3>stress-run<em> [Test | Suite | OpenBenchmarking ID | Test Result] ...</em>
</h3>
<p>This option will run the passed tests/suites in the multi-process stress-testing mode. The stress-run mode will not produce a result file but is rather intended for running multiple test profiles concurrently to stress / burn-in the system. The number of tests to run concurrently can be toggled via the PTS_CONCURRENT_TEST_RUNS environment variable and by default is set to a value of 2.</p>
<h3>strict-benchmark<em> [Test | Suite | OpenBenchmarking ID | Test Result] ...</em>
</h3>
<p>This option is equivalent to the `benchmark` option except it enables various options to run benchmarks an extended number of times for ensuring better statistical accuracy if enforcing strict controls over the data quality, in some cases running the benchmarks for 20+ times.</p>
<h3>strict-run<em> [Test | Suite | OpenBenchmarking ID | Test Result] ...</em>
</h3>
<p>This option is equivalent to the `run` option except it enables various options to run benchmarks an extended number of times for ensuring better statistical accuracy if enforcing strict controls over the data quality, in some cases running the benchmarks for 20+ times.</p>
<h1>Batch Testing</h1>
<h3>batch-benchmark<em> [Test | Suite | OpenBenchmarking ID | Test Result] ...</em>
</h3>
<p>This option and its arguments are equivalent to the benchmark option, but the process will be run in the Phoronix Test Suite batch mode.</p>
<h3>batch-install<em> [Test | Suite | OpenBenchmarking ID | Test Result] ...</em>
</h3>
<p>If you wish to run the install process in the Phoronix Test Suite batch mode but do not wish to run any tests at this time. Running the install process in the batch mode will use the default values and not prompt the user of any possible options, to ensure the process is fully automated.</p>
<h3>batch-run<em> [Test | Suite | OpenBenchmarking ID | Test Result] ...</em>
</h3>
<p>This option and its arguments are equivalent to the run option, but the process will be run in the Phoronix Test Suite batch mode.</p>
<h3>batch-setup</h3>
<p>This option is used to configure the batch mode options for the Phoronix Test Suite, which is subsequently written to the user configuration file. Among the options are whether to automatically upload the test results to OpenBenchmarking.org and prompting for the saved file name.</p>
<h3>default-benchmark<em> [Test | Suite | OpenBenchmarking ID | Test Result] ...</em>
</h3>
<p>This option will install the selected test(s) (if needed) and will proceed to run the test(s) in the defaults mode. This option is equivalent to running phoronix-test-suite with the install option followed by the default-run option.</p>
<h3>default-run<em> [Test | Suite | OpenBenchmarking ID | Test Result] ...</em>
</h3>
<p>This option will run the selected test(s). The name of the test or suite must be supplied or the OpenBenchmarking.org ID or saved local file name. Multiple arguments can be supplied to run additional tests at the same time and save the results in a suite-like fashion. Unlike the normal run option, the default-run will not prompt the user to select from the available test options but will instead use the default options as automatically set by pts-core or the test profile. Use batch-run to automatically test all of the available options.</p>
<h3>dry-run<em> [Test | Suite | OpenBenchmarking ID | Test Result] ...</em>
</h3>
<p>This option and its arguments pre-set the Phoronix Test Suite batch run mode with enforcing of defaults to not save any results and other behavior intended for a dry/test run. This option is primarily intended for testing/evaluation purposes.</p>
<h3>internal-run<em> [Test | Suite | OpenBenchmarking ID | Test Result] ...</em>
</h3>
<p>This option and its arguments pre-set the Phoronix Test Suite batch run mode with sane values for carrying out benchmarks in a semi-automated manner and without uploading any of the result data to the public OpenBenchmarking.org.</p>
<h1>OpenBenchmarking.org</h1>
<h3>clone-result<em> [OpenBenchmarking ID] ...</em>
</h3>
<p>This option will download a local copy of a file that was saved to OpenBenchmarking.org, as long as a valid public ID is supplied.</p>
<h3>dump-suites-to-git</h3>
<p>This option will create a Git repository of OpenBenchmarking.org test suites.</p>
<h3>dump-tests-to-git</h3>
<p>This option will create a Git repository of OpenBenchmarking.org test profiles.</p>
<h3>enable-repo</h3>
<p>This option is used if wanting to add a new OpenBenchmarking.org account/repository to your system for enabling third-party/unofficial test profiles and test suites.</p>
<h3>list-recommended-tests</h3>
<p>This option will list recommended test profiles for benchmarking sorted by hardware sub-system. The recommended tests are determined via querying OpenBenchmarking.org and determining the most popular tests for a given environment based upon the number of times a test profile has been downloaded, the number of test results available on OpenBenchmarking.org for a given test profile, the age of the test profile, and other weighted factors.</p>
<h3>make-openbenchmarking-cache</h3>
<p>This option will attempt to cache the test profile/suite meta-data from OpenBenchmarking.org for all linked repositories. This is useful if you're going to be running the Phoronix Test Suite / Phoromatic behind a firewall or without any Internet connection. Those with unrestricted Internet access or not utilizing a large local deployment of the Phoronix Test Suite / Phoromatic shouldn't need to run this command.</p>
<h3>ob-test-profile-analyze</h3>
<p>This option is intended for test profile creators and generates a range of meta-data and other useful information that can be submitted to OpenBenchmarking.org to provide more verbose information for users of your test profiles.</p>
<h3>openbenchmarking-changes</h3>
<p>This option will list recent changes to test profiles of enabled OpenBenchmarking.org repositories.</p>
<h3>openbenchmarking-login</h3>
<p>This option is used for controlling your Phoronix Test Suite client options for OpenBechmarking.org and syncing the client to your account.</p>
<h3>openbenchmarking-refresh</h3>
<p>This option is used for refreshing the stored OpenBenchmarking.org repostory information and other data. The Phoronix Test Suite will automatically refresh this data every three days or when other thresholds are exceeded, but this command can be used to manually refresh/updates the data.</p>
<h3>openbenchmarking-repositories</h3>
<p>This option will list the OpenBenchmarking.org repositories currently linked to this Phoronix Test Suite client instance.</p>
<h3>openbenchmarking-uploads</h3>
<p>This option will list any recent test result uploads from the system's IP address to OpenBenchmarking.org.</p>
<h3>recently-added-tests</h3>
<p>This option will list the most recently added (newest) test profiles.</p>
<h3>upload-result<em> [Test Result]</em>
</h3>
<p>This option is used for uploading a test result to OpenBenchmarking.org.</p>
<h3>upload-test-profile</h3>
<p>This option can be used for uploading a test profile to your account on OpenBenchmarking.org. By uploading your test profile to OpenBenchmarking.org, others are then able to browse and access this test suite for easy distribution in a seamless manner by other Phoronix Test Suite clients.</p>
<h3>upload-test-suite<em> [Suite]</em>
</h3>
<p>This option can be used for uploading a test suite to your account on OpenBenchmarking.org. By uploading your test suite to OpenBenchmarking.org, others are then able to browse and access this test suite for easy distribution.</p>
<h1>Information</h1>
<h3>info<em> [Test | Suite | OpenBenchmarking ID | Test Result]</em>
</h3>
<p>This option will show details about the supplied test, suite, virtual suite, or result file.</p>
<h3>intersect<em> [Test | Suite | OpenBenchmarking ID | Test Result] ...</em>
</h3>
<p>This option will print the test profiles present in all passed result files / test suites. Two or more results/suites must be passed and printed will be all of the common test profiles.</p>
<h3>list-all-tests</h3>
<p>This option will list all test profiles that are available from the enabled OpenBenchmarking.org repositories. Unlike the other test listing options, list-all-tests will show deprecated tests, potentially broken tests, or other tests not recommended for all environments. The only check in place is ensuring the test profiles are at least compatible with the operating system in use.</p>
<h3>list-available-suites</h3>
<p>This option will list all test suites that are available from the enabled OpenBenchmarking.org repositories.</p>
<h3>list-available-tests</h3>
<p>This option will list all test profiles that are available from the enabled OpenBenchmarking.org repositories where supported on the system and are of a verified state. If the system has no Internet access, it will only list the test profiles where the necesary test assets are available locally on the system or on an available network cache (the same behavior as using the list-cached-tests sub-command), unless using the list-all-tests option to override this behavior.</p>
<h3>list-available-virtual-suites</h3>
<p>This option will list all available virtual test suites that can be dynamically created based upon the available tests from enabled OpenBenchmarking.org repositories.</p>
<h3>list-cached-tests</h3>
<p>This option will list all test profiles where any needed test profiles are already cached or available from the local system under test. This is primarily useful if testing offline/behind-the-firewall and other use-cases where wanting to rely only upon local data.</p>
<h3>list-installed-dependencies</h3>
<p>This option will list all of the packages / external test dependencies that are already installed on the system that the Phoronix Test Suite may potentially depend upon by test profiles.</p>
<h3>list-installed-suites</h3>
<p>This option will list all suites that are currently installed on the system.</p>
<h3>list-installed-tests</h3>
<p>This option will list all test profiles that are currently installed on the system.</p>
<h3>list-missing-dependencies</h3>
<p>This option will list all of the packages / external test dependencies that are missing from the system that the Phoronix Test Suite may potentially need by select test profiles.</p>
<h3>list-not-installed-tests</h3>
<p>This option will list all test profiles that are supported and available but presently NOT installed on the system.</p>
<h3>list-possible-dependencies</h3>
<p>This option will list all of the packages / external test dependencies that are are potentially used by the Phoronix Test Suite.</p>
<h3>list-saved-results</h3>
<p>This option will list all of the saved test results found on the system.</p>
<h3>list-test-status</h3>
<p>This sub-command provides a verbose look at all tests installed/uninstalled on the system and whether any errors were encountered at install-time or run-time and other test installation/runtime metrics for complementing other Phoronix Test Suite sub-command outputs.</p>
<h3>list-test-usage</h3>
<p>This option will list various details about installed tests and their usage.</p>
<h3>search</h3>
<p>This option provides command-line searching abilities for test profiles / test suites / test results. The search query can be passed as a parameter otherwise the user is prompted to input their search query..</p>
<h3>test-to-suite-map</h3>
<p>This option will list all test profiles and any test suites each test belongs to.</p>
<h1>Asset Creation</h1>
<h3>build-suite</h3>
<p>This option will guide the user through the process of generating their own test suite, which they can then run. Optionally, passed as arguments can be the test(s) or suite(s) to add to the suite to be created, instead of being prompted through the process.</p>
<h3>create-test-profile</h3>
<p>This option can be used for creating a Phoronix Test Suite test profile by answering questions about the test for constructing the test profile XML meta-data and handling other boiler-plate basics for getting started in developing new tests.</p>
<h3>debug-benchmark<em> [Test | Suite | OpenBenchmarking ID | Test Result] ...</em>
</h3>
<p>This option is intended for use by test profile writers and is identical to the <em>run</em> option but will yield more information during the run process that can be used to debug issues with a test profile or to verify the test profile is functioning correctly.</p>
<h3>debug-install<em> [Test | Suite | OpenBenchmarking ID | Test Result] ...</em>
</h3>
<p>This option is intended for use by test profile writers and is identical to the install option but will yield more information during the run process that can be used to debug issues with a test profile installer or to verify the test profile is functioning correctly.</p>
<h3>debug-result-parser<em> [Test | Suite | OpenBenchmarking ID | Test Result] ...</em>
</h3>
<p>This option is intended for use by test profile writers and is used for debugging a result parser. No test execution is done, but there must already be PTS-generated .log files present within the test's installation directory.</p>
<h3>debug-test-download-links<em> [Test | Suite | OpenBenchmarking ID | Test Result]</em>
</h3>
<p>This option will check all download links within the specified test profile(s) to ensure there are no broken URLs.</p>
<h3>download-test-files<em> [Test | Suite | OpenBenchmarking ID | Test Result] ...</em>
</h3>
<p>This will download the selected test file(s) to the Phoronix Test Suite download cache but will not install the tests.</p>
<h3>dump-documentation</h3>
<p>This option is used for re-generating the Phoronix Test Suite documentation.</p>
<h3>inspect-test-profile<em> [Test]</em>
</h3>
<p>This option can be used for inspecting a Phoronix Test Suite test profile with providing inside details on test profiles for debugging / evaluation / learning purposes.</p>
<h3>rebuild-test-suite<em> [Suite]</em>
</h3>
<p>This option will regenerate the local test suite XML file against the OpenBenchmarking.org specification. This can be used to clean up any existing XML syntax / styling issues, etc.</p>
<h3>validate-result-file</h3>
<p>This option can be used for validating a Phoronix Test Suite result file as being compliant against the OpenBenchmarking.org specification.</p>
<h3>validate-test-profile<em> [Test]</em>
</h3>
<p>This option can be used for validating a Phoronix Test Suite test profile as being compliant against the OpenBenchmarking.org specification.</p>
<h3>validate-test-suite<em> [Suite]</em>
</h3>
<p>This option can be used for validating a Phoronix Test Suite test suite as being compliant against the OpenBenchmarking.org specification.</p>
<h1>Result Management</h1>
<h3>auto-sort-result-file<em> [Test Result]</em>
</h3>
<p>This option is used if you wish to automatically attempt to sort the results by their result identifier string. Alternatively, if using the environment variable "SORT_BY" other sort modes can be used, such as SORT_BY=date / SORT_BY=date-desc for sorting by the test run-time/date.</p>
<h3>compare-results-to-baseline<em> [Test Result] [Test Result]</em>
</h3>
<p>This option will allows you to specify a result as a baseline (first parameter) and a second result file (second parameter) that will offer some analysis for showing how the second result compares to the first in matching tests.</p>
<h3>compare-results-two-way<em> [Test Result]</em>
</h3>
<p>This option will allows you to specify a result file and from there to compare two individual runs within that result file for looking at wins/losses and other metrics in a head-to-head type comparison.</p>
<h3>edit-result-file<em> [Test Result]</em>
</h3>
<p>This option is used if you wish to edit the title and description of an existing result file.</p>
<h3>extract-from-result-file<em> [Test Result]</em>
</h3>
<p>This option will extract a single set of test results from a saved results file that contains multiple test results that have been merged. The user is the prompted to specify a new result file name and select which result identifier to extract.</p>
<h3>keep-results-in-result-file<em> [Test Result]</em>
</h3>
<p>This option is the inverse of the remove-results-from-result-file sub-command. If you wish to remove all results but those listed from a given result file, this option can be used. The user must specify a saved results file and then they will be prompted to provide a string to search for in keeping those results in that given result file but removing all other data.</p>
<h3>merge-results<em> [Test Result] ...</em>
</h3>
<p>This option will manually merge multiple sets of test results generated by the Phoronix Test Suite.</p>
<h3>remove-incomplete-results-from-result-file<em> [Test Result]</em>
</h3>
<p>This option is used if there are test results (benchmarks) to be dropped from a given result file for having incomplete data, either a test run did not attempt to run that benchmark or failed to properly run. The user must specify a saved results file and the command will then attempt to find any results with incomplete/missing data and prompt the user with confirmation to remove them.</p>
<h3>remove-result<em> [Test Result]</em>
</h3>
<p>This option will permanently remove the saved file set that is set as the first argument.</p>
<h3>remove-result-from-result-file<em> [Test Result]</em>
</h3>
<p>This option is used if there are test results (benchmarks) to be dropped from a given result file. The user must specify a saved results file and then they will be prompted to select the tests/benchmarks to remove.</p>
<h3>remove-results-from-result-file<em> [Test Result]</em>
</h3>
<p>This option is used if there are test results (benchmarks) to be dropped from a given result file. The user must specify a saved results file and then they will be prompted to provide a string to search for in removing those results from that given result file.</p>
<h3>remove-run-from-result-file<em> [Test Result]</em>
</h3>
<p>This option is used if there is a set of test results you wish to remove/delete from a saved results file. The user must specify a saved results file and then they will be prompted to select the results identifier associated with the results they wish to remove.</p>
<h3>rename-identifier-in-result-file<em> [Test Result]</em>
</h3>
<p>This option is used if you wish to change the name of the identifier in a test results file that is shown in the Phoronix Test Suite Results Viewer and the contained graphs.</p>
<h3>rename-result-file<em> [Test Result]</em>
</h3>
<p>This option is used if you wish to change the name of the saved name of a result file.</p>
<h3>reorder-result-file<em> [Test Result]</em>
</h3>
<p>This option is used if you wish to manually change the order in which test results are shown in the Phoronix Test Suite Results Viewer and the contained graphs. The user must specify a saved results file and then they will be prompted to select the results identifiers one at a time in the order they would like them to be displayed from left to right.</p>
<h3>show-result<em> [Test Result]</em>
</h3>
<p>Open up the test results in the Phoronix Test Suite Result Viewer or on OpenBenchmarking.org.</p>
<h1>Other</h1>
<h3>commands</h3>
<p>This option will display a short list of possible Phoronix Test Suite commands.</p>
<h3>debug-dependency-handler</h3>
<p>This option is used for testing the distribution-specific dependency handler for external dependencies.</p>
<h3>debug-render-test</h3>
<p>This option is used during the development of the Phoronix Test Suite software for testing of the result and graph rendering code-paths This option will download a large number of reference test results from LinuxBenchmarking.com.</p>
<h3>debug-self-test</h3>
<p>This option is used during the development of the Phoronix Test Suite software for testing of internal interfaces, commands, and other common code-paths. The produced numbers should only be comparable for the same version of the Phoronix Test Suite, on the same hardware/software system, conducted on the same day of testing. This isn't intended as any scientific benchmark but simply to stress common PHP code-paths and looking for hot areas to optimize, etc.</p>
<h3>help</h3>
<p>This option will display a list of available Phoronix Test Suite commands and possible parameter types.</p>
<h3>version</h3>
<p>This option will display the Phoronix Test Suite client version.</p>
<h1>Result Analysis</h1>
<h3>analyze-run-times<em> [Test Result]</em>
</h3>
<p>This option will read a saved test results file and print the statistics about how long the testing took to complete.</p>
<h3>executive-summary<em> [Test Result]</em>
</h3>
<p>This option will attempt to auto-generate a textual executive summary for a result file to highlight prominent results / averages.</p>
<h3>result-file-confidence<em> [Test Result]</em>
</h3>
<p>This option will read a saved test results file and display various statistics on the confidence of the results with the standard deviation, three-sigma values, and other metrics while color-coding "passing" results in green.</p>
<h3>result-file-stats<em> [Test Result]</em>
</h3>
<p>This option is used if you wish to analyze a result file by seeing various statistics on the result data for result files containing at least two sets of data.</p>
<h3>wins-and-losses<em> [Test Result]</em>
</h3>
<p>This option is used if you wish to analyze a result file to see which runs produced the most wins/losses of those result identifiers in the saved file.</p>
<h3>workload-topology<em> [Test Result]</em>
</h3>
<p>This option will read a saved test results file and print the test profiles contained within and their arrangement within different test suites for getting an idea as to the workload topology/make-up / logical groupings of the benchmarks being run.</p>
<h1>Modules</h1>
<h3>auto-load-module</h3>
<p>This option can be used for easily adding a module to the AutoLoadModules list in the Phoronix Test Suite user configuration file. That list controls what PTS modules are automatically loaded on start-up of the Phoronix Test Suite.</p>
<h3>list-modules</h3>
<p>This option will list all of the available Phoronix Test Suite modules on this system.</p>
<h3>module-info<em> [Phoronix Test Suite Module]</em>
</h3>
<p>This option will show detailed information on a Phoronix Test Suite module such as the version, developer, and a description of its purpose.</p>
<h3>module-setup<em> [Phoronix Test Suite Module]</em>
</h3>
<p>This option will allow you to configure all available end-user options for a Phoronix Test Suite module. These options are then stored within the user's configuration file. Not all modules may have options that can be configured by the end-user.</p>
<h3>test-module<em> [Phoronix Test Suite Module]</em>
</h3>
<p>This option can be used for debugging a Phoronix Test Suite module.</p>
<h3>unload-module</h3>
<p>This option can be used for easily removing a module from the AutoLoadModules list in the Phoronix Test Suite user configuration file. That list controls what modules are automatically loaded on start-up of the Phoronix Test Suite.</p>
<h1>Debugging</h1>
<h3>check-tests<em> [Test]</em>
</h3>
<p>This option will perform a check on one or more test profiles to determine if there have been any vendor changes to the filename, filesize, url location, md5 and sha256 checksums.</p>
<h3>diagnostics</h3>
<p>This option will print information that is useful to developers when debugging problems with the Phoronix Test Suite and/or test profiles and test suites.</p>
<h3>dump-file-info</h3>
<p>This option will dump the MD5 / SHA256 hashes and file size for a given file.</p>
<h3>dump-openbenchmarking-indexes</h3>
<p>This option is used for dumping the parsed output of OpenBenchmarking.org index files (metadata).</p>
<h3>dump-phodevi-smart-cache</h3>
<p>This option is used for displaying the contents of the Phodevi smart cache on the system.</p>
<h3>dump-possible-options</h3>
<p>This option will print all possible phoronix-test-suite sub-commands.</p>
<h3>dump-unhandled-dependencies</h3>
<p>This option will list missing entries in the external dependencies XML file for the operating system under test. This option is used if wanting to help find missing dependency XML data to fill in for contributing to upstream Phoronix Test Suite.</p>
<h3>list-failed-installs</h3>
<p>This option will list all test profiles that were attempted to be installed on the local system but failed to be installed. Where applicable, the possible error(s) from the test installation are also reported to assist in debugging.</p>
<h3>list-test-errors</h3>
<p>This sub-command is complementary to list-failed-installs. Rather than listing test installation errors, list-test-errors is used for displaying past test run-time errors. This option will list all test profiles that produced an error previously when running the test profile / benchmark. If a test profile later successfully ran the test with any given option(s) without errors, the error is then removed from the archive. This option is intended to be helpful in debugging test profile issues later on for having a persistent collection of run-time errors.</p>
<h3>list-unsupported-tests</h3>
<p>This option will list all available test profiles that are available from the enabled OpenBenchmarking.org repositories but are NOT SUPPORTED on the given hardware/software platform. This is mainly a debugging option for those looking for test profiles to potentially port to new platforms, etc.</p>
<h1>User Configuration</h1>
<h3>enterprise-setup</h3>
<p>This option can be run by enterprise users immediately after package installation or as part of an in-house setup script. Running this command will ensure the phoronix-test-suite program is never interrupted on new runs to accept user agreement changes and defaults the anonymous usage reporting to being disabled and other conservative defaults.</p>
<h3>network-info</h3>
<p>This option will print information detected by the Phoronix Test Suite around the system's network configuration.</p>
<h3>network-setup</h3>
<p>This option allows the user to configure how the Phoronix Test Suite connects to OpenBenchmarking.org and other web-services. Connecting through an HTTP proxy can be configured through this option.</p>
<h3>user-config-reset</h3>
<p>This option can be used for resetting the Phoronix Test Suite user configuration file to its default state.</p>
<h3>user-config-set</h3>
<p>This option can be used for setting an XML value in the Phoronix Test Suite user configuration file.</p>
<h3>variables</h3>
<p>This option will print all of the official environment variables supported by the Phoronix Test Suite for user configuration purposes. These environment variables are also listed as part of the official Phoronix Test Suite documentation while this command will also show the current value of the variables if currently set.</p>
<h1>Result Export</h1>
<h3>result-file-raw-to-csv<em> [Test Result]</em>
</h3>
<p>This option will read a saved test results file and output the raw result file run data to a CSV file. This raw (individual) result file output is intended for data analytic purposes where the result-file-to-csv is more end-user-ready.</p>
<h3>result-file-to-csv<em> [Test Result]</em>
</h3>
<p>This option will read a saved test results file and output the system hardware and software information along with the results to a CSV output. The CSV (Comma Separated Values) output can then be loaded into a spreadsheet for easy viewing. The outputted file appears in the user home directory or can otherwise be controlled via the OUTPUT_DIR and OUTPUT_FILE environment variables.</p>
<h3>result-file-to-html<em> [Test Result]</em>
</h3>
<p>This option will read a saved test results file and output the system hardware and software information along with the results to pure HTML file. No external files are required for CSS/JavaScript or other assets. The graphs are rendered as inline SVG. This is a pure HTML-only representation of the results for emailing or other easy analysis outside of the Phoronix Test Suite. The outputted file appears in the user home directory or can otherwise be controlled via the OUTPUT_DIR and OUTPUT_FILE environment variables.</p>
<h3>result-file-to-json<em> [Test Result]</em>
</h3>
<p>This option will read a saved test results file and output the basic result information to JSON (JavaScript Object Notation). The outputted file appears in the user home directory or can otherwise be controlled via the OUTPUT_DIR and OUTPUT_FILE environment variables.</p>
<h3>result-file-to-pdf<em> [Test Result]</em>
</h3>
<p>This option will read a saved test results file and output the system hardware and software information along with the results to a PDF file. The outputted file appears in the user home directory or can otherwise be controlled via the OUTPUT_DIR and OUTPUT_FILE environment variables.</p>
<h3>result-file-to-suite<em> [Test Result]</em>
</h3>
<p>This option will guide the user through the process of generating their own test suite, which they can then run, that is based upon an existing test results file.</p>
<h3>result-file-to-text<em> [Test Result]</em>
</h3>
<p>This option will read a saved test results file and output the system hardware and software information to the terminal. The test results are also outputted.</p>
<h1>Phoromatic</h1>
<h3>start-phoromatic-server</h3>
<p>Start the Phoromatic web server for controlling local Phoronix Test Suite client systems to facilitate automated and repeated test orchestration and other automated features targeted at the enterprise.</p>
<h1>Result Viewer</h1>
<h3>start-result-viewer</h3>
<p>Start the web-based result viewer.</p>
<hr style="height: 50px; border: 0;"><a name="ModuleOptions"></a><h1>Module Options</h1>
<p>The following list is the modules included with the Phoronix Test Suite that are intended to extend the functionality of pts-core. Some of these options have commands that can be run directly in a similiar manner to the other Phoronix Test Suite user commands. Some modules are just meant to be loaded directly by adding the module name to the AutoLoadModules tag in ~/.phoronix-test-suite/user-config.xml or via the PTS_MODULES environment variable. A list of available modules is also available by running <em>phoronix-test-suite list-modules.</em></p><hr>
<h2>Backup Creation + Restore</h2>
<p>This is a module for creating backups of the Phoronix Test Suite / Phoromatic and allows for restoring of created backups. The backup will be in ZIP or TAR format. If only a path is specified, the file-name will be auto-generated with a current time-stamp.</p>
<p><strong>phoronix-test-suite backup.create</strong></p>
<p><strong>phoronix-test-suite backup.restore</strong></p>
<h2>System Maintenance / Cleanup</h2>
<p>This module can be used for system maintenance cleanup tasks around the Phoronix Test Suite. Currently implemented is support for automatically un-installing tests that have not been run in a period of time. When the module is loaded via the REMOVE_TESTS_OLDER_THAN environment variable, it will be automatically invoked at the end of running any benchmarks. Or this module can be manually invoked with the command: phoronix-test-suite cleanup.tests.</p>
<p><strong>phoronix-test-suite cleanup.tests</strong></p>
<p>This module utilizes the following environment variables: REMOVE_TESTS_OLDER_THAN.</p>
<h2>Dummy Module</h2>
<p>This is a simple module intended for developers to just demonstrate some of the module functions.</p>
<p><strong>phoronix-test-suite dummy_module.dummy-command</strong></p>
<p>This is a simple module intended for developers to just demonstrate some of the module functions.</p>
<h2>Flush Caches</h2>
<p>Loading this module will ensure caches (page cache, swap, etc) automatically get flushed prior to running any test.</p>
<p>This module utilizes the following environment variables: PTS_FLUSH_CACHES.</p>
<h2>Result Exporter To HTML</h2>
<p>This module allows basic exporting of results to HTML for saving either to a file locally (specified using the EXPORT_RESULTS_HTML_FILE_TO environment variable) or to a mail account (specified using the EXPORT_RESULTS_HTML_EMAIL_TO environment variable). EXPORT_RESULTS_HTML_EMAIL_TO supports multiple email addresses delimited by a comma.</p>
<p>This module utilizes the following environment variables: EXPORT_RESULTS_HTML_EMAIL_TO, EXPORT_RESULTS_HTML_FILE_TO.</p>
<h2>Linux Perf Framework Reporter</h2>
<p>Setting LINUX_PERF=1 will auto-load and enable this Phoronix Test Suite module. The module also depends upon running a modern Linux kernel (supporting perf) and that the perf binary is available via standard system paths. Depending upon system permissions you may be limited to using perf as root or adjusting the /proc/sys/kernel/perf_event_paranoid setting.</p>
<p>This module utilizes the following environment variables: LINUX_PERF.</p>
<h2>Dynamic Result Viewer</h2>
<p>This module pre-loads the HTTP dynamic result viewer for Phoronix Test Suite data.</p>
<p><strong>phoronix-test-suite load_dynamic_result_viewer.start</strong></p>
<h2>Log Exporter</h2>
<p>This module allows for easily exporting test run logs and system logs to external locations via specifying the directory paths via the COPY_TEST_RUN_LOGS_TO and COPY_SYSTEM_LOGS_TO environment variables.</p>
<p>This module utilizes the following environment variables: COPY_TEST_RUN_LOGS_TO, COPY_SYSTEM_LOGS_TO.</p>
<h2>MATISK</h2>
<p>My Automated Test Infrastructure Setup Kit</p>
<p><strong>phoronix-test-suite matisk.run</strong></p>
<p><strong>phoronix-test-suite matisk.template</strong></p>
<h2>OpenBenchmarking.org Auto Comparison</h2>
<p>This module prints comparable OpenBenchmarking.org results in the command-line for reference purposes as tests are being run. OpenBenchmarking.org is automatically queried for results to show based on the test comparison hash and the system type (mobile, desktop, server, cloud, workstation, etc). No other system information or result data is transmitted.</p>
<p><strong>phoronix-test-suite ob_auto_compare.debug</strong></p>
<h2>Performance Per Dollar/Cost Calculator</h2>
<p>Setting the COST_PERF_PER_DOLLAR= environment variable to whatever value of the system cost/component you are running a comparison on will yield extra graphs that calculate the performance-per-dollar based on the test being run. The COST_PERF_PER_DOLLAR environment variable is applied just to the current test run identifier. Set the COST_PERF_PER_UNIT= environment variable if wishing to use a metric besides dollar/cost. The COST_PERF_PER_HOUR value can be used rather than COST_PERF_PER_DOLLAR if wishing to calculate the e.g. cloud time or other compute time based on an hourly basis.</p>
<p><strong>phoronix-test-suite perf_per_dollar.add</strong></p>
<p>This module utilizes the following environment variables: COST_PERF_PER_DOLLAR, COST_PERF_PER_UNIT, COST_PERF_PER_HOUR.</p>
<h2>Performance Tip Prompts</h2>
<p>This module alerts the user if the system configuration may not be the right one for achieving the best performance with the target benchmark(s). This initial version of the module actually cares only about the BFQ I/O scheduler and powersave governor checks.</p>
<p><strong>phoronix-test-suite perf_tips.show</strong></p>
<p>This module utilizes the following environment variables: SUPPRESS_PERF_TIPS.</p>
<p>This module alerts the user if the system configuration may not be the right one for achieving the best performance with the target benchmark(s). This initial version of the module actually cares only about the BFQ I/O scheduler: it gives a warning if BFQ is being used with an incorrect configuration in a disk benchmark, and suggests the right configuration to use. For the moment it only works for existing, throughput-based tests. It will need to be extended for responsiveness and soft real-time-latency tests.</p>
<h2>Benchmarking Compiler PGO Impact</h2>
<p>This module makes it easy to test a compiler PGO (Profile Guided Optimization) performance impact by running a test without PGO optimizations, capturing the PGO profile, rebuilding the tests with the PGO profile generated, and then repeat the benchmarks.</p>
<p><strong>phoronix-test-suite pgo.benchmark</strong></p>
<h2>Phoromatic Client</h2>
<p>The Phoromatic client is used for connecting to a Phoromatic server (Phoromatic.com or a locally run server) to facilitate the automatic running of tests, generally across multiple test nodes in a routine manner. For more details visit http://www.phoromatic.com/. This module is intended to be used with Phoronix Test Suite 5.2+ clients and servers.</p>
<p><strong>phoronix-test-suite phoromatic.connect</strong></p>
<p><strong>phoronix-test-suite phoromatic.explore</strong></p>
<p><strong>phoronix-test-suite phoromatic.upload-result</strong></p>
<p><strong>phoronix-test-suite phoromatic.set-root-admin-password</strong></p>
<p><strong>phoronix-test-suite phoromatic.list-results</strong></p>
<p><strong>phoronix-test-suite phoromatic.clone</strong></p>
<p><strong>phoronix-test-suite phoromatic.export-results-for-account-schedules</strong></p>
<p>The Phoromatic module contains the client support for interacting with Phoromatic and Phoromatic Tracker services.</p>
<h2>Pushover.net</h2>
<p>Submit notifications to your iOS/Android mobile devices of test results in real-time as push notifications, etc. Using the Pushover.net API.</p>
<p>This module utilizes the following environment variables: PUSHOVER_NET_USER.</p>
<h2>Report Test Time Graphs</h2>
<p>Setting the RUN_TIMES_ARE_A_BENCHMARK=1 environment variable will automatically create additional graphs for each test run plotting the run-time needed for each test being executed. Setting the INSTALL_TIMES_ARE_A_BENCHMARK=1 environment variable will automatically create additional graphs for each test run plotting the time required for the test installation. Setting the INSTALL_SIZES_ARE_A_BENCHMARK=1 environment variable will automatically create additional graphs for each test run plotting the size of the installed test directory.</p>
<p>This module utilizes the following environment variables: RUN_TIMES_ARE_A_BENCHMARK, INSTALL_TIMES_ARE_A_BENCHMARK, INSTALL_SIZES_ARE_A_BENCHMARK.</p>
<h2>Result Notifier</h2>
<p>A notification module.</p>
<h2>Custom Result Export Methods</h2>
<p>A simple example module about interfacing with Phoronix Test Suite core for dumping result files in a custom format.</p>
<p><strong>phoronix-test-suite results_custom_export.nf</strong></p>
<h2>System Monitor</h2>
<p>This module contains sensor monitoring support.</p>
<p>This module utilizes the following environment variables: MONITOR, PERFORMANCE_PER_WATT, PERFORMANCE_PER_SENSOR, MONITOR_INTERVAL, MONITOR_PER_RUN.</p>
<p>Monitoring these sensors is as easy as running your normal Phoronix Test Suite commands but at the beginning of the command add: MONITOR=<selected sensors>. For example, this will monitor the CPU temperature and voltage during tests:
MONITOR=cpu.temp,cpu.voltage phoronix-test-suite benchmark universe
For some of the sensors there is an ability to monitor specific device, e.g. cpu.usage.cpu0 or hdd.read-speed.sda. If the PERFORMANCE_PER_WATT environment variable is set, a performance per Watt graph will also be added, assuming the system's power consumption can be monitored. PERFORMANCE_PER_SENSOR= will allow similar behavior but for arbitrary sensors. Below are all of the sensors supported by this version of the Phoronix Test Suite.
Supported Options:
- all
- all.ambient
- ambient.temp
- all.cgroup
- cgroup.cpu-usage
- all.cpu
- cpu.fan-speed
- cpu.freq
- all.cpu.freq
- cpu.freq.cpu0
- cpu.freq.cpu1
- cpu.freq.cpu2
- cpu.freq.cpu3
- cpu.freq.cpu4
- cpu.freq.cpu5
- cpu.freq.cpu6
- cpu.freq.cpu7
- cpu.freq.cpu8
- cpu.freq.cpu9
- cpu.freq.cpu10
- cpu.freq.cpu11
- cpu.freq.cpu12
- cpu.freq.cpu13
- cpu.freq.cpu14
- cpu.freq.cpu15
- cpu.peak-freq
- cpu.power
- cpu.temp
- cpu.usage
- all.cpu.usage
- cpu.usage.cpu0
- cpu.usage.cpu1
- cpu.usage.cpu2
- cpu.usage.cpu3
- cpu.usage.cpu4
- cpu.usage.cpu5
- cpu.usage.cpu6
- cpu.usage.cpu7
- cpu.usage.cpu8
- cpu.usage.cpu9
- cpu.usage.cpu10
- cpu.usage.cpu11
- cpu.usage.cpu12
- cpu.usage.cpu13
- cpu.usage.cpu14
- cpu.usage.cpu15
- cpu.usage.summary
- cpu.voltage
- all.gpu
- gpu.fan-speed
- gpu.freq
- gpu.memory-usage
- gpu.power
- gpu.temp
- gpu.usage
- gpu.voltage
- all.hdd
- hdd.read-speed
- all.hdd.read-speed
- hdd.read-speed.sda
- hdd.read-speed.sdb
- hdd.read-speed.nvme0n1
- hdd.temp
- all.hdd.temp
- hdd.temp.sda
- hdd.temp.sdb
- hdd.temp.nvme0n1
- hdd.write-speed
- all.hdd.write-speed
- hdd.write-speed.sda
- hdd.write-speed.sdb
- hdd.write-speed.nvme0n1
- all.memory
- memory.temp
- memory.usage
- all.swap
- swap.usage
- all.sys
- sys.fan-speed
- sys.iowait
- sys.power
- sys.temp
- sys.voltage
- all.sys.voltage
NOTE: Use the "system-sensors" command to see what sensors are available for monitoring on the system.</p>
<h2>Test Timeout</h2>
<p>This module allows killing a test if it exceeds a defined threshold, such as if the test is hung, etc. TEST_TIMEOUT_AFTER= environment variable can be used for controlling the behavior. When this variable is set, the value will can be set to "auto" or a positive integer. The value indicates the number of minutes until a test run should be aborted, such as for a safeguard against hung/deadlocked processes or other issues. Setting this to a high number as a backup would be recommended for fending off possible hangs / stalls in the testing process if the test does not quit on its own for whatever reason. If the value is "auto", it will quit if the time of a test run exceeds 3x the average time it normally takes the particular test to complete its run.</p>
<p>This module utilizes the following environment variables: TEST_TIMEOUT_AFTER.</p>
<h2>Timed Screenshot</h2>
<p>This is a module that will take a screenshot of the system at a pre-defined interval. ImageMagick must be installed onto the system prior to using this module.</p>
<p>This module utilizes the following environment variables: SCREENSHOT_INTERVAL.</p>
<h2>Toggle Screensaver</h2>
<p>This module toggles the system's screensaver while the Phoronix Test Suite is running. At this time, the GNOME and KDE screensavers are supported.</p>
<p>This module utilizes the following environment variables: HALT_SCREENSAVER.</p>
<h2>Linux Turbostat Dumper</h2>
<p>Setting TURBOSTAT_LOG=_DIR_ will auto-load and enable this Phoronix Test Suite module. The module will -- if turbostat is installed on the system and the user is root -- allow dumping of the TurboStat data to the specified directly on a per-test basis. This allows easily collecting of turbostat logs for each test being run. If the TURBOSTAT_LOG= value does not point to a directory, the TurboStat output will be appended to the test run log files.</p>
<p>This module utilizes the following environment variables: TURBOSTAT_LOG.</p>
<h2>Update Checker</h2>
<p>This module checks to see if the Phoronix Test Suite -- and its tests and suites -- are up to date plus also handles message of the day information.</p>
<h2>Utilize Wine On Linux Benchmarking</h2>
<p>This module when activated via the USE_WINE environment variable on Linux systems will override the test profile OS target to Windows and attempt to run the (Windows) tests under Wine, if installed on the system. USE_WINE can be either set to the name of the desired wine command or the absolute path to the wine binary you wish to use for benchmarking.</p>
<p>This module utilizes the following environment variables: USE_WINE.</p>
<h2>System Event Watchdog</h2>
<p>This module has support for stopping/interrupting tests if various system issues occur, like a temperature sensor exceeds a defined threshold.</p>
<p>This module utilizes the following environment variables: WATCHDOG_SENSOR, WATCHDOG_SENSOR_THRESHOLD, WATCHDOG_MAXIMUM_WAIT.</p>
<hr style="height: 50px; border: 0;"><a name="InstallationInstructions"></a><h1>Installation Instructions</h1>
<h1>Setup Overview</h1>
<p>The Phoronix Test Suite supports Linux, Apple macOS, Microsoft Windows, Solaris, Hurd, BSD, and other operating system environments. The only Linux distribution-specific code deals with the external dependencies support feature that are set by individual test profiles. If you are not running one of the supported Linux distributions, Solaris, BSD, or macOS, you may need to install a package manually (as instructed by the Phoronix Test Suite) in order for a test to run. An example of an external dependency would be GCC and the OpenGL Utility Toolkit being needed for test profiles that build an OpenGL benchmark from source-code.</p>
<p>Among the distributions where the Phoronix Test Suite has been officially tested include Ubuntu, Fedora, Mandriva / Mageia, Gentoo, PCLinuxOS, Arch Linux, Pardus, OpenSuSE, Optware, webOS, Zenwalk, CentOS, Red Hat Enterprise Linux, Oracle Linux, Scientific Linux, Debian, Mint, Alpine Linux, Void Linux, Intel Clear Linux, and Amazon Linux EC2.</p>
<p>Among the tested BSD distributions are FreeBSD, NetBSD, OpenBSD, and DragonflyBSD. Tested Solaris distributions include Oracle Solaris 11, OpenIndiana, and Illumos.</p>
<h1>Dependencies</h1>
<p>The only required dependency for the Phoronix Test Suite is PHP 5.3 or newer. On Linux distributions, the needed package is commonly called <em>php5-cli</em> or <em>php-cli</em> or <em>php7</em> or <em>php</em>. It is important to note that only PHP for the command-line is needed and not a web server (Apache) or other packages commonly associated with PHP and its usage by web-sites. The PHP5 version required is PHP 5.3+ and can also be found at <a href="http://www.php.net/">www.php.net</a>. PHP 7 and PHP 8 are also fully supported by the Phoronix Test Suite.</p>
<p>The <em>phoronix-test-suite.bat</em> Windows launcher for the Phoronix Test Suite will automatically download and setup PHP on the local system if PHP is not present already.</p>
<p>The Phoronix Test Suite does not need to be installed system-wide but can simply be run from the extracted phoronix-test-suite folder as the local user.</p>
<p>As part of the PHP requirement, the following PHP extensions are required and/or highly recommended in order to take advantage of the Phoronix Test Suite capabilities:</p>
<ul>
<li><strong>PHP DOM</strong> is needed for XML operations and must be installed for the Phoronix Test Suite to function.</li>
<li><strong>PHP ZIP</strong> is needed for file compression and decompression and specifically dealing with test profiles and suites obtained via OpenBenchmarking.org or when uploading such tests and suites.</li>
<li><strong>PHP OpenSSL</strong> is used for enabling HTTPS communication with Phoronix Test Suite / OpenBenchmarking.org servers.</li>
<li><strong>PHP GD</strong> is highly recommended for rendering of test data to JPEG and PNG image formats and is selectively used in other image operations.</li>
<li><strong>PHP Zlib</strong> is highly recommended for greater data compression when dealing with remote OpenBenchmarking.org assets.</li>
<li><strong>PHP PCNTL</strong> is used for multi-threaded system sensor monitoring support during the testing process and other threaded tasks by the Phoronix Test Suite module framework.</li>
<li><strong>PHP POSIX</strong> is used for reliably obtaining more system information in an efficient manner.</li>
<li><strong>PHP CURL</strong> is supported as an alternative networking library for improved network performance in downloading test files and other operations.</li>
<li><strong>PHP FPDF</strong> is used to generate PDF reports of test data.</li>
</ul>
<p>Without all of these extensions, some capabilities of the Phoronix Test Suite will not be available. Many of these packages are enabled by default and do not require any additional installation steps on most Linux distributions, otherwise they are often found in the package vendor's repository.</p>
<h1>Notes</h1>
<h2>General</h2>
<p>You may need to modify the <em>php.ini</em> file on your system in order to support uploading results to OpenBenchmarking.org or logging into your OpenBenchmarking.org account. The <em>allow_url_fopen</em>, <em>file_uploads</em>, and <em>allow_url_include</em> options must be set to true in the PHP configuration.</p>
<p>Major updates to the Phoronix Test Suite are released on a quarterly basis. The latest stable and development versions of the Phoronix Test Suite are available at <a href="http://www.phoronix-test-suite.com/">Phoronix-Test-Suite.com</a>. The Git repository where the latest Phoronix Test Suite code is provided is hosted at <a href="https://github.com/phoronix-test-suite/phoronix-test-suite">github.com/phoronix-test-suite</a> and can be cloned/pulled from the <em>https://github.com/phoronix-test-suite/phoronix-test-suite.git</em> repository location. The latest upstream development code is housed in the master tree while older Phoronix Test Suite releases are available in their respective Git branches based upon the release's code-name.</p>
<p>If building the PHP package from upstream sources, it should just be a matter of running <em>./configure</em> with the <em>--enable-zip</em> flag (all other requirements should be apart of the stock PHP configuration) to satisfy the PHP needs of the Phoronix Test Suite.</p>
<h2>File Structure</h2>
<p>If manually changing the location of the <em>phoronix-test-suite</em> launcher file, the <em>PTS_USER_PATH</em> environment variable must be adjusted inside the file to reflect the absolute location that leads to the root directory of the <em>pts</em> and <em>pts-core</em> folders. The <em>pts-core</em> directory contains the "engine" of the Phoronix Test Suite.</p>
<h2>Running Locally</h2>
<p>The Phoronix Test Suite can be simply extracted from the downloaded <em>.tar.gz</em> or <em>.zip</em> file or it can also be installed system-wide. If you just wish to run the Phoronix Test Suite without installing it, open a terminal and run <em>./phoronix-test-suite <options></em> from the same directory.</p>
<h2>Generic Installation</h2>
<p>Running <em>install-sh</em> from the root directory of the Phoronix Test Suite will install the software for system-wide access. By default the <em>phoronix-test-suite</em> executable is in <em>/usr/bin/</em>, the Phoronix Test Suite files in <em>/usr/share/phoronix-test-suite/</em>, and the documentation in /<em>usr/share/doc/phoronix-test-suite/</em>. Root access is required. The default installation prefix is /usr/ but can be adjusted as the first argument (example: <em>install-sh /home/user/</em> to install the Phoronix Test Suite in your home directory).</p>
<h2>Debian/Ubuntu Installation</h2>
<p>Debian/Ubuntu users are able to follow the Generic Installation instructions or can obtain a Debian Package from the Phoronix Test Suite web-site. The package contains the <em>phoronix-test-suite</em> executable in <em>/usr/bin/</em>, the Phoronix Test Suite files in <em>/usr/share/phoronix-test-suite/</em>, and the documentation in <em>/usr/share/doc/phoronix-test-suite/</em>.</p>
<h2>Fedora / Red Hat Installation</h2>
<p>The Phoronix Test Suite can be installed on Fedora, Red Hat Enterprise Linux, and CentOS systems using the generic installation method. Alternatively, a <em>phoronix-test-suite</em> package is available in recent versions of the Fedora repository and in the EPEL (Extra Packages for Enterprise Linux) repository for Red Hat Enterprise Linux. However, at times this package may be out-of-date compared to upstream stable.</p>
<h2>BSD Installation</h2>
<p>The Phoronix Test Suite also supports *BSD operating systems. However, like the Solaris support, not all test profiles are compatible with BSD operating systems, but should run well on the likes of FreeBSD and DragonFlyBSD.</p>
<h2>MacOS Installation</h2>
<p>The Phoronix Test Suite is fully supported on Apple's macOS operating system. PHP ships with macOS by default on macOS 12 and older so it's simply a matter of downloading the Phoronix Test Suite package, extracting it, and running the executable. For tests that rely upon a compiler, Apple's XCode with GCC and LLVM can be utilized. On newer versions of macOS not shipping with PHP by default, <a href="https://brew.sh/">Homebrew</a> can be used for installing PHP or building PHP from source. The Phoronix Test Suite also supports making use of Homebrew for acquiring necessary Phoronix Test Suite dependencies on macOS.</p>
<hr style="height: 50px; border: 0;"><a name="PhoronixTestSuiteOnWindows"></a><h1>Phoronix Test Suite On Windows</h1>
<h2>Introduction</h2>
<p>Phoronix Test Suite 8.0 introduced rewritten Windows support that is at near feature parity to the program's long-standing support for Linux, macOS, and BSD operating systems.</p>
<p>The Phoronix Test Suite Windows support currently targets <strong>Windows 10 x64</strong>, <strong>Windows 11 x64</strong> and <strong>Windows Server 2016 x64</strong> and later. Earlier versions of Windows, namely Windows Server 2012 and Windows 8, may work to some extent but some hardware/software reporting features and other capabilities may be missing or report warning messages. The Phoronix Test Suite Windows support is also exclusively focused on x86 64-bit support: the Phoronix Test Suite itself will run on x86 32-bit but many of the program dependencies are configured for making use of 64-bit binaries.</p>
<h2>Windows Setup / Dependencies</h2>
<p>As with Phoronix Test Suite on Linux and other operating systems, the principal dependency is on PHP. Running the <em>phoronix-test-suite.bat</em> file launcher for the Phoronix Test Suite on Windows will attempt to download and setup PHP on the system under <em>C:\PHP</em> as the default location should PHP support not be found within your system's <em>Program Files</em> directories. The PHP Windows build does depend upon Microsoft Visual C++ redistributable libraries, which the Windows launcher will also attempt to download and install if needed.</p>
<p>The Phoronix Test Suite on Windows does depend upon <a href="https://www.cygwin.com/">Cygwin</a> for its Bash interpreter and other basic utilities to ease the process of porting test profiles to Windows with being able to use many of the same test installation scripts on Windows/Linux/macOS/BSD/Solaris then largely unmodified. Most of the Windows tests depend upon their respective native Windows applications/binaries while this Cygwin support is a convenience for handling these Bash setup scripts and also some test profiles that depend upon a GNU toolchain. The Phoronix Test Suite will attempt to download and setup Cygwin on the system if Cygwin isn't found in its default location of <em>C:\cygwin64</em>.</p>
<p>Various test profiles may depend upon other "external dependencies" like Python, PERL, Steam, and Java, as examples. The Phoronix Test Suite as with its support for other operating systems and Linux distributions will attempt to install these needed dependencies on a per-test basis when needed if existing support is not detected on the system.</p>
<h2>Running The Phoronix Test Suite On Windows</h2>
<p>The Phoronix Test Suite can run from its local directory and does not need to be "installed" to a system path or any other "setup" process prior to execution. On a clean install of Windows or Windows Server, deploying the Phoronix Test Suite is designed to be as easy and straight-forward as possible:</p>
<p>1. Download the Phoronix Test Suite from <a href="https://github.com/phoronix-test-suite/phoronix-test-suite">Phoronix-Test-Suite on GitHub</a> (<a href="https://github.com/phoronix-test-suite/phoronix-test-suite/archive/master.zip">zip file</a>).</p>
<p>2. From the Command Prompt or PowerShell, enter the <em>phoronix-test-suite</em> directory whether it be from Git or a zipped download.</p>
<p>3. Run the <em>phoronix-test-suite.bat</em> file that should proceed to run the Phoronix Test Suite just as you would on any other operating system. If needed the Phoronix Test Suite will try to initially download and setup PHP if needed followed by the attempted automatic Cygwin setup, etc.</p>
<p>4. Any of the Phoronix Test Suite commands from other operating systems should work on Windows. If you are new to the Phoronix Test Suite, you may enjoy a bit more guided experience by running the <strong>phoronix-test-suite shell</strong> command.</p>
<h2>Test Profiles On Windows</h2>
<p>As of 2021, around 100 test profiles are currently compatible with the Phoronix Test Suite on Windows. This includes many of the popular benchmarks and other interesting test cases. Over time more test profiles will continue to be ported to Windows where applicable and there are also some Windows-only tests also supported for execution by the Phoronix Test Suite.</p>
<h2>Getting Started</h2>
<p>Besides <strong>phoronix-test-suite shell</strong> and <strong>phoronix-test-suite help</strong>, there is also <strong>phoronix-test-suite interactive</strong> for helping new users understand Phoronix Test Suite benchmarking. Long story short, it should be as easy as running <strong>phoronix-test-suite benchmark stockfish</strong> or <strong>phoronix-test-suite benchmark crafty</strong> as some examples for carrying out automated, cross-platform benchmarks in a side-by-side and fully-reproducible manner.</p>
<hr style="height: 50px; border: 0;"><a name="ExternalDependencies"></a><h1>External Dependencies</h1>
<p>The Phoronix Test Suite has a feature known as "External Dependencies" where the Phoronix Test Suite can attempt to automatically install some of the test-specific dependencies on supported distributions. If running on a distribution where there is currently no External Dependencies profile, the needed package name(s) are listed for manual installation.</p>
<p>Below are a list of the operating systems that currently have external dependencies support within the Phoronix Test Suite for the automatic installation of needed test files.</p><hr><ul></ul><li>Alpine Linux</li><li>Amazon</li><li>Angstrom</li><li>Arch Linux</li><li>Clear Linux</li><li>ClearOS</li><li>ClearOS Core Server</li><li>Debian</li><li>DragonFlyBSD</li><li>Fedora</li><li>Gentoo</li><li>Linux Embedded Development Environment</li><li>Linux Mint</li><li>Mac OS X</li><li>MacPorts</li><li>Mageia</li><li>Mandriva</li><li>Microsoft Windows</li><li>MidnightBSD</li><li>NetBSD</li><li>OpenBSD</li><li>OpenIndiana</li><li>OpenMandriva</li><li>OpenMandrivaLinux</li><li>OpenSolaris</li><li>OpenSuSE</li><li>Optware</li><li>Oracle Server</li><li>PCLinuxOS</li><li>Pardus Linux</li><li>Red Hat Enterprise</li><li>Red Hat Enterprise Server</li><li>SUSE</li><li>SUSE Linux</li><li>Scientific</li><li>ScientificSL</li><li>Solus</li><li>Solus Linux</li><li>Termux</li><li>Ubuntu</li><li>Void Linux</li><li>Zenwalk</li><li>macOS Brew</li>
<hr style="height: 50px; border: 0;"><a name="Configuration"></a><h1>Configuration</h1>
<h1>User Files & Folders</h1>
<p>These files/folders are the default locations when running as a non-root Phoronix Test Suite user. When running as root, the paths may appear in standard system paths like <em>/etc/phoronix-test-suite.xml</em>.</p>
<p><strong>~/.phoronix-test-suite/user-config.xml</strong></p>
<p>This is a per-user configuration file. Among the information stored here is the test options, locations for storing files, and batch mode options. This file is formatted in XML. When run as root, this path is <em>/etc/phoronix-test-suite.xml</em>.</p>
<p><strong>~/.phoronix-test-suite/graph-config.json</strong></p>
<p>This is a per-user configuration file for storing graph attributes. The adjustable options include HTML hex color codes for different areas of the graph, dimensions of the graph, and font sizes. This file is formatted in JSON.</p>
<p><strong>~/.phoronix-test-suite/download-cache/</strong></p>
<p>This directory contains test packages that have been downloaded for test profiles. For more information on the download cache.</p>
<p><strong>~/.phoronix-test-suite/installed-tests/</strong></p>
<p>This directory is where tests are installed by default. Each test has its own directory within a sub-directory of <em>installed-tests/</em> based upon its OpenBenchmarking.org repository. In the test's folder is a <em>pts-install.json</em> file used for managing the installation.</p>
<p><strong>~/.phoronix-test-suite/test-results/</strong></p>
<p>This directory is where tests results are saved by default. Each saved file has its own directory. In the saved directory is then a <em>composite.xml</em> file containing the useful results while in the <em>test-X.xml</em> files are back-ups of the results.</p>
<p><strong>~/.phoronix-test-suite/modules-data/</strong></p>
<p>This is the directory where any Phoronix Test Suite modules should save any files to, within a sub-directory of the module's name. The module configuration settings are also stored within this directory.</p>
<p><strong>~/.phoronix-test-suite/test-profiles/</strong></p>
<p>This is the directory where test profiles are stored.</p>
<p><strong>~/.phoronix-test-suite/test-suites/</strong></p>
<p>This is the directory where test suites are stored.</p>
<hr style="height: 50px; border: 0;"><a name="MainConfigurationFile"></a><h1>Main Configuration File</h1>
<p>The main configuration file is <em>user-config.xml</em> (located at <em>~/.phoronix-test-suite/user-config.xml</em> or <em>/etc/phoronix-test-suite.xml</em> when running as root/admin) contains the user configuration options for the Phoronix Test Suite. To edit any option, open the configuration file within your preferred text editor. Alternatively, you can use the <em>user-config-set</em> option with the Phoronix Test Suite to update settings. For example, to set the download cache with the Phoronix Test Suite, execute <em>phoronix-test-suite user-config-set CacheDirectory=~/cache-directory/</em>.</p>
<h1>OpenBenchmarking Options</h1>
<h3>AnonymousUsageReporting</h3>
<p>If this option is set to <em>TRUE</em>, anonymous usage information and statistics, like the tests that are run and their length of run, will be reported to <a href="http://www.openbenchmarking.org/">OpenBenchmarking.org</a> for analytical reasons. All submitted information is kept anonymous. For more information on the anonymous usage reporting, read the Phoronix Test Suite documentation.</p>
<h3>IndexCacheTTL</h3>
<p>The time to live for OpenBenchmarking.org index caches. This is an integer representing the number of days before an index cache should be automatically refreshed from OpenBenchmarking.org. The default value is <em>3</em> while setting the value to <em>0</em> will disable automatic refreshing of caches (caches can be manually updated at anytime using the respective command).</p>
<h3>AlwaysUploadSystemLogs</h3>
<p>If this option is set to <em>TRUE</em>, the system logs (i.e. dmesg, lspci, lsusb, Xorg.0.log) will always be uploaded to OpenBenchmarking.org when uploading your test results. Otherwise the user is prompted whether to attach the system logs with their results.</p>
<h3>AllowResultUploadsToOpenBenchmarking</h3>
<p>This option defines whether to allow/support result uploads to OpenBenchmarking.org. If set to <em>FALSE</em>, the user will not be prompted to allow uploading of test results to the public site.</p>
<h1>General Options</h1>
<h3>DefaultBrowser</h3>
<p>The Phoronix Test Suite will automatically attempt to launch the system's default web browser when needed. This is done first by checking for x-www-browser and then xdg-open. If neither command is available, the Phoronix Test Suite will fallback to checking for Firefox, Epiphany, Mozilla, or the open command. If you wish to override the default browser that the Phoronix Test Suite selects, set this tag to the command name of the browser you wish to use. Leaving this tag empty will have the Phoronix Test Suite determine the default web browser.</p>
<h3>UsePhodeviCache</h3>
<p>If this option is set to <em>TRUE</em>, the Phoronix Test Suite will use the Phodevi smart cache (if available). The Phodevi smart cache will automatically cache relevant system hardware/software attributes that can be safely stored and will be used until the system's software/hardware has changed or the system rebooted. Enabling this option will speed up the detection of installed hardware and software through the Phoronix Test Suite. If this option is set to <em>FALSE</em>, Phodevi will not generate a smart cache. The default value is <em>TRUE</em>.</p>
<h3>DefaultDisplayMode</h3>
<p>This option affects how text is displayed on the command-line interface during the testing process. If this option is set to <em>DEFAULT</em>, the text interface will be the traditional Phoronix Test Suite output. If this option is set to <em>CONCISE</em>, the display mode is shorter and more concise. This is the default mode used during batch testing. The default value is <em>DEFAULT</em>.</p>
<h3>PhoromaticServers</h3>
<p>This option can be used to specify the IP address(es) and port(s) of any Phoromatic Servers you wish to connect to for obtaining cached data, connecting to Phoromatic as a client test system, etc. The Phoronix Test Suite will attempt zero-conf network discovery but if that fails you can add the <em>IP:port</em> (the Phoromatic Server's HTTP port) to this element for targeted probing by the Phoronix Test Suite. Multiple Phoromatic Servers can be added if delimited by a comma; e.g. <em>IP:port,IP:port, IP:port</em>.</p>
<h1>Modules Options</h1>
<h3>AutoLoadModules</h3>
<p>This tag contains a string of the names of the Phoronix Test Suite modules to load by default when running the Phoronix Test Suite. Multiple modules can be listed when delimited by a comma. Modules that load via setting an environment variable can also be specified here (i.e. <em>FORCE_AA=8</em> as an option in this string to load the <em>graphics_override</em> module with the 8x forced anti-aliasing). The default value is <em>toggle_screensaver, update_checker</em>.</p>
<h1>Installation Options</h1>
<h3>RemoveDownloadFiles</h3>
<p>If this option is set to <em>TRUE</em>, once a test has been installed the downloaded files will be removed. Enabling this option will conserve disk space and in nearly all circumstances will not result in any problems. However, if a test profile directly depends upon a file that was downloaded (as opposed to something extracted from a downloaded file during the installation process), enabling this option will cause issues. If this option is set to <em>FALSE</em>, the downloaded files will not be removed unless the test is uninstalled. The default value is <em>FALSE</em>.</p>
<h3>SearchMediaForCache</h3>
<p>If this option is set to <em>TRUE</em>, when installing a test it will automatically look for a Phoronix Test Suite download cache on removable media that is attached and mounted on the system. On the Linux operating system, the Phoronix Test Suite looks for devices mounted within the <em>/media/</em> or <em>/Volumes/</em> directories. If a download cache is found (a <em>download-cache/</em> folder within the drive's root directory) and a file it is looking for with matching MD5/SHA256 check-sum, the file will be automatically copied. Otherwise the standard download cache is checked. If this option is set to <em>FALSE</em>, removable media devices are not checked. The default value is <em>TRUE</em>.</p>
<h3>SymLinkFilesFromCache</h3>
<p>If this option is set to <em>TRUE</em>, during the test installation process when a file is found in a Phoronix Test Suite download cache, instead of copying the file just provide a symbolic link to the file. Enabling this option will conserve disk space and in nearly all circumstances will not result in any issues, permitting the download cache files are always mounted during testing and are not located on removable media. If this option is set to <em>FALSE</em>, the files will be copied from the download cache. The default value is <em>FALSE</em>.</p>
<h3>PromptForDownloadMirror</h3>
<p>If this option is set to <em>TRUE</em>, when downloading a test file the user will be prompted to select a mirror when multiple mirrors available. This option is targeted for those in remote regions or where their download speed may be greatly affected depending upon the server. If this option is set to <em>FALSE</em>, the Phoronix Test Suite will randomly pick a mirror. The default value is <em>FALSE</em>.</p>
<h3>EnvironmentDirectory</h3>
<p>This option sets the directory where tests will be installed to by the Phoronix Test Suite. The full path to the directory on the local file-system should be specified, though <em>~</em> is a valid character for denoting the user's home directory. The default value is <em>~/.phoronix-test-suite/installed-tests/</em>.</p>
<h3>CacheDirectory</h3>
<p>This option sets the directory for the main download cache. The download cache is checked when installing a test while attempting to locate a needed test file. If the file is found in the download cache, it will not be downloaded from there instead of an Internet mirror. When running <em>phoronix-test-suite make-download-cache</em>, files are automatically copied to this directory. The full path to the directory should be specified, though <em>~</em> is a valid character for denoting the user's home directory. Specifying an HTTP or FTP URL is valid. The default value is <em>~/.phoronix-test-suite/download-cache/</em>. Multiple cache directories can be specified as of Phoronix Test Suite 2.2 with each directory being delimited by a colon.</p>
<h1>Testing Options</h1>
<h3>SleepTimeBetweenTests</h3>
<p>This option sets the time (in seconds) to sleep between running tests. The default value is <em>8</em>.</p>
<h3>SaveSystemLogs</h3>
<p>If this option is set to <em>TRUE</em>, when saving the results from a test it will also save various system details and logs to a sub-directory of the result file's location. Among the logs that will be archived include the X.Org log, dmesg, and lspci outputs. These system details may also be saved if a test suite explicitly requests this information be saved. If this option is set to <em>FALSE</em>, the system details / logs will not be saved by default. The default value is <em>FALSE</em>. When running in batch mode or using a Phoronix Certification and Qualification Suite, the logs will be saved regardless of this user setting.</p>
<h3>SaveInstallationLogs</h3>
<p>If this option is set to <em>TRUE</em>, when saving the results from a test it will archive the complete output generated by the test during its earlier installation process. The log(s) are then saved to a sub-directory of the result file's location. If this option is set to <em>FALSE</em>, the full test logs will not be saved. The default value is <em>FALSE</em>. When running in batch mode or using a Phoronix Certification and Qualification Suite, the logs will be saved regardless of this user setting.</p>
<h3>RemoveTestInstallOnCompletion</h3>
<p>If this option is set to <em>TRUE</em>, after a test has been completed, if that test profile is no longer present later in the test queue, the test installation will be removed from the disk. If the test is to be run at a later time, it will need to be re-installed. This is useful for embedded environments or Live CD/DVDs where the available memory (RAM) for storage may be limited.</p>
<h3>SaveTestLogs</h3>
<p>If this option is set to <em>TRUE</em>, when saving the results from a test it will archive the complete output of each test's run generated by the application under test itself. The default value is <em>FALSE</em>.</p>
<h3>ResultsDirectory</h3>
<p>This option sets the directory where test results will be saved by the Phoronix Test Suite. The full path to the directory on the local file-system should be specified, though <em>~</em> is a valid character for denoting the user's home directory. The default value is <em>~/.phoronix-test-suite/test-results/</em>.</p>
<h3>AlwaysUploadResultsToOpenBenchmarking</h3>
<p>This option defines whether test results should always be uploaded to OpenBenchmarking.org upon their completion. If this value is set to <em>FALSE</em>, the user will be prompted each time whether the results should be uploaded to OpenBenchmarking.org, unless running in batch mode where the value is pre-defined. The default value is <em>FALSE</em>.</p>
<h3>AutoSortRunQueue</h3>
<p>This option defines whether the Phoronix Test Suite should sort the queue of tests to run based upon their title and category of tests. If <em>FALSE</em>, the run queue won't be sorted and they will be run in the order they were added.</p>
<h3>ShowPostRunStatistics</h3>
<p>If <em>TRUE</em>, the Phoronix Test Suite will show various test run statistics / comparison data based upon the test results / result file being tested after the testing has finished.</p>
<h1>TestResultValidation Options</h1>
<h3>DynamicRunCount</h3>
<p>If this option is set to <em>TRUE</em>, the Phoronix Test Suite will automatically increase the number of times a test is to be run if the standard deviation of the test results exceeds a predefined threshold. This option is set to <em>TRUE</em> by default and is designed to ensure the statistical signifiance of the test results. The run count will increase until the standard deviation falls below the threshold or when the total number of run counts exceeds twice the amount that is set to run by default from the given test profile. Under certain conditions the run count may also increase further.</p>
<h3>LimitDynamicToTestLength</h3>
<p>If <em>DynamicRunCount</em> is set to <em>TRUE</em>, this option sets a limit on the maximum length per trial run that a test can execute (in minutes) for the run count to be adjusted. This option is to prevent tests that take a very long amount of time to run from consuming too much time. By default this value is set to <em>20</em> minutes.</p>
<h3>StandardDeviationThreshold</h3>
<p>This option defines the overall standard deviation threshold (as a percent) for the Phoronix Test Suite to dynamically increase the run count of a test if this limit is exceeded. The default value is <em>3.50</em>.</p>
<h3>ExportResultsTo</h3>
<p>This option can specify a file (either the absolute path or relative if contained within <em>~/.phoronix-test-suite/</em> where a set of test results will be passed as the first argument as a string with each of the test results being delimited by a colon. If the executed script returns an exit status of <em>0</em> the results are considered valid, if the script returns an exit status of <em>1</em> the Phoronix Test Suite will request the test be run again.</p>
<h1>ResultViewer Options</h1>
<h3>WebPort</h3>
<p>The default HTTP web port to use for launching the web-based result viewer. If the value is set to <em>RANDOM</em>, a random open web port will be used.</p>
<h3>LimitAccessToLocalHost</h3>
<p>If this value is set to <em>TRUE</em> (default), the web-based result viewer is only accessible by the local host. If the value is <em>FALSE</em>, anyone with access to the IP/port can access the result viewer.</p>
<h3>AccessKey</h3>
<p>An access key / password can be optionally supplied as a basic precaution particularly for web-accessible result viewers that aren't limited to the local host. Set the string value here of the desired key/password that the user will be prompted to enter when trying to access the result viewer.</p>
<h3>AllowSavingResultChanges</h3>
<p>This allows saving result file changes (notes, modifying result files, etc) of result files from the web-based result viewer. Besides needing to be set to <em>TRUE</em>, the result file directory must also be write-enabled.</p>
<h3>AllowDeletingResults</h3>
<p>This option is similar to <em>AllowSavingResultChanges</em> but controls the behavior of whether results can be permanently removed. Besides needing to be set to <em>TRUE</em>, the result file directory must also be write-enabled.</p>
<h1>Batch Mode Options</h1>
<p>The batch mode options are only used when using either the <em>batch-run</em> or <em>batch-benchmark</em> options with the Phoronix Test Suite. This mode is designed to fully automate the operation of the Phoronix Test Suite except for areas where the user would like to be prompted. To configure the batch mode options, it is recommended to run <em>phoronix-test-suite batch-setup</em> instead of modifying these values by hand.</p>
<h3>SaveResults</h3>
<p>If this option is set to <em>TRUE</em>, when running in batch mode the test results will be automatically saved.</p>
<h3>OpenBrowser</h3>
<p>If this option is set to <em>TRUE</em>, when running in batch mode the web-browser will automatically open when displaying test results. If this option is set to <em>FALSE</em>, the web-browser will not be opened.</p>
<h3>UploadResults</h3>
<p>If this option is set to <em>TRUE</em>, when running in batch mode the test results will be automatically uploaded to <a href="http://www.openbenchmarking.org/">OpenBenchmarking.org</a>.</p>
<h3>PromptForTestIdentifier</h3>
<p>If this option is set to <em>TRUE</em>, when running in batch mode the user will be prompted to enter a test identifier. If this option is set to <em>FALSE</em>, a test identifier will be automatically generated.</p>
<h3>PromptForTestDescription</h3>
<p>If this option is set to <em>TRUE</em>, when running in batch mode the user will be prompted to enter a test description. If this option is set to <em>FALSE</em>, the default test description will be used.</p>
<h3>PromptSaveName</h3>
<p>If this option is set to <em>TRUE</em>, when running in batch mode the user will be prompted to enter a test name. If this option is set to <em>FALSE</em>, a test name will be automatically generated.</p>
<h1>Networking Options</h1>
<h3>NoInternetCommunication</h3>
<p>If you wish to disable Internet communication within the Phoronix Test Suite by default, set this option to <em>TRUE</em>. The default value is <em>FALSE</em>. Setting this to <em>FALSE</em> will still allow Phoromatic to communicate with network servers such as for intranet-based download caches or a Phoromatic Server. Internet support is generally required for downloading test profiles from OpenBenchmarking.org, acquiring necessary test files from their respective sources, etc.</p>
<h3>NoNetworkCommunication</h3>
<p>If you wish to disable network support (including Internet access) entirely within the Phoronix Test Suite, set this option to <em>TRUE</em>. The default value is <em>FALSE</em>.</p>
<h3>Timeout</h3>
<p>This is the read timeout (in seconds) for network connections. The default value is <em>20</em>.</p>
<h3>ProxyAddress</h3>
<p>If you wish to use a HTTP proxy server to allow the Phoronix Test Suite to communicate with OpenBenchmarking.org and other web services, enter the IP address / server name of the proxy server in this tag. If the proxy address and port tags are left empty but the <em>http_proxy</em> environment variable is set, the Phoronix Test Suite will attempt to use that as the proxy information.</p>
<h3>ProxyPort</h3>
<p>If using a proxy server, enter the TCP port in this tag.</p>
<h1>Server Options</h1>
<h3>RemoteAccessPort</h3>
<p>If you wish to allow remote access to the built-in web-based interface to the Phoronix Test Suite when running its built-in web server, set the port number for remote access here. Port 80 is the common HTTP port but the Phoronix Test Suite web-interface can be easily set to other port numbers. If you do not wish to allow remote access, use the default value of <em>FALSE</em> or <em>-1</em>. If the value is set to <em>RANDOM</em>, a random port number will be chosen.</p>
<h3>Password</h3>
<p>If you wish to require a password when entering the web-based interface to the Phoronix Test Suite -- either locally or remotely -- specify the password here using the password's SHA256 sum as the value.</p>
<h3>WebSocketPort</h3>
<p>The default port to use when running a WebSocket server. If no port is assigned or <em>RANDOM</em> is set, a random port will be chosen.</p>
<h3>AdvertiseServiceZeroConf</h3>
<p>If this option is set to <em>TRUE</em> when starting a Phoromatic Server instance, the software will attempt to broadcast its service using zeroconf networking (Avahi on Linux assuming <em>avahi-publish</em> is present).</p>
<h3>AdvertiseServiceOpenBenchmarkRelay</h3>
<p>If this option is set to <em>TRUE</em> when starting a Phoromatic Server instance, the software will broadcast the local IP/port of the server to a private OpenBenchmarking.org service so that if any other user on the local IP block from the same global IP address is in search of a Phoromatic Server, the IP address will be relayed. This is an alternative or complementary to the zero-conf/Avahi option above to help systems running the Phoronix Test Suite client on a LAN discover a Phoromatic Server for easy setup and/or download cache support for faster test setup/installation.</p>
<h3>PhoromaticStorage</h3>
<p>The location for the Phoromatic Server to store test results of connected systems, account information, etc. The default location is <em>~/.phoronix-test-suite/phoromatic/</em>.</p>
<hr style="height: 50px; border: 0;"><a name="EnvironmentVariables"></a><h1>Environment Variables</h1>
<h2>DONT_BALANCE_TESTS_FOR_SUBSYSTEMS</h2>
<p><em>If this value is true, the Phoronix Test Suite stress-run manager will not attempt to distribute the selected test(s) among available hardware subsystems. For stress runs with tests covering multiple subsystems (e.g. CPU, GPU, RAM), the default behavior is try to ensure the tests to run concurrently are as balanced across the tested subsystems as possible.</em></p>
<p>The value can be of type: boolean (TRUE / FALSE).
The variable is relevant for: stress-run mode.
</p>
<h2>DONT_TRY_TO_ENSURE_TESTS_ARE_UNIQUE</h2>
<p><em>When running in the stress-run mode, the default behavior will try to ensure when tests are running concurrently that as many unique tests as possible are being run. Setting this value to try will avoid that check and just attempt to truly randomize the tests being run concurrently without regard for trying to avoid duplicates.</em></p>
<p>The value can be of type: boolean (TRUE / FALSE).
The variable is relevant for: stress-run mode.
</p>
<h2>FORCE_ABSOLUTE_MIN_TIMES_TO_RUN</h2>
<p><em>This option is similar to FORCE_MIN_TIMES_TO_RUN but is *absolute* in ensuring each test will run at least that number of times and not subject to change of any timed cut-offs or other factors.</em></p>
<p>The value can be of type: positive integer.
The variable is relevant for: test execution / benchmarking.
</p>
<h2>FORCE_MIN_DURATION_PER_TEST</h2>
<p><em>This option can be used to specify the minimum number of times to run a given benchmark. Rather than relying on a static times-to-run count, the test will keep looping until the time has exceeded this number (in minutes).</em></p>
<p>The value can be of type: positive integer.
The variable is relevant for: test execution / benchmarking.
</p>
<h2>FORCE_MIN_TIMES_TO_RUN</h2>
<p><em>This option is similar to FORCE_TIMES_TO_RUN but is used for specifying the minimum possible number of times to run. Unlike FORCE_TIMES_TO_RUN, the run count can still exceed this value if the deviation between results or other factors are too high.</em></p>
<p>The value can be of type: positive integer.
The variable is relevant for: test execution / benchmarking.
</p>
<h2>FORCE_MIN_TIMES_TO_RUN_CUTOFF</h2>
<p><em>Used in conjunction with the FORCE_MIN_TIMES_TO_RUN, the FORCE_MIN_TIMES_TO_RUN_CUTOFF can be used for specifyingg the amount of time (in minutes) before foregoing additional runs. This allows cutting off the testing early if this time threshold has been reached.</em></p>