Skip to content

Commit

Permalink
Add simple report generator (#3602)
Browse files Browse the repository at this point in the history
* Add simple report generator

This will report on the state of the ARCHIVE, BACKUP, and CACHE on-disk trees
in addition to the state of the SQL database. (I'm going to leave analyzing
and reporting on the Opensearch database for another time, since this is "off
books" weekend upstream work!)

This packages the ad hoc SQL queries I've been doing to monitor the server as
a CLI utility, plus some more.

Here's the output of `pbench-report-generator --all` on the production server:

```
Archive report:
  117,446 tarballs consuming 21.7 TB
  The smallest tarball is 1.0 kB, pbench-user-benchmark__2020.04.03T11.05.44
  The biggest tarball is 41.1 GB, uperf_Azure_RHEL-8.10.0-20240116.45_x86_64_gen2_pci_netvsc_quick_D240125T014727_2024.01.25T01.47.28
Backup report:
  117,447 tarballs consuming 21.7 TB
Cache report:
  97,464 datasets consuming 45.6 TB
  4 datasets have never been unpacked, 0 are missing reference timestamps, 0 have bad size metadata
  The smallest cache is 24.6 kB, pbench-user-benchmark__2020.04.03T11.05.44
  The biggest cache is 110.5 GB, trafficgen_RHOSP16.2-RHEL8.3-nrt-OVS-OFFLOAD-PVP-LossTests_tg:trex_r:none_fs:64,128,256,512,1024,1500_nf:1024_fm:si_td:bi_ml:0.002,0.0005,0.0001_tt:bs__2020-12-26T03:16:38
  The least recently used cache was referenced Dec 11, specjbb2005__2023.09.22T00.22.28
  The most recently used cache was referenced today, uperf_rhel84_4.18.0.277_kernel_10gb_jumbo_2021.01.26T09.51.18
SQL storage report:
  Table                Rows       Storage   
  -------------------- ---------- ----------
  alembic_version               1    57.3 kB
  audit                   683,922   224.7 MB
  datasets                117,449    34.3 MB
  templates                    12   221.2 kB
  server_settings               0    24.6 kB
  users                        11    81.9 kB
  dataset_metadata        352,344   217.9 MB
  dataset_operations      340,986    29.1 MB
  api_keys                      5    81.9 kB
  indexmaps               291,510    79.7 GB
Operational states:
  UPLOAD states:
          OK  117,449
  TOOLINDEX states:
       READY  106,112
  INDEX states:
          OK  106,112
      FAILED      494
           CODE  7:    365  Bad metadata.log file encountered
           CODE  1:    128  Operational error while indexing
           CODE 12:      1  Unexpected error encountered
       READY   10,819
```
  • Loading branch information
dbutenhof committed Jan 29, 2024
1 parent e4e3da3 commit 369736b
Show file tree
Hide file tree
Showing 4 changed files with 524 additions and 1 deletion.

0 comments on commit 369736b

Please sign in to comment.