AlertLogPkg: Enabling and Disabling Passed/Affirmations Checked

[JimLewis]

From an external user:
I am struggling with configuration procedure SetAlertLogOptions in conjunction with ReportAlerts. I am not sure, if there is a basic misunderstanding or a bug causing this observed behavior. In attached screenshot you will see the contradictions. My assumption was that “passed” and “affirmations” counts are skipped in report like “DisabledAlerts”:

With the settings:

ReportAlertLogOptions
-------------------------
. . . 
PrintPassedVar:            false
PrintAffirmationsVar:   false
...

Report Alerts produces:

%% DONE    FAILED   Report Summary   Total Error(s) = 3   Failures: 0  Errors: 3  Warnings: 0  Passed:  33  Affirmations Checked:  35 at 4355 ns

[JimLewis]

If we look at the code in ReportAlerts for this we see:

      if PrintPassedVar or (AffirmCheckCount /= 0) or PrintAffirmationsVar then -- Print if passed or printing affirmations
        write(buf, "  Passed: " & to_string(PassedCount)) ;
      end if;
      if (AffirmCheckCount /= 0) or PrintAffirmationsVar then
        write(buf, "  Affirmations Checked: " & to_string(AffirmCheckCount)) ;
      end if ;

Currently if there are Affirmations, then we print them. The assumption is that if you used affirmations, you see the value in printing them.

If there are not affirmations and PrintAffirmationsVar is set, then we print them - which prints 0.

The reason we want to print them when they are 0, is it is a measure of the test quality. When a 0 prints it indicates there was no self checking in the test - which then would mean we have a vacuous pass.

It is possible that we could re-think this some.

The other thing to note is that if you are using OSVVM scripts, you may want to switch to using EndOfTestReports. EndOfTestReports includes a call to ReportAlerts. It also contains other calls so that OSVVM can produce the HTML and XML (JUnit) test reports - as well as the Alert and Functional Coverage detailed reports.

The HTML reports always contain all of the information always, so not including it in the ReportAlerts report is not as big of concern as it was at the time when ReportAlerts was the only reporting mechanism.