Add CONFIDENCE_BASIS link for CLM #368

m-linner-ericsson · 2023-07-20T15:24:05Z

Applicable Issues

Fixes #323

Description of the Change

Added a CONFIDENCE_BASIS link with inspiration from IVs VERIFICATION_BASIS

Alternate Designs

N/A

Benefits

The user of Eiffel can now express in what CONTEXT they send the CLM event but also what basis they used for compiling the confidence.

Possible Drawbacks

None that I can think of

Sign-off

Developer's Certificate of Origin 1.1

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I
have the right to submit it under the open source license
indicated in the file; or

(b) The contribution is based upon previous work that, to the best
of my knowledge, is covered under an appropriate open source
license and I have the right under that license to submit that
work with modifications, whether created in whole or in part
by me, under the same open source license (unless I am
permitted to submit under a different license), as indicated
in the file; or

(c) The contribution was provided directly to me by some other
person who certified (a), (b) or (c) and I have not modified
it.

(d) I understand and agree that this project and the contribution
are public and that a record of the contribution (including all
personal information I submit with it, including my sign-off) is
maintained indefinitely and may be redistributed consistent with
this project or the open source license(s) involved.

Signed-off-by: Mattias Linnér mattias.linner@ericsson.com

m-linner-ericsson · 2023-07-20T15:25:33Z

Diff of YAML and JSON

(venv) emalinn@E-5CG1504MQG:~/github/forks/eiffel$ diff -u definitions/EiffelConfidenceLevelModifiedEvent/3.3.0.yml definitions/EiffelConfidenceLevelModifiedEvent/3.4.0.yml
--- definitions/EiffelConfidenceLevelModifiedEvent/3.3.0.yml    2023-07-20 11:19:01.198861585 +0200
+++ definitions/EiffelConfidenceLevelModifiedEvent/3.4.0.yml    2023-07-20 17:15:49.318468297 +0200
@@ -139,7 +139,20 @@
       any_type: false
       types:
         - EiffelConfidenceLevelModifiedEvent
+  CONFIDENCE_BASIS:
+    description: 'Used to declare the basis for which confidence statement(s) this event have used.
+      The __CAUSE__ link tells what caused the event sending whereas __CONFIDENCE_BASIS__ declares the reason for
+      selecting the provided `data.name` and/or `data.value` '
+    required: false
+    multiple: true
+    targets:
+      any_type: false
+      types:
+        - EiffelTestCaseTriggeredEvent
+        - EiffelTestSuiteStartedEvent
 _history:
+  - version: 3.4.0
+    changes: Add CONFIDENCE_BASIS link (see [Issue 323](https://github.com/eiffel-community/eiffel/issues/323)).
   - version: 3.3.0
     introduced_in: edition-orizaba
     changes: Add EiffelArtifactDeployedEvent as legal target type for
(venv) emalinn@E-5CG1504MQG:~/github/forks/eiffel$ diff -u schemas/EiffelConfidenceLevelModifiedEvent/3.3.0.json schemas/EiffelConfidenceLevelModifiedEvent/3.4.0.json
--- schemas/EiffelConfidenceLevelModifiedEvent/3.3.0.json       2023-07-20 17:16:28.688467573 +0200
+++ schemas/EiffelConfidenceLevelModifiedEvent/3.4.0.json       2023-07-20 17:16:28.748467572 +0200
@@ -18,9 +18,9 @@
         "version": {
           "type": "string",
           "enum": [
-            "3.3.0"
+            "3.4.0"
           ],
-          "default": "3.3.0"
+          "default": "3.4.0"
         },
         "time": {
           "type": "integer"

m-linner-ericsson · 2023-07-20T15:26:37Z

Setting to draft per #323 (comment)

t-persson · 2023-08-01T06:16:07Z

eiffel-vocabulary/EiffelConfidenceLevelModifiedEvent.md

@@ -84,6 +84,12 @@ __Legal targets:__ [EiffelConfidenceLevelModifiedEvent](../eiffel-vocabulary/Eif
 __Multiple allowed:__ Yes  
 __Description:__ Used in events summarizing multiple confidence levels. Example use case: the confidence level "allTestsOk" summarizes the confidence levels "unitTestsOk, "scenarioTestsOk" and "deploymentTestsOk", and consequently links to them via __SUB_CONFIDENCE_LEVEL__. This is intended for purely descriptive, rather than prescriptive, use.

+### CONFIDENCE_BASIS
+__Required:__ No  
+__Legal targets:__ [EiffelTestCaseTriggeredEvent](../eiffel-vocabulary/EiffelTestCaseTriggeredEvent.md), [EiffelTestSuiteStartedEvent](../eiffel-vocabulary/EiffelTestSuiteStartedEvent.md)  


Shouldn't we base our confidence on the actual results of the test case and/or test suite instead? I.e. Shouldn't legal targets be TestCaseFinished and TestSuiteFinished?

I'll take that question with me when we discuss the PR.

I have assumed that we always should point to the triggered event but let's hear what the others think.

What does "when we discuss the PR" mean? Will you bring it to a community meeting? If so, when? :)

We will discuss this on the 31 of August during the community meeting.

@t-persson In f3af483 I have changed to link to the finished events instead.

e-backmark-ericsson · 2023-08-31T13:05:04Z

Outcome from community meeting on Aug 31:

CONFIDENCE_BASIS shall point to TCF and TSF for protocol conformance among other reasons
Add CLM as a valid target for CONFIDENCE_BASIS
Add a statement on SUB_CONFIDENCE_LEVEL that it might be deprecated in favor of CONFIDENCE_BASIS
Send updated PR to Eiffel Community maillist to collect comments

* Changed the valid link to the finished events * Added CLM as valid link type * Added future deprication note for SUB_CONFIDENCE_LEVEL Signed-off-by: Mattias Linnér <mattias.linner@ericsson.com>

magnusbaeck · 2023-09-21T15:27:17Z

definitions/EiffelConfidenceLevelModifiedEvent/3.4.0.yml

+  CONFIDENCE_BASIS:
+    description: 'Used to declare the basis for which confidence statement(s) this event have used. 
+      The __CAUSE__ link tells what caused the event sending whereas __CONFIDENCE_BASIS__ declares the reason for
+      selecting the provided `data.name` and/or `data.value`.'
+    required: false
+    multiple: true
+    targets:
+      any_type: false
+      types:
+        - EiffelConfidenceLevelModifiedEvent
+        - EiffelTestCaseFinishedEvent
+        - EiffelTestSuiteFinishedEvent


Sorry for noticing this only now, but would you mind moving up this link type so that they're in alphabetical order? I suspect we're sorting them when the Markdown is rendered but having them alphabetical in the source file would make things more navigable.

Also, we generally use __fieldname__ to reference other field names, not backticks. Compare output of git grep '<backtick>data\.' -- definitions with git grep '__data\.' -- definitions (replace with an actual backtick obviously; I couldn't get the formatting right here).

No we don't sort it as far as I could see but both issues are fixed in ebf4416

Add CONFIDENCE_BASIS link

39cfb40

m-linner-ericsson requested a review from a team as a code owner July 20, 2023 15:24

m-linner-ericsson marked this pull request as draft July 20, 2023 15:25

m-linner-ericsson self-assigned this Jul 20, 2023

t-persson reviewed Aug 1, 2023

View reviewed changes

Updates after review

f3af483

* Changed the valid link to the finished events * Added CLM as valid link type * Added future deprication note for SUB_CONFIDENCE_LEVEL Signed-off-by: Mattias Linnér <mattias.linner@ericsson.com>

m-linner-ericsson marked this pull request as ready for review September 20, 2023 14:08

m-linner-ericsson requested a review from a team September 20, 2023 14:09

Remove trailing spaces

7dabfd4

e-backmark-ericsson previously approved these changes Sep 20, 2023

View reviewed changes

magnusbaeck requested changes Sep 21, 2023

View reviewed changes

Sort in alphabetical order and use bold instead of fixed font

ebf4416

m-linner-ericsson dismissed e-backmark-ericsson’s stale review via ebf4416 October 11, 2023 08:27

m-linner-ericsson requested review from magnusbaeck and e-backmark-ericsson October 11, 2023 08:30

e-backmark-ericsson approved these changes Oct 11, 2023

View reviewed changes

magnusbaeck approved these changes Oct 12, 2023

View reviewed changes

m-linner-ericsson merged commit d09c69f into eiffel-community:master Oct 12, 2023
2 checks passed

m-linner-ericsson deleted the bases_for_CLM branch October 12, 2023 17:13

magnusbaeck added the protocol All protocol changes label Oct 12, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add CONFIDENCE_BASIS link for CLM #368

Add CONFIDENCE_BASIS link for CLM #368

m-linner-ericsson commented Jul 20, 2023

m-linner-ericsson commented Jul 20, 2023

m-linner-ericsson commented Jul 20, 2023

t-persson Aug 1, 2023

m-linner-ericsson Aug 4, 2023

t-persson Aug 8, 2023 •

edited

m-linner-ericsson Aug 18, 2023

m-linner-ericsson Sep 20, 2023

e-backmark-ericsson commented Aug 31, 2023

magnusbaeck Sep 21, 2023

m-linner-ericsson Oct 11, 2023

Add CONFIDENCE_BASIS link for CLM #368

Add CONFIDENCE_BASIS link for CLM #368

Conversation

m-linner-ericsson commented Jul 20, 2023

Applicable Issues

Description of the Change

Alternate Designs

Benefits

Possible Drawbacks

Sign-off

m-linner-ericsson commented Jul 20, 2023

m-linner-ericsson commented Jul 20, 2023

t-persson Aug 1, 2023

Choose a reason for hiding this comment

m-linner-ericsson Aug 4, 2023

Choose a reason for hiding this comment

t-persson Aug 8, 2023 • edited

Choose a reason for hiding this comment

m-linner-ericsson Aug 18, 2023

Choose a reason for hiding this comment

m-linner-ericsson Sep 20, 2023

Choose a reason for hiding this comment

e-backmark-ericsson commented Aug 31, 2023

magnusbaeck Sep 21, 2023

Choose a reason for hiding this comment

m-linner-ericsson Oct 11, 2023

Choose a reason for hiding this comment

t-persson Aug 8, 2023 •

edited