Change IUT to Implementation Under Test

[m-linner-ericsson]

Applicable Issues

Fixes #359

Description of the Change

Changed Item Under Test to Implementation Under Test in

• EiffelIssueVerifiedEvent
• EiffelTestCaseFinishedEvent
• EiffelTestCaseTriggeredEvent
• EiffelTestExecutionRecipeCollectionCreatedEvent
• EiffelTestSuiteFinishedEvent

Alternate Designs

None

Benefits

As explained in #359 ETSI defines IUT as Implementation Under Test and we should use standard nomenclature.

Possible Drawbacks

None that I can think off

Sign-off

Developer's Certificate of Origin 1.1

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I
have the right to submit it under the open source license
indicated in the file; or

(b) The contribution is based upon previous work that, to the best
of my knowledge, is covered under an appropriate open source
license and I have the right under that license to submit that
work with modifications, whether created in whole or in part
by me, under the same open source license (unless I am
permitted to submit under a different license), as indicated
in the file; or

(c) The contribution was provided directly to me by some other
person who certified (a), (b) or (c) and I have not modified
it.

(d) I understand and agree that this project and the contribution
are public and that a record of the contribution (including all
personal information I submit with it, including my sign-off) is
maintained indefinitely and may be redistributed consistent with
this project or the open source license(s) involved.

Signed-off-by: Mattias Linnér mattias.liner@gmail.com

[m-linner-ericsson]

Diff of schemas and yaml

(venv) emalinn@E-5CG1504MQG:~/github/forks/eiffel$ diff -u definitions/EiffelIssueVerifiedEvent/4.3.0.yml definitions/EiffelIssueVerifiedEvent/4.3.1.yml
--- definitions/EiffelIssueVerifiedEvent/4.3.0.yml      2023-07-05 08:54:43.870319175 +0200
+++ definitions/EiffelIssueVerifiedEvent/4.3.1.yml      2023-07-14 14:34:31.357779128 +0200
@@ -100,7 +100,7 @@
       types:
         - EiffelIssueDefinedEvent
   IUT:
-    description: Identifies the Item Under Test; in other words, the
+    description: Identifies the Implementation Under Test; in other words, the
       entity for which the issue has been verified.
     required: true
     multiple: false
@@ -129,6 +129,8 @@
         - EiffelTestCaseFinishedEvent
         - EiffelTestSuiteFinishedEvent
 _history:
+  - version: 4.3.1
+    changes: Change IUT to Implementation Under Test (see [Issue 235](https://github.com/eiffel-community/eiffel/issues/359)).
   - version: 4.3.0
     introduced_in: edition-orizaba
     changes: Add artifact deployed event as legal IUT target (see [Issue 239](https://github.com/eiffel-community/eiffel/issues/239)).
(venv) emalinn@E-5CG1504MQG:~/github/forks/eiffel$ diff -u schemas/EiffelIssueVerifiedEvent/4.3.0.json schemas/EiffelIssueVerifiedEvent/4.3.1.json
--- schemas/EiffelIssueVerifiedEvent/4.3.0.json 2023-07-14 13:51:08.817763089 +0200
+++ schemas/EiffelIssueVerifiedEvent/4.3.1.json 2023-07-14 13:51:08.867763090 +0200
@@ -18,9 +18,9 @@
         "version": {
           "type": "string",
           "enum": [
-            "4.3.0"
+            "4.3.1"
           ],
-          "default": "4.3.0"
+          "default": "4.3.1"
         },
         "time": {
           "type": "integer"
(venv) emalinn@E-5CG1504MQG:~/github/forks/eiffel$ diff -u definitions/EiffelTestCaseFinishedEvent/3.3.0.yml definitions/EiffelTestCaseFinishedEvent/3.3.1.yml
--- definitions/EiffelTestCaseFinishedEvent/3.3.0.yml   2023-07-05 08:54:43.870319175 +0200
+++ definitions/EiffelTestCaseFinishedEvent/3.3.1.yml   2023-07-14 14:34:47.557779228 +0200
@@ -18,7 +18,7 @@
 _description: |-
   The EiffelTestCaseFinishedEvent declares that a previously started test case (declared by [EiffelTestCaseStartedEvent](./EiffelTestCaseStartedEvent.md)) has finished and reports the outcome.

-  Note that while similar, the __data.outcome__ object is different from that of [EiffelActivityFinishedEvent](./EiffelActivityFinishedEvent.md). The outcome of the test case reports not only the conclusion of the test case execution - whether the test case was successfully executed - but also passes a verdict on the item under test. To highlight this conceptual difference, both __data.outcome.verdict__ and __data.outcome.conclusion__ are included.
+  Note that while similar, the __data.outcome__ object is different from that of [EiffelActivityFinishedEvent](./EiffelActivityFinishedEvent.md). The outcome of the test case reports not only the conclusion of the test case execution - whether the test case was successfully executed - but also passes a verdict on the implementation under test. To highlight this conceptual difference, both __data.outcome.verdict__ and __data.outcome.conclusion__ are included.

   Also note that unlike [EiffelTestSuiteFinishedEvent](./EiffelTestSuiteFinishedEvent.md), EiffelTestCaseFinishedEvent must report both __data.outcome.verdict__ and __data.outcome.conclusion__.
 type: object
@@ -46,7 +46,7 @@
           conclusion:
             _description: |-
               A terse standardized conclusion of the test case, designed to be machine readable.
-              SUCCESSFUL signifies that the test case was successfully concluded. Note that this does not imply that the item under test passed the tests.
+              SUCCESSFUL signifies that the test case was successfully concluded. Note that this does not imply that the implementation under test passed the tests.
               FAILED signifies that the test case could not be successfully executed. To exemplify, one or more tests failed to run due to required environments being unavailable.
               ABORTED signifies that the test case was aborted before it could be concluded.
               TIMED_OUT signifies that the test case did not conclude within the allowed time frame.
@@ -182,6 +182,8 @@
       types:
         - EiffelTestCaseTriggeredEvent
 _history:
+  - version: 3.3.1
+    changes: Change IUT to Implementation Under Test (see [Issue 235](https://github.com/eiffel-community/eiffel/issues/359)).
   - version: 3.3.0
     introduced_in: edition-arica
     changes: Add schema URL to the meta object (see [Issue 280](https://github.com/eiffel-community/eiffel/issues/280)).
(venv) emalinn@E-5CG1504MQG:~/github/forks/eiffel$ diff -u schemas/EiffelTestCaseFinishedEvent/3.3.0.json schemas/EiffelTestCaseFinishedEvent/3.3.1.json
--- schemas/EiffelTestCaseFinishedEvent/3.3.0.json      2023-07-14 13:51:10.287763098 +0200
+++ schemas/EiffelTestCaseFinishedEvent/3.3.1.json      2023-07-14 13:51:10.337763099 +0200
@@ -18,9 +18,9 @@
         "version": {
           "type": "string",
           "enum": [
-            "3.3.0"
+            "3.3.1"
           ],
-          "default": "3.3.0"
+          "default": "3.3.1"
         },
         "time": {
           "type": "integer"
(venv) emalinn@E-5CG1504MQG:~/github/forks/eiffel$ diff -u definitions/EiffelTestCaseTriggeredEvent/3.5.0.yml definitions/EiffelTestCaseTriggeredEvent/3.5.1.yml
--- definitions/EiffelTestCaseTriggeredEvent/3.5.0.yml  2023-07-05 08:54:43.870319175 +0200
+++ definitions/EiffelTestCaseTriggeredEvent/3.5.1.yml  2023-07-14 14:36:22.237779811 +0200
@@ -162,7 +162,7 @@
       types:
         - EiffelFlowContextDefinedEvent
   IUT:
-    description: Identifies the Item Under Test; in other words, the
+    description: Identifies the Implementation Under Test; in other words, the
       entity that is about to be tested.
     required: true
     multiple: false
@@ -201,6 +201,8 @@
       types:
         - EiffelTestCaseTriggeredEvent
 _history:
+  - version: 3.5.1
+    changes: Change IUT to Implementation Under Test (see [Issue 235](https://github.com/eiffel-community/eiffel/issues/359)).
   - version: 3.5.0
     introduced_in: edition-orizaba
     changes: Add ORIGINAL_TRIGGER link (see [Issue 246](https://github.com/eiffel-community/eiffel/issues/246)).
(venv) emalinn@E-5CG1504MQG:~/github/forks/eiffel$ diff -u schemas/EiffelTestCaseTriggeredEvent/3.5.0.json schemas/EiffelTestCaseTriggeredEvent/3.5.1.json
--- schemas/EiffelTestCaseTriggeredEvent/3.5.0.json     2023-07-14 13:51:11.107763103 +0200
+++ schemas/EiffelTestCaseTriggeredEvent/3.5.1.json     2023-07-14 13:51:11.167763104 +0200
@@ -18,9 +18,9 @@
         "version": {
           "type": "string",
           "enum": [
-            "3.5.0"
+            "3.5.1"
           ],
-          "default": "3.5.0"
+          "default": "3.5.1"
         },
         "time": {
           "type": "integer"
(venv) emalinn@E-5CG1504MQG:~/github/forks/eiffel$ diff -u definitions/EiffelTestExecutionRecipeCollectionCreatedEvent/4.3.0.yml definitions/EiffelTestExecutionRecipeCollectionCreatedEvent/4.3.1.yml
 schema-2023-07-05 08:54:43.870319175 +0200ecipeCollectionCreatedEvent/4.3.0.yml
+++ definitions/EiffelTestExecutionRecipeCollectionCreatedEvent/4.3.1.yml       2023-07-14 14:35:07.187779349 +0200
@@ -24,7 +24,7 @@

   The __data__ object consists of two main parts. __data.selectionStrategy__ identifies the strategy used to select the test cases and generate the recipe collection, while __data.batches__ or __data.batchesUri__ contain or reference, respectively, the recipes. The recipes are grouped in batches, which are used to control the order of execution of test cases. Every batch has a priority to let the test executor order them in sequence, but within each batch no assumptions are made as to the execution order the test cases. This way the recipe collection can either allow the executor a high degree of freedom in scheduling the test executions, and/or prescribe the exact sequential order in which they must be executed. Each event SHALL include one and only one of __data.batches__ and __data.batchesUri__.

-  Finally, each recipe (__data.batches.recipes__) consists of two parts: the test case to execute, and the constraints of that execution. The EiffelTestExecutionRecipeCollectionCreatedEvent does not control the syntax of these constraints, as the nature of such constraints are highly dependent on technology domain and test execution framework. That being said, there are three questions that typically need to be answered: what is the item under test, in what kind of environment is it to be tested, and what are the test parameters? Note the distinction between test case and test execution: it is perfectly legal for a single test case to appear multiple times within the same EiffelTestExecutionRecipeCollectionCreatedEvent, but (presumably) with different constraints.
+  Finally, each recipe (__data.batches.recipes__) consists of two parts: the test case to execute, and the constraints of that execution. The EiffelTestExecutionRecipeCollectionCreatedEvent does not control the syntax of these constraints, as the nature of such constraints are highly dependent on technology domain and test execution framework. That being said, there are three questions that typically need to be answered: what is the implementation under test, in what kind of environment is it to be tested, and what are the test parameters? Note the distinction between test case and test execution: it is perfectly legal for a single test case to appear multiple times within the same EiffelTestExecutionRecipeCollectionCreatedEvent, but (presumably) with different constraints.
 type: object
 properties:
   meta:
@@ -134,7 +134,7 @@
                       constraints. Instead, this property is a list
                       of key-value pairs on the same format as [data.customData](../customization/custom-data.md).
                       That being said, there are three questions that
-                      typically need to be answered: what is the item
+                      typically need to be answered: what is the implementation
                       under test, in what kind of environment is it
                       to be tested, and what are the test parameters?'
                     type: array
@@ -237,6 +237,8 @@
       types:
         - EiffelFlowContextDefinedEvent
 _history:
+  - version: 4.3.1
+    changes: Change IUT to Implementation Under Test (see [Issue 235](https://github.com/eiffel-community/eiffel/issues/359)).
   - version: 4.3.0
     introduced_in: edition-arica
     changes: Add schema URL to the meta object (see [Issue 280](https://github.com/eiffel-community/eiffel/issues/280)).
(venv) emalinn@E-5CG1504MQG:~/github/forks/eiffel$ diff -u schemas/EiffelTestExecutionRecipeCollectionCreatedEvent/4.3.0.json schemas/EiffelTestExecutionRecipeCollectionCreatedEvent/4.3.1.json
--- schemas/EiffelTestExecutionRecipeCollectionCreatedEvent/4.3.0.json  2023-07-14 13:51:11.687763107 +0200
+++ schemas/EiffelTestExecutionRecipeCollectionCreatedEvent/4.3.1.json  2023-07-14 13:51:11.747763107 +0200
@@ -18,9 +18,9 @@
         "version": {
           "type": "string",
           "enum": [
-            "4.3.0"
+            "4.3.1"
           ],
-          "default": "4.3.0"
+          "default": "4.3.1"
         },
         "time": {
           "type": "integer"
(venv) emalinn@E-5CG1504MQG:~/github/forks/eiffel$ diff -u definitions/EiffelTestSuiteFinishedEvent/3.3.0.yml definitions/EiffelTestSuiteFinishedEvent/3.3.1.yml
--- definitions/EiffelTestSuiteFinishedEvent/3.3.0.yml  2023-07-05 08:54:43.870319175 +0200
+++ definitions/EiffelTestSuiteFinishedEvent/3.3.1.yml  2023-07-14 14:35:22.977779446 +0200
@@ -44,7 +44,7 @@
           conclusion:
             _description: |-
               A terse standardized conclusion of the test suite, designed to be machine readable. Unlike in [EiffelTestCaseFinishedEvent](./EiffelTestCaseFinishedEvent.md), this property is optional. It offers a method of summarizing the conclusion of the test suite as a whole, but may be skipped.
-              SUCCESSFUL signifies that the test suite was successfully concluded. Note that this does not imply that the item under test passed the tests.
+              SUCCESSFUL signifies that the test suite was successfully concluded. Note that this does not imply that the implementation under test passed the tests.
               FAILED signifies that the test suite could not be successfully executed. To exemplify, one or more tests failed to run due to required environments being unavailable.
               ABORTED signifies that the test suite was aborted before it could be concluded.
               TIMED_OUT signifies that the test suite did not conclude within the allowed time frame.
@@ -149,6 +149,8 @@
       types:
         - EiffelTestSuiteStartedEvent
 _history:
+  - version: 3.3.1
+    changes: Change IUT to Implementation Under Test (see [Issue 235](https://github.com/eiffel-community/eiffel/issues/359)).
   - version: 3.3.0
     introduced_in: edition-arica
     changes: Add schema URL to the meta object (see [Issue 280](https://github.com/eiffel-community/eiffel/issues/280)).
(venv) emalinn@E-5CG1504MQG:~/github/forks/eiffel$ diff -u schemas/EiffelTestSuiteFinishedEvent/3.3.0.json schemas/EiffelTestSuiteFinishedEvent/3.3.1.json
--- schemas/EiffelTestSuiteFinishedEvent/3.3.0.json     2023-07-14 13:51:12.067763109 +0200
+++ schemas/EiffelTestSuiteFinishedEvent/3.3.1.json     2023-07-14 13:51:12.127763110 +0200
@@ -18,9 +18,9 @@
         "version": {
           "type": "string",
           "enum": [
-            "3.3.0"
+            "3.3.1"
           ],
-          "default": "3.3.0"
+          "default": "3.3.1"
         },
         "time": {
           "type": "integer"

[m-linner-ericsson]

Thanks @magnusbaeck for your handy script https://github.com/magnusbaeck/eiffel/blob/diff/diff_defs.py

[magnusbaeck]

Everything looks fine, but would you mind ripping out the makefile patch to a separate PR? It's clearly unrelated to the IUT stuff.

[m-linner-ericsson]

Everything looks fine, but would you mind ripping out the makefile patch to a separate PR? It's clearly unrelated to the IUT stuff.

Sure, in 0438773 I reverted the Makefile change. Now part of #372

[magnusbaeck]

I don't think it's entirely obvious that we have to issue new event versions for this change. We're not changing the data schema nor any event semantics. Where do we draw the line between typo corrections (that I think we can agree shouldn't warrant new event versions) and this change?

[e-backmark-ericsson]

I agree with Magnus on his last comment. I don't think we need to step the event versions here.

I wouldn't expect any event producer/consumer to change anything to abide to the changes in this PR, since I believe item under test and implementation under test are semantically equivalent. I might be wrong in that statement though, and if we can identify a semantic difference between the two terms, that might force users of Eiffel to change how they use the IUT links then I can accept that we step the event versions.

[e-backmark-ericsson]

Approving this as-is, since it was decided on the latest Eiffel TC meeting that any change to the protocol yaml file should result in stepped event versions.
https://github.com/eiffel-community/community/blob/master/meetings/MEETINGS_TC_2023.md#aug-17-2023