[e2e] Update remaining tests and add missing comparison coverage #7363

unlikelyzero · 2024-01-09T17:31:19Z

Closes #7305
Addresses #7040 #7307

Describe your changes:

Generally brings all of our visual tests up to speed to address #7307 And add coverage that Charles needs

All Submissions:

Have you followed the guidelines in our Contributing document?
Have you checked to ensure there aren't other open Pull Requests for the same update/change?
Is this change backwards compatible? For example, developers won't need to change how they are calling the API or how they've extended core plugins such as Tables or Plots.

Author Checklist

Changes address original issue?
Tests included and/or updated with changes?
Has this been smoke tested?
Have you associated this PR with a type: label? Note: this is not necessarily the same as the original issue.
Have you associated a milestone with this PR? Note: leave blank if unsure.
Is this a breaking change to be called out in the release notes?
Testing instructions included in associated issue OR is this a dependency/testcase change?

Reviewer Checklist

Changes appear to address issue?
Reviewer has tested changes by following the provided instructions?
Changes appear not to be breaking changes?
Appropriate automated tests included?
Code style and in-line documentation are appropriate?

codecov · 2024-01-09T17:36:09Z

Codecov Report

Attention: 9 lines in your changes are missing coverage. Please review.

Comparison is base (6fd7b6f) 55.85% compared to head (89b5bab) 55.84%.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #7363      +/-   ##
==========================================
- Coverage   55.85%   55.84%   -0.01%     
==========================================
  Files         659      659              
  Lines       26243    26245       +2     
  Branches     2549     2549              
==========================================
- Hits        14658    14657       -1     
- Misses      10878    10881       +3     
  Partials      707      707

Flag	Coverage Δ
e2e-full	`40.81% <ø> (-0.27%)`	⬇️
e2e-stable	`59.04% <ø> (-0.03%)`	⬇️
unit	`48.88% <50.00%> (+0.01%)`	⬆️

Files	Coverage Δ
src/api/menu/components/MenuComponent.vue	`50.00% <ø> (ø)`
src/api/menu/components/SuperMenu.vue	`66.66% <ø> (ø)`
...plugins/displayLayout/components/TelemetryView.vue	`2.40% <ø> (ø)`
...lugins/faultManagement/FaultManagementListItem.vue	`5.00% <ø> (ø)`
src/plugins/folderView/components/GridItem.vue	`0.00% <ø> (ø)`
src/plugins/folderView/components/ListItem.vue	`50.00% <ø> (ø)`
src/plugins/gauge/components/GaugeComponent.vue	`61.49% <ø> (ø)`
src/plugins/imagery/components/ImageControls.vue	`35.38% <100.00%> (ø)`
src/plugins/imagery/components/ImageThumbnail.vue	`18.91% <ø> (ø)`
...ins/imagery/components/ImageryViewMenuSwitcher.vue	`33.33% <ø> (ø)`
... and 22 more

... and 7 files with indirect coverage changes

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6fd7b6f...89b5bab. Read the comment docs.

deploysentinel · 2024-01-09T17:36:12Z

Current Playwright Test Results Summary

✅ 174 Passing - ❌ 1 Failing - ⚠️ 1 Flaky

Run may still be in progress, this comment will be updated as current testing workflow or job completes...

(Last updated on 01/11/2024 06:48:31am UTC)

Run Details

Running Job e2e-stable on CircleCI

Commit: 1ac2c8a

Started: 01/11/2024 06:42:39am UTC

❌ Failures

📄 functional/plugins/telemetryTable/telemetryTable.e2e.spec.js • 1 Failure

Test Case Results

Test Case	Last 7 days Failures	Last 7 days Flakes
Telemetry Table Supports filtering using Regex Retry 2 • Retry 1 • Initial Attempt Error: Test timeout of 60000ms exceeded. `Test timeout of 60000ms exceeded.`	13.33% (8) 8 / 60 runs failed over last 7 days	0% (0) 0 / 60 runs flaked over last 7 days

⚠️ Flakes

📄 functional/plugins/notebook/restrictedNotebook.e2e.spec.js • 1 Flake

Test Case Results

Test Case	Last 7 days Failures	Last 7 days Flakes
Restricted Notebook with a page locked and with an embed @addinit Disallows embeds to be deleted if page locked @addinit Retry 1 • Initial Attempt	6.15% (4) 4 / 65 runs failed over last 7 days	49.23% (32) 32 / 65 runs flaked over last 7 days

View Detailed Build Results

…e-visual-tests

unlikelyzero · 2024-01-09T17:51:20Z

src/plugins/folderView/components/GridItem.vue

        <span class="c-grid-item__metadata__type">{{ item.type.name }}</span>
      </div>
    </div>
    <div class="c-grid-item__controls">
-      <div class="is-status__indicator" :title="`This item is ${status}`"></div>
+      <div
+        class="is-status__indicator"


unlikelyzero · 2024-01-09T18:16:33Z

src/ui/layout/AboutDialog.vue

@@ -57,7 +60,7 @@
          </p>
        </div>
        <h2>Version Information</h2>
-        <ul class="t-info l-info s-info">
+        <ul data-testid="versionInfo" class="t-info l-info s-info">


@ozyx not really sure what else to add here. I guess I could use an 'id'

It's a ul so it implicitly has the role of list. I think we would just need a label.

ozyx

Nice work! Just a few comments

ozyx · 2024-01-10T21:11:47Z

src/ui/layout/AboutDialog.vue

@@ -22,7 +22,7 @@
 <template>
  <!-- eslint-disable vue/no-v-html -->
  <div class="c-about c-about--splash">
-    <div class="c-about__image c-splash-image"></div>
+    <div class="c-about__image c-splash-image" role="img" alt="Open MCT Splash Logo"></div>


ozyx · 2024-01-10T21:12:44Z

src/ui/layout/AboutDialog.vue

@@ -57,7 +60,7 @@
          </p>
        </div>
        <h2>Version Information</h2>
-        <ul class="t-info l-info s-info">
+        <ul data-testid="versionInfo" class="t-info l-info s-info">


It's a ul so it implicitly has the role of list. I think we would just need a label.

src/plugins/plot/chart/MctChart.vue

src/plugins/imagery/components/ImageControls.vue

…e-visual-tests

ozyx · 2024-01-11T21:47:59Z

.gitignore

+# VSCode
+.vscode/settings.json


ozyx · 2024-01-11T21:48:26Z

e2e/playwright-watch.config.js

@@ -17,8 +12,7 @@ const config = {
    timeout: 200 * 1000,
    reuseExistingServer: true //This was originally disabled to prevent differences in local debugging vs. CI. However, it significantly speeds up local debugging.
  },
-  maxFailures: MAX_FAILURES, //Limits failures to 5 to reduce CI Waste
-  workers: NUM_WORKERS, //Limit to 2 for CircleCI Agent
+  workers: '75%', //Limit to 75% of the CPU to support running with dev server


ozyx · 2024-01-11T21:48:48Z

e2e/tests/framework/appActions.e2e.spec.js

@@ -155,7 +155,7 @@ test.describe('AppActions', () => {

    await page.goto('./#/browse/mine');
    //Click the Create button
-    await page.click('button:has-text("Create")');
+    await page.getByRole('button', { name: 'Create' }).click();


about dang time

ozyx

Awesome work! Just a few comments

ozyx · 2024-01-11T21:49:18Z

e2e/tests/framework/generateLocalStorageData.e2e.spec.js

      type: 'Display Layout',
      name: 'Child Layout 2',
      parent: parent.uuid
    });

-    await page.goto(parent.url);
+    await page.goto(parent.url, { waitUntil: 'domcontentloaded' });


we don't need the waitUntil here unless you noticed some different behavior?

ozyx · 2024-01-11T21:49:30Z

e2e/tests/framework/generateLocalStorageData.e2e.spec.js

@@ -107,7 +106,7 @@ test.describe('Generate Visual Test Data @localStorage @generatedata', () => {
      parent: parent.uuid
    });

-    await page.goto(parent.url);
+    await page.goto(parent.url, { waitUntil: 'domcontentloaded' });


ozyx · 2024-01-11T21:49:43Z

e2e/tests/framework/generateLocalStorageData.e2e.spec.js

@@ -134,7 +133,7 @@ test.describe('Generate Visual Test Data @localStorage @generatedata', () => {
    await page.locator('button[title="More actions"]').click();


missed a spot?

ozyx · 2024-01-11T21:50:28Z

e2e/tests/functional/planning/ganttChart.e2e.spec.js

@@ -69,7 +69,7 @@ test.describe('Gantt Chart', () => {
      .getByRole('dialog')
      .filter({ hasText: 'This action will replace the current Plan. Do you want to continue?' });
    await expect(replaceModal).toBeVisible();
-    await page.getByRole('button', { name: 'OK' }).click();
+    await page.getByRole('button', { name: 'Ok', exact: true }).click();


It's Ok and not OK ?

ozyx · 2024-01-11T21:50:51Z

e2e/tests/functional/plugins/gauge/gauge.e2e.spec.js

@@ -53,10 +51,10 @@ test.describe('Gauge', () => {

    // Navigate to the gauge and verify that
    // the SWG appears in the elements pool
-    await page.goto(gauge.url);
-    await editButtonLocator.click();
+    await page.goto(gauge.url, { waitUntil: 'domcontentloaded' });


don't think we need the waitUntil here

ozyx · 2024-01-11T21:51:25Z

e2e/tests/functional/plugins/gauge/gauge.e2e.spec.js

@@ -109,7 +107,7 @@ test.describe('Gauge', () => {
      description: 'https://github.com/nasa/openmct/issues/5356'
    });
    //Click the Create button
-    await page.click('button:has-text("Create")');
+    await page.getByRole('button', { name: 'Create' }).click();

    // Click the object specified by 'type'
    await page.click(`li[role='menuitem']:text("Gauge")`);


missed one?

ozyx · 2024-01-11T21:52:33Z

package.json

@@ -8,7 +8,6 @@
    "@axe-core/playwright": "4.8.2",
    "@babel/eslint-parser": "7.23.3",
    "@braintree/sanitize-url": "6.0.4",
-    "@deploysentinel/playwright": "0.3.4",


addressed

ozyx

LGTM! Nice one

remove remaining page.locators

8953b32

unlikelyzero requested review from charlesh88 and ozyx January 9, 2024 17:31

unlikelyzero added this to the Target:3.3.0 milestone Jan 9, 2024

unlikelyzero added the type:maintenance tests, chores, or project maintenance label Jan 9, 2024

unlikelyzero mentioned this pull request Jan 9, 2024

New action to reload an individual view and all of its children #7362

Merged

15 tasks

unlikelyzero added 2 commits January 9, 2024 09:49

Merge branch 'master' of https://github.com/nasa/openmct into fix-mor…

d4310d3

…e-visual-tests

remove icon

18899bc

unlikelyzero commented Jan 9, 2024

View reviewed changes

unlikelyzero added 4 commits January 10, 2024 12:50

update test to remove bad locators

121e0ed

Update tests to match aria changes

746e8fa

duplicate aria-label from title

774bbfc

Update Edit action to be more explicit

c9bae39

ozyx previously requested changes Jan 10, 2024

View reviewed changes

unlikelyzero added 13 commits January 10, 2024 13:33

update the watch file to better suite long runs

f041845

fix locators

d2155c8

fix locator

a37642e

revert to action.name for test

8884ae0

PER JESSE'S INTENT

49b4f1e

review comments

f9122d9

fix tests

4dd515f

more fixes

5affd36

update text

d6d0739

review comment

29244c0

locator fixes

d846c1f

oops

7c96dcd

oops

b75e333

unlikelyzero added 5 commits January 11, 2024 08:38

remove deploysentinel

316fb00

remove deploysentinel

d15a6f1

fix generatelocal storage

090bba2

lint fix

9dcf95a

new locator

a1c2844

unlikelyzero requested a review from ozyx January 11, 2024 19:06

header component test added

9bad40a

unlikelyzero added the pr:e2e:couchdb npm run test:e2e:couchdb label Jan 11, 2024

github-actions bot removed the pr:e2e:couchdb npm run test:e2e:couchdb label Jan 11, 2024

unlikelyzero added 3 commits January 11, 2024 13:16

telemetryView tests

b3a5cf2

Merge branch 'master' of https://github.com/nasa/openmct into fix-mor…

26c0f81

…e-visual-tests

2024

003d7c2

ozyx reviewed Jan 11, 2024

View reviewed changes

.gitignore

Comment on lines +18 to +19

# VSCode

.vscode/settings.json

Copy link

Member

ozyx Jan 11, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😞

ozyx reviewed Jan 11, 2024

View reviewed changes

driveby: fix from missing pr review

4b27e09

ozyx reviewed Jan 11, 2024

View reviewed changes

unlikelyzero added 3 commits January 11, 2024 13:53

add delay for known issue

72fe139

feedback

1d5c287

comments

89b5bab

unlikelyzero added the pr:e2e:couchdb npm run test:e2e:couchdb label Jan 11, 2024

unlikelyzero requested a review from ozyx January 11, 2024 22:01

github-actions bot removed the pr:e2e:couchdb npm run test:e2e:couchdb label Jan 11, 2024

unlikelyzero enabled auto-merge (squash) January 11, 2024 22:43

ozyx approved these changes Jan 11, 2024

View reviewed changes

unlikelyzero merged commit 6ce340c into master Jan 11, 2024
31 of 39 checks passed

unlikelyzero deleted the fix-more-visual-tests branch January 11, 2024 22:47

unlikelyzero modified the milestones: Target:3.3.0, Target:4.0.0 Jan 17, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[e2e] Update remaining tests and add missing comparison coverage #7363

[e2e] Update remaining tests and add missing comparison coverage #7363

unlikelyzero commented Jan 9, 2024 •

edited

codecov bot commented Jan 9, 2024 •

edited

deploysentinel bot commented Jan 9, 2024 •

edited

Test Case Results

Test Case Results

unlikelyzero Jan 9, 2024

unlikelyzero Jan 9, 2024

ozyx Jan 10, 2024

ozyx left a comment

ozyx Jan 10, 2024

ozyx Jan 10, 2024

ozyx Jan 11, 2024

ozyx Jan 11, 2024

ozyx Jan 11, 2024

ozyx left a comment

ozyx Jan 11, 2024

ozyx Jan 11, 2024

ozyx Jan 11, 2024

ozyx Jan 11, 2024

ozyx Jan 11, 2024

ozyx Jan 11, 2024

ozyx Jan 11, 2024

ozyx left a comment

		@@ -134,7 +133,7 @@ test.describe('Generate Visual Test Data @localStorage @generatedata', () => {
		await page.locator('button[title="More actions"]').click();

[e2e] Update remaining tests and add missing comparison coverage #7363

[e2e] Update remaining tests and add missing comparison coverage #7363

Conversation

unlikelyzero commented Jan 9, 2024 • edited

Describe your changes:

All Submissions:

Author Checklist

Reviewer Checklist

codecov bot commented Jan 9, 2024 • edited

Codecov Report

deploysentinel bot commented Jan 9, 2024 • edited

Current Playwright Test Results Summary

❌ Failures

Test Case Results

⚠️ Flakes

Test Case Results

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ozyx left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ozyx left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ozyx left a comment

Choose a reason for hiding this comment

unlikelyzero commented Jan 9, 2024 •

edited

codecov bot commented Jan 9, 2024 •

edited

deploysentinel bot commented Jan 9, 2024 •

edited