Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[e2e] Update playwright eslint plugin #7581

Open
wants to merge 12 commits into
base: master
Choose a base branch
from
3 changes: 2 additions & 1 deletion .eslintrc.cjs
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,8 @@ module.exports = {
browser: true,
es6: true,
jasmine: true,
amd: true
amd: true,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think we can set this to false or remove it altogether

node: true
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice

},
globals: {
_: 'readonly'
Expand Down
12 changes: 9 additions & 3 deletions e2e/.eslintrc.cjs
Original file line number Diff line number Diff line change
@@ -1,14 +1,20 @@
/* eslint-disable no-undef */
module.exports = {
extends: ['plugin:playwright/playwright-test'],
extends: ['plugin:playwright/recommended'],
rules: {
'playwright/max-nested-describe': ['error', { max: 1 }]
},
overrides: [
{
files: ['tests/visual/*.spec.js'],
files: ['**/*.visual.spec.js'],
rules: {
'playwright/no-wait-for-timeout': 'off'
'playwright/expect-expect': 'off'
}
},
{
files: ['**/*.perf.spec.js'],
rules: {
'playwright/expect-expect': 'off'
}
}
]
Expand Down
12 changes: 11 additions & 1 deletion e2e/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -383,7 +383,7 @@ By adhering to this principle, we can create tests that are both robust and refl
1. Avoid creating locator aliases. This likely means that you're compensating for a bad locator. Improve the application instead.
1. Leverage `await page.goto('./', { waitUntil: 'domcontentloaded' });` instead of `{ waitUntil: 'networkidle' }`. Tests run against deployments with websockets often have issues with the networkidle detection.

#### How to make tests faster and more resilient
#### How to make tests faster and more resilient to application changes
1. Avoid app interaction when possible. The best way of doing this is to navigate directly by URL:

```js
Expand All @@ -398,6 +398,16 @@ By adhering to this principle, we can create tests that are both robust and refl
- Initial navigation should _almost_ always use the `{ waitUntil: 'domcontentloaded' }` option.
1. Avoid repeated setup to test a single assertion. Write longer tests with multiple soft assertions.
This ensures that your changes will be picked up with large refactors.
1. Use [user-facing locators](https://playwright.dev/docs/best-practices#use-locators) (Now a eslint rule!)

```js
page.getByRole('button', { name: 'Create' } )
```
Instead of
```js
page.locator('.c-create-button')
```
Note: `page.locator()` can be used in performance tests as xk6-browser does not yet support the new `page.getBy` pattern and css lookups can be [1.5x faster](https://serpapi.com/blog/css-selectors-faster-than-getbyrole-playwright/)
Comment on lines +401 to +410
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion:

  1. Prefer using user-facing locators for better readability and maintainability:
// Recommended
page.getByRole('button', { name: 'Create' })

Instead of the less intuitive:

// Not recommended
page.locator('.c-create-button')

Note: In performance tests, page.locator() may still be used as xk6-browser lacks support for the page.getBy pattern. Additionally, CSS selectors can be up to 1.5x faster than role-based selectors.


##### Utilizing LocalStorage
1. In order to save test runtime in the case of tests that require a decent amount of initial setup (such as in the case of testing complex displays), you may use [Playwright's `storageState` feature](https://playwright.dev/docs/api/class-browsercontext#browser-context-storage-state) to generate and load localStorage states.
Expand Down
6 changes: 3 additions & 3 deletions e2e/tests/functional/couchdb.e2e.spec.js
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ test.describe('CouchDB Status Indicator with mocked responses @couchdb', () => {

//Go to baseURL
await page.goto('./#/browse/mine?hideTree=true&hideInspector=true', {
waitUntil: 'networkidle'
waitUntil: 'domcontentloaded'
});
await expect(page.locator('div:has-text("CouchDB is connected")').nth(3)).toBeVisible();
});
Expand All @@ -56,7 +56,7 @@ test.describe('CouchDB Status Indicator with mocked responses @couchdb', () => {

//Go to baseURL
await page.goto('./#/browse/mine?hideTree=true&hideInspector=true', {
waitUntil: 'networkidle'
waitUntil: 'domcontentloaded'
});
await expect(page.locator('div:has-text("CouchDB is offline")').nth(3)).toBeVisible();
});
Expand All @@ -71,7 +71,7 @@ test.describe('CouchDB Status Indicator with mocked responses @couchdb', () => {

//Go to baseURL
await page.goto('./#/browse/mine?hideTree=true&hideInspector=true', {
waitUntil: 'networkidle'
waitUntil: 'domcontentloaded'
});
await expect(page.locator('div:has-text("CouchDB connectivity unknown")').nth(3)).toBeVisible();
});
Expand Down
4 changes: 2 additions & 2 deletions e2e/tests/functional/forms.e2e.spec.js
Original file line number Diff line number Diff line change
Expand Up @@ -188,8 +188,8 @@ test.describe('Persistence operations @couchdb', () => {

// Both pages: Go to baseURL
await Promise.all([
page.goto('./', { waitUntil: 'networkidle' }),
page2.goto('./', { waitUntil: 'networkidle' })
page.goto('./', { waitUntil: 'domcontentloaded' }),
page2.goto('./', { waitUntil: 'domcontentloaded' })
]);

//Slow down the test a bit
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ test.describe.serial('Condition Set CRUD Operations on @localStorage @2p', () =>
description: 'https://github.com/nasa/openmct/issues/7421'
});
//Navigate to baseURL with injected localStorage
await page.goto(conditionSetUrl, { waitUntil: 'networkidle' });
await page.goto(conditionSetUrl, { waitUntil: 'domcontentloaded' });

//Assertions on loaded Condition Set in main view. This is a stateful transition step after page.goto()
await expect
Expand All @@ -87,7 +87,7 @@ test.describe.serial('Condition Set CRUD Operations on @localStorage @2p', () =>
expect.soft(page.locator('_vue=item.name=Unnamed Condition Set')).toBeTruthy();

//Reload Page
await Promise.all([page.reload(), page.waitForLoadState('networkidle')]);
await Promise.all([page.reload(), page.waitForLoadState('domcontentloaded')]);

//Re-verify after reload
await expect
Expand All @@ -100,7 +100,7 @@ test.describe.serial('Condition Set CRUD Operations on @localStorage @2p', () =>
test('condition set object can be modified on @localStorage', async ({ page, openmctConfig }) => {
const { myItemsFolderName } = openmctConfig;

await page.goto(conditionSetUrl, { waitUntil: 'networkidle' });
await page.goto(conditionSetUrl, { waitUntil: 'domcontentloaded' });

//Assertions on loaded Condition Set in main view. This is a stateful transition step after page.goto()
await expect
Expand Down Expand Up @@ -151,7 +151,7 @@ test.describe.serial('Condition Set CRUD Operations on @localStorage @2p', () =>
expect(page.locator('a:has-text("Renamed Condition Set")')).toBeTruthy();

//Reload Page
await Promise.all([page.reload(), page.waitForLoadState('networkidle')]);
await Promise.all([page.reload(), page.waitForLoadState('domcontentloaded')]);

//Verify Main section reflects updated Name Property
await expect
Expand Down Expand Up @@ -213,7 +213,7 @@ test.describe.serial('Condition Set CRUD Operations on @localStorage @2p', () =>

//Feature?
//Domain Object is still available by direct URL after delete
await page.goto(conditionSetUrl, { waitUntil: 'networkidle' });
await page.goto(conditionSetUrl, { waitUntil: 'domcontentloaded' });
await expect(page.locator('.l-browse-bar__object-name')).toContainText('Unnamed Condition Set');
});
});
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -519,7 +519,7 @@ test.describe('Display Layout', () => {
await page.reload();

// wait for annotations requests to be batched and requested
await page.waitForLoadState('networkidle');
await page.waitForLoadState('domcontentloaded');
// Network requests for the composite telemetry with multiple items should be:
// 1. a single batched request for annotations
expect(networkRequests.length).toBe(1);
Expand All @@ -531,7 +531,7 @@ test.describe('Display Layout', () => {
await page.reload();

// wait for annotations to not load (if we have any, we've got a problem)
await page.waitForLoadState('networkidle');
await page.waitForLoadState('domcontentloaded');

// In real time mode, we don't fetch annotations at all
expect(networkRequests.length).toBe(0);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -531,7 +531,7 @@ test.describe('Example Imagery in Flexible layout', () => {

// Click text=OK
await Promise.all([
page.waitForNavigation({ waitUntil: 'networkidle' }),
page.waitForNavigation({ waitUntil: 'domcontentloaded' }),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

intellisense says this is deprecated

page.click('button:has-text("OK")'),
//Wait for Save Banner to appear
page.waitForSelector('.c-message-banner__message')
Expand Down Expand Up @@ -575,7 +575,7 @@ test.describe('Example Imagery in Tabs View', () => {

// Click text=OK
await Promise.all([
page.waitForNavigation({ waitUntil: 'networkidle' }),
page.waitForNavigation({ waitUntil: 'domcontentloaded' }),
page.click('button:has-text("OK")'),
//Wait for Save Banner to appear
page.waitForSelector('.c-message-banner__message')
Expand Down Expand Up @@ -1023,7 +1023,7 @@ async function createImageryView(page) {

// Click text=OK
await Promise.all([
page.waitForNavigation({ waitUntil: 'networkidle' }),
page.waitForNavigation({ waitUntil: 'domcontentloaded' }),
page.click('button:has-text("OK")'),
//Wait for Save Banner to appear
page.waitForSelector('.c-message-banner__message')
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -152,7 +152,7 @@ test.describe('ExportAsJSON Disabled Actions', () => {
test.describe('ExportAsJSON ProgressBar @couchdb', () => {
let folder;
test.beforeEach(async ({ page }) => {
await page.goto('./', { waitUntil: 'networkidle' });
await page.goto('./', { waitUntil: 'domcontentloaded' });
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i worry about this breaking the couchdb tests

// Perform actions to create the domain object
folder = await createDomainObjectWithDefaults(page, {
type: 'Folder'
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ test.describe('Notebook Tests with CouchDB @couchdb', () => {

// Create Notebook
testNotebook = await createDomainObjectWithDefaults(page, { type: 'Notebook' });
await page.goto(testNotebook.url, { waitUntil: 'networkidle' });
await page.goto(testNotebook.url, { waitUntil: 'domcontentloaded' });
});

test('Inspect Notebook Entry Network Requests', async ({ page }) => {
Expand All @@ -58,7 +58,7 @@ test.describe('Notebook Tests with CouchDB @couchdb', () => {
page.click('[aria-label="Add Page"]')
]);
// Ensures that there are no other network requests
await page.waitForLoadState('networkidle');
await page.waitForLoadState('domcontentloaded');

// Assert that only two requests are made
// Network Requests are:
Expand All @@ -77,7 +77,7 @@ test.describe('Notebook Tests with CouchDB @couchdb', () => {
// 2) The shared worker event from 👆 POST request
notebookElementsRequests = [];
await nbUtils.enterTextEntry(page, 'First Entry');
await page.waitForLoadState('networkidle');
await page.waitForLoadState('domcontentloaded');
expect(notebookElementsRequests.length).toBeLessThanOrEqual(2);

// Add some tags
Expand Down Expand Up @@ -141,7 +141,7 @@ test.describe('Notebook Tests with CouchDB @couchdb', () => {
// 4) The shared worker event from 👆 POST request
notebookElementsRequests = [];
await nbUtils.enterTextEntry(page, 'Fourth Entry');
page.waitForLoadState('networkidle');
page.waitForLoadState('domcontentloaded');
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i don't know if these are necessarily 1 to 1 here. The networkidles serve a purpose. I think maybe we need to wait for network events specifically to complete? We should just leave these as warnings for now


expect(filterNonFetchRequests(notebookElementsRequests).length).toBeLessThanOrEqual(4);

Expand All @@ -153,7 +153,7 @@ test.describe('Notebook Tests with CouchDB @couchdb', () => {
// 4) The shared worker event from 👆 POST request
notebookElementsRequests = [];
await nbUtils.enterTextEntry(page, 'Fifth Entry');
page.waitForLoadState('networkidle');
page.waitForLoadState('domcontentloaded');

expect(filterNonFetchRequests(notebookElementsRequests).length).toBeLessThanOrEqual(4);

Expand All @@ -164,7 +164,7 @@ test.describe('Notebook Tests with CouchDB @couchdb', () => {
// 4) The shared worker event from 👆 POST request
notebookElementsRequests = [];
await nbUtils.enterTextEntry(page, 'Sixth Entry');
page.waitForLoadState('networkidle');
page.waitForLoadState('domcontentloaded');

expect(filterNonFetchRequests(notebookElementsRequests).length).toBeLessThanOrEqual(4);
});
Expand Down Expand Up @@ -227,7 +227,7 @@ async function addTagAndAwaitNetwork(page, tagName) {
page.locator(`[aria-label="Autocomplete Options"] >> text=${tagName}`).click(),
expect(page.locator(`[aria-label="Tag"]:has-text("${tagName}")`)).toBeVisible()
]);
await page.waitForLoadState('networkidle');
await page.waitForLoadState('domcontentloaded');
}

/**
Expand All @@ -246,5 +246,5 @@ async function removeTagAndAwaitNetwork(page, tagName) {
)
]);
await expect(page.locator(`[aria-label="Tag"]:has-text("${tagName}")`)).toBeHidden();
await page.waitForLoadState('networkidle');
await page.waitForLoadState('domcontentloaded');
}
6 changes: 3 additions & 3 deletions e2e/tests/functional/plugins/plot/missingPlotObj.e2e.spec.js
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ test.describe('Handle missing object for plots', () => {
await page.evaluate(`window.localStorage.setItem('mct', '${JSON.stringify(parsedData)}')`);

//Reloads page and clicks on stacked plot
await Promise.all([page.reload(), page.waitForLoadState('networkidle')]);
await Promise.all([page.reload(), page.waitForLoadState('domcontentloaded')]);

//Verify Main section is there on load
await expect
Expand Down Expand Up @@ -92,7 +92,7 @@ async function makeStackedPlot(page, myItemsFolderName) {
await page.locator('li[role="menuitem"]:has-text("Stacked Plot")').click();

await Promise.all([
page.waitForNavigation({ waitUntil: 'networkidle' }),
page.waitForNavigation({ waitUntil: 'domcontentloaded' }),
page.locator('button:has-text("OK")').click(),
//Wait for Save Banner to appear
page.waitForSelector('.c-message-banner__message')
Expand Down Expand Up @@ -153,7 +153,7 @@ async function createSineWaveGenerator(page) {
await page.locator('li[role="menuitem"]:has-text("Sine Wave Generator")').click();

await Promise.all([
page.waitForNavigation({ waitUntil: 'networkidle' }),
page.waitForNavigation({ waitUntil: 'domcontentloaded' }),
page.locator('button:has-text("OK")').click(),
//Wait for Save Banner to appear
page.waitForSelector('.c-message-banner__message')
Expand Down
2 changes: 1 addition & 1 deletion e2e/tests/functional/renaming.e2e.spec.js
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ import { expect, test } from '../../baseFixtures.js';
test.describe('Renaming objects', () => {
test.beforeEach(async ({ page }) => {
// Go to baseURL
await page.goto('./', { waitUntil: 'networkidle' });
await page.goto('./', { waitUntil: 'domcontentloaded' });
});

test('When renaming objects, the browse bar and various components all update', async ({
Expand Down