Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

E2e tests overhaul #2024

Draft
wants to merge 1 commit into
base: v1.x-2022-07
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
1 change: 1 addition & 0 deletions .eslintrc.js
Expand Up @@ -21,6 +21,7 @@ module.exports = {
'jest/no-disabled-tests': 'off',
'jest/no-export': 'off',
'jsx-a11y/iframe-has-title': 'off',
'jest/no-standalone-expect': 'off',
'no-console': 'off',
'no-constant-condition': 'off',
'jest/no-done-callback': 'off',
Expand Down
5 changes: 5 additions & 0 deletions .github/workflows/tests_and_lint.yml
Expand Up @@ -71,3 +71,8 @@ jobs:
working-directory: ./templates/demo-store
env:
CI: true

- name: Run the Vitest E2E tests
run: yarn test-e2e-new
env:
CI: true
11 changes: 9 additions & 2 deletions package.json
Expand Up @@ -5,7 +5,8 @@
"workspaces": [
"templates/*",
"packages/*",
"packages/playground/*"
"packages/playground/*",
"test/**/fixture"
],
"engines": {
"node": ">=14"
Expand All @@ -23,6 +24,7 @@
"test:base": "jest",
"test:coverage": "jest --coverage",
"test-e2e": "jest --config jest-e2e.config.ts --runInBand",
"test-e2e-new": "vitest run --config ./test/vitest.config.ts",
"test:vite-ci": "cross-env FORCE_COLOR=true jest && jest --config jest-e2e.config.ts",
"test:hydrogen:vite": "yarn workspace @shopify/hydrogen test:vitest:ci",
"test:hydrogen-ui": "yarn workspace @shopify/hydrogen-ui test:ci",
Expand All @@ -45,7 +47,9 @@
"@typescript-eslint/parser": "^4.20.0",
"abort-controller": "^3.0.0",
"alex": "^9.1.0",
"change-case": "^4.1.2",
"cross-env": "^7.0.3",
"debug": "^4.3.4",
"eslint": "^8.16.0",
"eslint-plugin-hydrogen": "^0.12.1",
"eslint-plugin-node": "^11.1.0",
Expand All @@ -55,12 +59,15 @@
"faker": "^5.5.3",
"fast-glob": "^3.2.11",
"fs-extra": "^10.0.0",
"get-port": "^3.1.0",
"glob": "^7.2.0",
"jest": "^26.6.3",
"jsonlint": "^1.6.3",
"lint-staged": "^10.5.4",
"miniflare": "^1.3.3",
"node-fetch": "^2.6.7",
"npm-run-all": "^4.1.5",
"playwright": "^1.22.2",
"playwright-chromium": "^1.13.0",
"prettier": "^2.2.1",
"sirv": "^1.0.12",
Expand All @@ -70,6 +77,7 @@
"type-fest": "^2.12.0",
"typescript": "^4.7.2",
"vite": "^2.9.0",
"vitest": "^0.19.1",
"yorkie": "^2.0.0"
},
"gitHooks": {
Expand All @@ -91,7 +99,6 @@
]
},
"resolutions": {
"unified": "9.2.2",
"@shopify/cli-kit": "3.6.1"
},
"version": "0.0.0"
Expand Down
1 change: 1 addition & 0 deletions packages/hydrogen/src/index.ts
Expand Up @@ -10,6 +10,7 @@ export * from './client.js';
* The following are exported from this file because they are intended to be available
* *only* on the server.
*/

export {
ServerPropsProvider,
ServerPropsContext,
Expand Down
14 changes: 14 additions & 0 deletions test/basic-route/basic-route.test.ts
@@ -0,0 +1,14 @@
import {beforeAll} from 'vitest';
import {describe, MINIMAL_TEMPLATE} from '../test-framework';

describe(
'basic-route',
({fs}) => {
beforeAll(async () => {
await fs.write(MINIMAL_TEMPLATE);
});
},
{
routes: import.meta.glob('./routes/*.server.*'),
}
);
13 changes: 13 additions & 0 deletions test/basic-route/routes/index.server.tsx
@@ -0,0 +1,13 @@
export default function home() {
return <div className="text">Hello world</div>;
}

export const tests = ({expect, it, browser}) => {
it('Loads hello world', async () => {
await browser.navigate('/');

const text = await browser.text('.text');

expect(text).toBe('Hello world');
});
};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seeing this now in isolation - I like the colocation of component and test, but I don't like how it feels a little unclear (and magical?) how things are getting rendered and where it's coming from.

For example, it's not very clear that the default export above is getting rendered, or even how it's getting rendered.

Maybe this is an area where I wouldn't mind having to be more explicit, even if it means writing more code?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure I follow here, can you show an example of how you would prefer it to work?
The way I understand this, we render route X and we include the tests for route X in the same file... what's missing?

Copy link
Contributor

@frehner frehner Aug 24, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess I'm just thinking about other test frameworks, where you call something like render(<Home/>), and was thinking that I like the explicitness of that.

Somewhat related: in the current situation, how would you change the props passed to <Home/> to test different code paths? Is that possible?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think that's possible since these are e2e tests and depends on what the browser sends 🤔 -- might be wrong though

81 changes: 81 additions & 0 deletions test/multiple-routes/multiple-routes.test.tsx
@@ -0,0 +1,81 @@
import {beforeAll, expect} from 'vitest';
import {describe, MINIMAL_TEMPLATE} from '../test-framework';

describe(
'multiple-routes',
({fs, browser, it}) => {
beforeAll(async () => {
await fs.write(MINIMAL_TEMPLATE);

await fs.write(
'src/routes/index.server.jsx',
`
import {Link} from '@shopify/hydrogen';

export default function Home() {
return (
<>
<h1>Home</h1>
<Link className="btn" to="/about">About</Link>
</>
);
}
`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It kind of stinks that these have to be written as strings, as you lose features such as linting, formatting, typechecking, etc. (especially as part of the CI, which is helpful for keeping external contributions cleaner too)

I guess you don't have to create them as strings, though, right? They can be files in the filesystem? These were just done for ease of use?

Copy link
Contributor Author

@cartogram cartogram Aug 23, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah these can be files, I also think there is probably a tagged template literal (similar to gql) that may solve some of the issues you identified. Either one exists of we could create something fairly easily.

Copy link
Contributor

@frandiox frandiox Aug 24, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is my main question as well... what's the benefit of having this as strings over having files? Do we have an example or use-case where this approach is clearly better? 🤔

Maybe we testing multiple routes so that we can colocate tests with the content of the files?

);
await fs.write(
'src/routes/about.server.jsx',
`
import Counter from '../components/Counter.client';

export default function About() {
return (
<>
<h1 className="about">About</h1>
<Counter />
</>
);
}
`
);
await fs.write(
'src/components/Counter.client.jsx',
`
import {useState} from 'react';

export default function Counter() {
const [count, setCount] = useState(0);

return (
<div>
<h2 className="count">Count is {count}</h2>
<button className="increase" onClick={() => setCount(count + 1)}>
increase count
</button>
</div>
);
}
`
);
});

it('shows the homepage, navigates to about, and increases the count', async () => {
await browser.navigate('/');

expect(await browser.text('h1')).toBe('Home');

await browser.click('.btn');

expect(await browser.url().endsWith('/about')).toBeTruthy();
expect(await browser.text('.about')).toBe('About');
expect(await browser.text('.count')).toBe('Count is 0');

await browser.click('.increase');

expect(await browser.text('.count')).toBe('Count is 1');
});
},
{
modes: ['node-dev', 'node-prod'],
routes: import.meta.glob('./routes/*.server.*'),
}
);