Skip to content

Latest commit

 

History

History
480 lines (380 loc) · 16.5 KB

cypress.md

File metadata and controls

480 lines (380 loc) · 16.5 KB
Raw

Cypress

  • run tests on production build yarn build && yarn start && yarn cypress, docs

    Since Cypress is testing a real Next.js application, it requires the Next.js server to be running prior to starting Cypress. We recommend running your tests against your production code to more closely resemble how your application will behave.

  • for Cypress and Next.js NODE_ENV should be production to avoid recompiling

  • next-auth docs

  • Next.js docs, with-cypress Github example

  • Cypress Real world app repo, todomvc example

  • short Youtube playlist, Github repo

  • .should('exist') on cypress instead of expect() in jest

  • start prod build with test db and seed in beforeAll

  • test DATABASE_URL is passed at runtime to prisma, not next.js app on build, but dev mode is also set at runtime in server.ts (it's just compiled, env var is not inlined)

  • when app is started on host db hostname must be localhost, when app is in docker db hostname must be npb-test-db service name (dev and test)

  • setup:

  • beforeAll, afterAll -> before(), after() in Cypress

  • must use tsconfig.json for name.test.ts Typescript tests

// add in cypress/support/commands.js
import '@testing-library/cypress/add-commands';

  • create admin user for cypress?

  • ignore specific console error in Cypress

Cypress.on('uncaught:exception', (err, runnable) => {
  // we expect a 3rd party library error with message 'list not defined'
  // and don't want to fail the test so we return false
  if (err.message.includes('list not defined')) {
    return false;
  }
});
  • contains() same as findByText()
  • wait() for requests (GET search posts) with @alias tutorial, docs
// needed for wait()
cy.intercept('GET', `${Routes.API.POSTS}*`).as('searchPosts');

// wait for http request
cy.wait('@searchPosts');
  • assert redirect to another page - it's browser, just url and UI
// assert redirect to home
cy.url().should('eq', Cypress.config().baseUrl + '/');
cy.findByRole('heading', { name: /^home$/i }).should('exist');

  • todo: mock session discussion

  • seed db task example repo

  • docs all examples

  • cypress docker article

  • can NOT change and inspect next page in within(...) or invoke().then(...), tricky

  • intercept http must be before that GET or PATCH... call is made

// must be before click()
cy.intercept('PATCH', `${Routes.API.POSTS}*`).as('patchPost');

// edit title
cy.findByRole('button', { name: /update/i }).click();

cy.wait('@patchPost');
  • save element text for later, it() must use function(){}
context('post page', () => {
  beforeEach(() => {
    cy.visit('/');
    cy.findByText(/^log out$/i).should('exist');

    cy.get('.home__list .post-item:first-child h2').invoke('text').as('postTitle');
  });

  // MUST use function() instead of () => {} for this
  it.only('post page, edit and delete post', function () {
    // remember title
    const postTitle = this.postTitle as string;
    // ...
  });
});

Cookies

  • whole next-auth session is in next-auth.session-token cookie
// cannot clear session this way, even before beforeAll
Cypress.Cookies.defaults({
  preserve: cookieName,
});
// correct way, disable default cookies clear afterEach test
beforeEach(() => {
  Cypress.Cookies.preserveOnce('next-auth.session-token');
});

// clear after last test, so next run can run
// no, clear cookies in before(), this causes warning
after(async () => {
  cy.clearCookies();
  cy.getCookies().should('be.empty');
});

const loginAsAdmin = () => {
  // assert cookie after login
  cy.getCookie(cookieName).should('exist');
};

Commands

// cypress/support/index.js

Cypress.Commands.add('loginAsAdmin', () => {
  // ...
});

// usage
cy.loginAsAdmin();
  • types for commands
// cypress/global.d.ts

declare global {
  namespace Cypress {
    interface Chainable {
      seedDbViaUI: () => void;
      loginAsAdmin: () => void;
    }
  }
}

export {};

Tasks

  • use tasks in cypress/plugins/index.js to seed, teardown, not commands, returns promise and blocks, run any node process, tutorial, docs, example
// cypress/plugins/index.js

on('task', {
  'db:seed': async () => {
    await seedInstance.handledSeed();
    return null;
  },
  'db:teardown': async () => {
    await seedInstance.handledDeleteAllTables();
    return null;
  },
});

// call
cy.task('db:seed');
cy.task('db:teardown');

Cypress Docker

  • Bahmutov tutorial docs
  • test online website with cypress/included:4.1.0 Docker imagecypress-example-docker-compose-included, make npb-e2e-chrome service
  • app and cypress Dockerfiles and d-c.yml services, Dockerfile needed for additional npm devDependencies in Cypress image cypress-io/cypress-example-docker-compose, realistic use-case, app and tests in Docker, no Github Actions, custom Cypress install from package.json and base image (only needed for custom cypress install)
  • custom install - any imports inside /cypress folder
// cypress/support/commands.js
import '@testing-library/cypress/add-commands';

  • run test as correct non-root user from host repo, this Dockerfile, create user and group appuser with ids from the host passed via ARG

  • problem:

 Error: Webpack Compilation Error
npb-e2e         | [tsl] ERROR
npb-e2e         |       TS18002: The 'files' list in config file 'tsconfig.json' is empty.
  • solution:
  1. must have two extended tsconfig.json parent/child, same like on host
  2. and folder structure like on host (so imports and config can be the same)
  3. and started with cypress run --project ./tests-e2e from local package.json, (tests-e2e/package.json -> /app/package.json (root))
  4. Cypress must be started from same folder like package.json and node_modules
// tests-e2e/package.json
"scripts": {
  "test": "wait-on http://npb-app-test:3001 && cypress run --project ./tests-e2e"
},
  • uploads folder for images?

Cypress Github Actions

  • docs
  • example repo bahmutov/cypress-gh-action-included, manual, without cypres action run: cypress run, trivial example
  • docker-compose up -d db-containers in Github Actions bahmutov/chat.io
  • for Cypress in GA use cypress-io/github-action@v2 action or cypress/included:4.1.0 docker container???
  • Cypress Github Actions example, jobs: install, install-windows, ui-chrome-tests, ui-chrome-mobile-tests, ui-firefox-tests, no docker-compose.yml cypress-realworld-app, use this GA example
  • if you want te reuse local Cypress Docker container in GA you must rebuild container each time, for additional yarn dependencies? Better use cypress-io/github-action@v2
  • actions/upload-artifact@v3, actions/download-artifact@v3 to share build and dependencies between jobs

0 run cypress action on host or container in Github Actions Github, a lot of possible arguments for this action...

# run action directly in ubuntu
cypress-run:
  runs-on: ubuntu-20.04
  steps:
    - uses: actions/checkout@v2
    - uses: cypress-io/github-action@v4
      with:
        project: ./some/nested/folder # project option
# run action in container
cypress-run:
  runs-on: ubuntu-20.04
  container: cypress/browsers:node12.13.0-chrome78-ff70
  steps:
    - uses: actions/checkout@v2
    - uses: cypress-io/github-action@v4
      with:
        browser: chrome # default headless
jobs:
  install:
    runs-on: ubuntu-latest
    # image with just browsers without Cypress
    container: cypress/browsers:node16.14.2-slim-chrome100-ff99-edge
    steps:
      - name: Checkout
        uses: actions/checkout@v3

      # just install Cypress on bare image with browsers
      # action can install Cypress and run tests
      # probably to avoid Dockerfile with additional dependecies?
      # actually to reuse install step
      - name: Cypress install
        uses: cypress-io/github-action@v2
        with:
          runTests: false

        # reuse - save built code between jobs
        - name: Save build folder
        uses: actions/upload-artifact@v3
        with:
          name: build
          if-no-files-found: error
          path: build

  # reuse install job in other jobs
  ui-chrome-tests:
    # this, like depends_on
    needs: install

    steps:
      - name: Checkout
        uses: actions/checkout@v3

        # use built code it like this in ui-chrome-tests job
      - name: Download the build folders
        uses: actions/download-artifact@v3
        with:
          name: build
          path: build

Custom Cypress folder

  • --config-file vs --project github

  • docs

  • must use it because of local package.json

// i need this (moves entire folder with default cypress.json)
"cypress": "cypress open --project ./tests-e2e",
// and not this (moves only cypress.json)
"cypress": "cypress open --config-file tests-e2e/cypress.json",

  • cypress container needs: @testing-library/cypress, prisma, typescript, all imported files from next app, seed.js, all imports from seed.js (bcryptjs, faker), wait-on

  • be careful with imports from next.js app in Cypress tests, you need to copy them in container for Docker, and imports of their imports...

  • fakeUser as fixture, and not import from next.js app, docs, just require json

// 2-advanced-examples/files.spec.js
const requiredExample = require('../../fixtures/example');
npb-app-test    | error - ESLint: Failed to load plugin 'cypress' declared in '.eslintrc.json': Cannot find module 'eslint-plugin-cypress' Require stack: - /app/__placeholder__.js Referenced from: /app/.eslintrc.json

Reusable yarn script - function

  • better use env-cmd or dotenv-cli, actually this is dotenv-cli
  • but actually docker-compose.yml and .env.* files solve this
// original
"test:e2e:env:original": "dotenv -e .env.test.local -- sh -c 'yarn test:e2e'",
// print args
"with:env:debug": "fn() { echo \"1=$1 2=$2 3=$3\";}; fn --",
// fn
"with:env": "fn() { npx dotenv -e \"$3\" -- bash -c \"$2\";}; fn --",
// call
"test:e2e:env": "yarn with:env 'yarn test:e2e' '.env.test.local'"
//
// shorter form without yarn and ''
// always make separate yarn script without :env first
"with:env": "fn() { npx dotenv -e \"$3\" -- bash -c \"yarn $2\";}; fn --",
"test:e2e:env": "yarn with:env test:e2e .env.test.local"

Cypress env vars

// override, most right - most priority
cypress.config.js -> cypress.env.json -> CYPRESS_*  regular env var -> cypress run --env v1=val -> describe('', {env: {v1: 'val'}}), or it()

  • npb-e2e container does NOT need access to uploads folder, all is in npb-app-test and via axios, seed is ok

  • env overrides for Docker in docker-compose.yml

npb-e2e:
  # docker env override
  environment:
    # only cy.visit(), cy.request() url
    - CYPRESS_baseUrl=http://npb-app-test:3001
    # seed db_url
    - POSTGRES_HOSTNAME=npb-db-test
  # only db_url for seed, no Next.js vars
  env_file:
    - .env.test.local
  • app container env overrides, NEXTAUTH_URL and db_url, npb-app-prod?
# docker-compose.test.yml
npb-app-test:
  # docker env override
  environment:
    # ref to itself
    # NEXTAUTH_URL, NEXT_PUBLIC_BASE_URL in axiosInstance, imageLoader
    # NEXTAUTH_URL=$PROTOCOL://$HOSTNAME:$PORT
    - HOSTNAME=npb-app-test # will have CORS for api from localhost (browser)
    # db_url
    - POSTGRES_HOSTNAME=npb-db-test
  env_file:
    - .env.test
    - .env.test.local
  • environment: has precedence over env_file: in docker-compose.yml
  • look resloved env vars in Portainer
# expands later, remains same in container
NEXTAUTH_URL=$PROTOCOL://$HOSTNAME:$PORT
---
# must expand immediately for schema migrate, for seed (next.js) doesn't
# expands immediately, cannot override part
DATABASE_URL=postgresql://${POSTGRES_USER}...
  • dotenv-cli reddit?
  • list all states env.test.integration, env.e2e, env.test.local
# api integration
# local
# .env.test
HOSTNAME=localhost

# .env.test.local
POSTGRES_HOSTNAME=localhost
DATABASE_URL=postgresql://${POSTGRES_USER}:${POSTGRES_PASSWORD}@${POSTGRES_HOSTNAME}:${POSTGRES_PORT}/${POSTGRES_DB}?schema=public

# ----
# docker
# .env.test
# http://npb-db-test:3001/api/auth/session - cannot work as localhost...
HOSTNAME=localhost # maybe npb-app-test will work, no browser

# .env.test.local
POSTGRES_HOSTNAME=npb-db-test
DATABASE_URL=expanded

# ------------------
# cypress

# .env.e2e
HOSTNAME=npb-app-test
# Expected baseUrl to be a fully qualified URL (starting with `http://` or `https://`).
# Instead the value was: "$PROTOCOL://$HOSTNAME:$PORT"
# Cypress wont expand
CYPRESS_baseUrl=$PROTOCOL://$HOSTNAME:$PORT
# solution:
# expand it like this
CYPRESS_baseUrl=${PROTOCOL}://${HOSTNAME}:${PORT}

# .env.e2e.local
POSTGRES_HOSTNAME=npb-db-test
DATABASE_URL=expanded
  • conclusion: both docker test envs are the same, just pass them in d-c.yml as env_file, so .env.test.docker, .env.test.docker.local, next.js in docker will read vars directly without .env files
  • conclusion 2: better use single Javascript object or json for .envs
  • add d-c.e2e.yml and d-c.test.yml because of different command:
# e2e error
npb-e2e         |     1) search form filters posts by user
npb-e2e         |     ✓ pagination next and previous works (1023ms)
npb-e2e         | Deleting tables ...
npb-e2e         |     ✓ post item links (4824ms)
npb-e2e         |
npb-e2e         |
npb-e2e         |   2 passing (17s)
npb-e2e         |   1 failing
npb-e2e         |
npb-e2e         |   1) Home page
npb-e2e         |        search form filters posts by user:
npb-e2e         |      TestingLibraryElementError: Timed out retrying after 4000ms: Unable to find an accessible element with the role "link" and name `/^@user1$/i`

Explanation why waiting for db is needed beside depends-on

  • Docker Postgres docs

If there is no database when postgres starts in a container, then postgres will create the default database for you. While this is the expected behavior of postgres, this means that it will not accept incoming connections during that time.

ESlint Cypress

  • add tests-e2e/ to root .eslintignore
  • add .eslintrc.json in tests-e2e/, plugin Readme