Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate to yarn for installing dependencies #8762

Merged
merged 63 commits into from Feb 18, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
63 commits
Select commit Hold shift + click to select a range
8b436a0
migrate to yarn for install dependencies
Jan 20, 2021
6d12a3a
install emotion/react psammead peer dep
Jan 20, 2021
84a0d88
update bundle sizes
Jan 20, 2021
a8ffe06
update snapshots
Jan 20, 2021
3a34587
prevent running npm install
Jan 20, 2021
91f6d39
Merge branch 'latest' into migrate-to-yarn
Feb 2, 2021
43071df
add bolder npm install error message
Feb 2, 2021
8c836ce
test npm_execpath for testing
Feb 2, 2021
9ab623f
use yarn in GH actions
Feb 2, 2021
d81b953
make npm install error message slightly less bold
Feb 2, 2021
6c09be3
update bundle size config
Feb 2, 2021
809aac0
remove console.log
Feb 2, 2021
3bf9f74
use yarn install --frozen-lockfile
Feb 3, 2021
1f736c5
replace npm scripts commands with yarn
Feb 3, 2021
9cc523a
don't save puppeteer dep in yarn.lock
Feb 3, 2021
bd12fa2
remove unused script
Feb 3, 2021
6f8ea7c
Merge branch 'latest' into migrate-to-yarn
Feb 3, 2021
6a6addc
remove accidentally committed line of code
Feb 3, 2021
410a89c
enforce version of yarn
Feb 3, 2021
831f94c
don't save puppeteer dep in yarn.lock
Feb 3, 2021
0ed75df
remove unused script
Feb 3, 2021
8a5c63b
Merge branch 'latest' into migrate-to-yarn
Feb 3, 2021
56f0841
update contributing dox
Feb 3, 2021
93c3958
Merge branch 'migrate-to-yarn' of github.com:bbc/simorgh into migrate…
Feb 3, 2021
8a56381
loadable server and component packages must match
Feb 3, 2021
76ad6a9
later version of loadable introduces visual bugs
Feb 3, 2021
5ce7798
update int test snapshots
Feb 3, 2021
7e34ac0
Merge branch 'latest' into migrate-to-yarn
Feb 3, 2021
cd1667b
Merge branch 'migrate-to-yarn' into migrate-to-yarn-replace-npm-scripts
Feb 3, 2021
fdcb05d
Merge branch 'latest' into migrate-to-yarn
Feb 4, 2021
f86f61d
Merge branch 'migrate-to-yarn' of github.com:bbc/simorgh into migrate…
Feb 4, 2021
da93786
update contribution and link to from terminal
Feb 4, 2021
715c059
only nvm use required
Feb 4, 2021
f5ac634
add to and link to install yarn in root readme
Feb 4, 2021
31be900
add space in box
Feb 4, 2021
0b91d35
Merge branch 'latest' into migrate-to-yarn
Feb 4, 2021
5f51779
Merge branch 'latest' of github.com:bbc/simorgh into migrate-to-yarn
andrewscfc Feb 5, 2021
d522d16
update pre-push script
Feb 5, 2021
c4ba325
Merge branch 'migrate-to-yarn' of github.com:bbc/simorgh into migrate…
Feb 5, 2021
31f7a90
update yarn lock
andrewscfc Feb 5, 2021
49ed388
Merge branch 'migrate-to-yarn' of github.com:bbc/simorgh into migrate…
andrewscfc Feb 5, 2021
7b306e1
yarn import lock file
andrewscfc Feb 5, 2021
75ed4ad
update snapshots after yarn import
andrewscfc Feb 5, 2021
a8d9cc2
reset loadable changes to match latest
andrewscfc Feb 5, 2021
a24a7c3
reset bundleSizeConfig
andrewscfc Feb 5, 2021
9dae0a8
fix caching of node_modules in GH actions
Feb 10, 2021
9fdc15d
remove package-lock.json references
Feb 10, 2021
2a144ac
Merge branch 'latest' into migrate-to-yarn
Feb 12, 2021
dc63c18
update yarn.lock based on package-lock.json with yarn import command
Feb 12, 2021
f7df215
Merge pull request #8833 from bbc/migrate-to-yarn-replace-npm-scripts
Feb 12, 2021
2c077b7
remove `run` from yarn commands
Feb 12, 2021
57841a7
Merge branch 'latest' into migrate-to-yarn
Feb 15, 2021
def21db
Merge branch 'latest' into migrate-to-yarn
Feb 15, 2021
a619645
trigger build
Feb 15, 2021
4a8cac8
Merge branch 'latest' into migrate-to-yarn
Feb 16, 2021
3791687
remove @emotion/sheet webpack alias
Feb 16, 2021
2eb5ea7
match all emotion package versions
Feb 16, 2021
57c2d7e
upgrade emotion packages to latest
Feb 16, 2021
41f922e
update renovate docs to remove reference to forcing resolutions
andrewscfc Feb 17, 2021
444c512
Merge branch 'latest' into migrate-to-yarn
Feb 18, 2021
12c9d6e
use latest versions of emotion packages in prod bundle
Feb 18, 2021
ad1be8a
add yarn ci script to remove invalid transitive dependencies before n…
Feb 18, 2021
014cd07
move transitive emotion deps to resolutions
Feb 18, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/PULL_REQUEST_TEMPLATE.md
Expand Up @@ -17,5 +17,5 @@ _A very high-level summary of easily-reproducible changes that can be understood
**Testing:**

- [ ] Automated (jest and/or cypress) tests added (for new features) or updated (for existing features)
- [ ] If necessary, I have run the local E2E non-smoke tests relevant to my changes (`CYPRESS_APP_ENV=local CYPRESS_SMOKE=false npm run test:e2e:interactive`)
- [ ] If necessary, I have run the local E2E non-smoke tests relevant to my changes (`CYPRESS_APP_ENV=local CYPRESS_SMOKE=false yarn test:e2e:interactive`)
- [ ] This PR requires manual testing
6 changes: 3 additions & 3 deletions .github/workflows/simorgh-deploy-storybook.yml
Expand Up @@ -21,15 +21,15 @@ jobs:
uses: actions/cache@v2
with:
path: node_modules
key: node-modules-${{ hashFiles('package-lock.json') }}
key: node-modules-${{ hashFiles('yarn.lock') }}

- name: Install Node Modules
if: steps.cache.outputs.cache-hit != 'true'
run: npm ci
run: yarn ci

- name: Build Storybook
run: |
npm run build:storybook
yarn build:storybook
git config --global user.name "simorgh-bbc"
git config --global user.email "DENewsSimorghDev@bbc.co.uk"

Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/simorgh-e2e-tests.yml
Expand Up @@ -25,5 +25,5 @@ jobs:
- name: Run Simorgh E2Es
uses: cypress-io/github-action@v2
with:
build: npm run build
start: npm start
build: yarn build
start: yarn start
6 changes: 3 additions & 3 deletions .github/workflows/simorgh-integration-tests.yml
Expand Up @@ -32,11 +32,11 @@ jobs:
uses: actions/cache@v2
with:
path: node_modules
key: node-modules-${{ hashFiles('package-lock.json') }}
key: node-modules-${{ hashFiles('yarn.lock') }}

- name: Install Node Modules
if: steps.cache.outputs.cache-hit != 'true'
run: npm ci
run: yarn ci

- name: Integration Tests
run: npm run test:integration -- --ci
run: yarn test:integration -- --ci
17 changes: 9 additions & 8 deletions .github/workflows/simorgh-local-server-tests.yml
Expand Up @@ -31,28 +31,29 @@ jobs:
uses: actions/cache@v2
with:
path: node_modules
key: node-modules-${{ hashFiles('package-lock.json') }}
key: node-modules-${{ hashFiles('yarn.lock') }}

- name: Install Node Modules
if: steps.cache.outputs.cache-hit != 'true'
run: npm ci
run: yarn ci

- name: Build Simorgh
run: npm run build:local
run: yarn build:local

- name: Start Simorgh Server
run: nohup node build/server.js > /dev/null 2>&1 &

- name: Run AMP Validator
run: npm run amp:validate
run: yarn amp:validate

- name: Run bbc-a11y
run: npm run bbcA11y:ci
run: yarn bbcA11y:ci

- name: Run Lighthouse
run: npm run lighthouse
run: yarn lighthouse

- name: Run Puppeteer Tests
run: |
npm install --no-save puppeteer
npm run test:puppeteer
yarn add puppeteer
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We might be able to split this into installing puppeteer and running the tests. That way we can skip the puppeteer installation if it's in the cache.

yarn test:puppeteer
yarn remove puppeteer
8 changes: 4 additions & 4 deletions .github/workflows/simorgh-misc-checks.yml
Expand Up @@ -33,11 +33,11 @@ jobs:
uses: actions/cache@v2
with:
path: node_modules
key: node-modules-${{ hashFiles('package-lock.json') }}
key: node-modules-${{ hashFiles('yarn.lock') }}

- name: Install Node Modules
if: steps.cache.outputs.cache-hit != 'true'
run: npm ci
run: yarn ci

- name: Chromatic UI Tests
uses: chromaui/action@v1
Expand All @@ -62,10 +62,10 @@ jobs:
run: npx apache2-license-checker

- name: Duplicate Dependency Checker
run: npm run test:dependencies
run: yarn test:dependencies

- name: Linting
run: npm run test:lint
run: yarn test:lint

- name: NPM Audit
run: npx audit-ci --config audit-ci.json
8 changes: 4 additions & 4 deletions .github/workflows/simorgh-unit-tests.yml
Expand Up @@ -43,16 +43,16 @@ jobs:
uses: actions/cache@v2
with:
path: node_modules
key: node-modules-${{ hashFiles('package-lock.json') }}
key: node-modules-${{ hashFiles('yarn.lock') }}

- name: Install Node Modules
if: steps.cache.outputs.cache-hit != 'true'
run: npm ci
run: yarn ci

- name: Build Simorgh
run: |
echo ${GITHUB_REF##*/}
npm run build
yarn build

- name: Setup Code Climate Test Coverage
if: ${{ github.event_name == 'push' || github.event.pull_request.head.repo.full_name == 'bbc/simorgh' }} # Only run if PR originates from the Simorgh repo
Expand All @@ -64,7 +64,7 @@ jobs:
./cc-test-reporter before-build

- name: Unit Tests
run: npm run test:unit
run: yarn test:unit

- name: Report Code Climate Test Coverage
if: ${{ github.event_name == 'push' || github.event.pull_request.head.repo.full_name == 'bbc/simorgh' }} # Only run if PR originates from the Simorgh repo
Expand Down
1 change: 0 additions & 1 deletion .npmrc

This file was deleted.

1 change: 1 addition & 0 deletions .yarnrc
@@ -0,0 +1 @@
version=1.22.10
12 changes: 7 additions & 5 deletions AdHocCypress/README.md
@@ -1,17 +1,19 @@
# Ad Hoc Cypress Tests

## General approach to end-to-end (E2E) testing

This is documented in [the README in the primary directory of our cypress tests](https://github.com/bbc/simorgh/blob/latest/cypress/README.md).

## This suite of tests

The purpose of this test suite is to allow for testing of ad-hoc functionality within the Simorgh application, without affecting the tests which are executed as part of the End-to-End tests in both the Test and Live environments.

## Running Ad Hoc Tests

| Environment | Command |
| ----------- | --------------------------------------------- |
| local | `CYPRESS_APP_ENV=local npm run cypress:adhoc` |
| test | `CYPRESS_APP_ENV=test npm run cypress:adhoc` |
| live | `CYPRESS_APP_ENV=live npm run cypress:adhoc` |
| Environment | Command |
| ----------- | ------------------------------------------ |
| local | `CYPRESS_APP_ENV=local yarn cypress:adhoc` |
| test | `CYPRESS_APP_ENV=test yarn cypress:adhoc` |
| live | `CYPRESS_APP_ENV=live yarn cypress:adhoc` |

As with the primary Cypress tests, the same [environment variables](https://github.com/bbc/simorgh#environment-variables) can also be used with the `cypress:adhoc` command
53 changes: 39 additions & 14 deletions CONTRIBUTING.md
Expand Up @@ -36,19 +36,44 @@ If you never created a pull request before, [here is a great tutorial on how to

Fork the project, clone your fork, configure the remotes and install the dependencies:

1. Clone your fork of the repo into the current directory

```
git clone https://github.com/<YOUR ACCOUNT>/simorgh.git
```

2. Navigate to the newly cloned directory

```
cd simorgh
```

3. Assign the original repo to a remote called "upstream"

```
git remote add upstream https://github.com/bbc/simorgh
```

4. Use recommended version of node for the project, listed in [`.nvmrc`](https://github.com/bbc/simorgh/blob/latest/.nvmrc)

```
nvm use
```

### Installing dependencies

**Install Yarn**

The Simorgh project uses Yarn for package management. To install Yarn, run this command:

```
npm install --global yarn
```

**Install the project dependencies with Yarn**

```
# Clone your fork of the repo into the current directory
git clone https://github.com/<YOUR ACCOUNT>/simorgh.git
# Navigate to the newly cloned directory
cd simorgh
# Assign the original repo to a remote called "upstream"
git remote add upstream https://github.com/bbc/simorgh
# Use recommended version of node for the project, listed in `.nvmrc`
nvm use
# Install latest version of npm
npm install --global npm@latest
# Install the project dependencies
npm install
yarn install
```

If it has been a while since you've cloned the repo, pull from the `latest` branch
Expand Down Expand Up @@ -119,7 +144,7 @@ Pick a JSON file under `data/news/articles/[id].json`, and:
1. add an example of your block somewhere in the `content.model.blocks` array.
2. add your new component to the `blockTypes` array.

Run `npm run dev` and you should see your component at your article of choice, eg http://localhost:7080/news/articles/c0000000001o
Run `yarn dev` and you should see your component at your article of choice, eg http://localhost:7080/news/articles/c0000000001o

#### Update the schema

Expand Down Expand Up @@ -183,7 +208,7 @@ If on the other hand you are adding new files to the root directory of simorgh,

### `.env` is showing in my `git status`

The `.env` file should not be commited as it is often overwritten by the values in `envConfig/` at build time. There is a `postshrinkwrap` command which runs after an `npm install` so should be run during setup of the application.
The `.env` file should not be commited as it is often overwritten by the values in `envConfig/` at build time. There is a `postshrinkwrap` command which runs after an `yarn install` so should be run during setup of the application.

If the `.env` file is appearing in your `git status` it means it is now longer being assumed as unchanged, to fix this run:

Expand Down
44 changes: 26 additions & 18 deletions README.md
Expand Up @@ -158,20 +158,28 @@ Install Node. [https://nodejs.org/en/](https://nodejs.org/en/). We use the versi
nvm use
```

#### Install Yarn

The Simorgh project uses Yarn for package management. It is recommended to install Yarn through the npm package manager, which comes bundled with Node.js when you install it on your system. To install Yarn, run this command:

```
npm install --global yarn
```

Then you can run the following commands to install Simorgh

```
git clone git@github.com:bbc/simorgh.git
cd simorgh
npm install
yarn install
```

## Local Development

To run this application locally, with hot-reloading, run

```
npm run dev
yarn dev
```

The application will start on [http://localhost:7080](http://localhost:7080).
Expand Down Expand Up @@ -220,7 +228,7 @@ You can find other pages types by looking through our routes and their associate

We use Storybook for developing components in isolation from the Simorgh Application. You can access this at [https://bbc.github.io/simorgh/](https://bbc.github.io/simorgh/)

To run locally `npm run storybook`, it will then be available at [http://localhost:9001/](http://localhost:9001/). Introduction to and documentation for Storybook is here: [https://storybook.js.org/basics/introduction/](https://storybook.js.org/basics/introduction/).
To run locally `yarn storybook`, it will then be available at [http://localhost:9001/](http://localhost:9001/). Introduction to and documentation for Storybook is here: [https://storybook.js.org/basics/introduction/](https://storybook.js.org/basics/introduction/).

When viewing Video stories locally, make sure to use a BBC domain, as outlined in the [changing request location section](https://github.com/bbc/simorgh#changing-request-location). Video will not work in the hosted version of Storybook linked above for this reason.

Expand All @@ -231,9 +239,9 @@ Please also note that if you would like to see the components rendered with our
## Production build locally

To run this application locally with a production build, run:
`npm run build && npm run start`.
`yarn build && yarn start`.

We use `npm run build` locally which bundles the application pointing at localhost for data and static assets.
We use `yarn build` locally which bundles the application pointing at localhost for data and static assets.

## Using environment builds locally

Expand All @@ -243,14 +251,14 @@ To run TEST bundles on localhost:

- In `envConfig/test.env` change the values of:
- `LOG_DIR='/var/log/simorgh'` to `LOG_DIR='log'`
- Then run `rm -rf build && npm run build:test && npm run start`
- Then run `rm -rf build && yarn build:test && yarn start`
- Visit a test article: http://localhost:7080/news/articles/c0g992jmmkko

To run LIVE bundles on localhost:

- In `envConfig/live.env` change the values of:
- `LOG_DIR='/var/log/simorgh'` to `LOG_DIR='log'`
- Then run `rm -rf build && npm run build:live && npm run start`
- Then run `rm -rf build && yarn build:live && yarn start`
- Visit a live article: http://localhost:7080/news/articles/c8xxl4l3dzeo

## Changing request location
Expand All @@ -273,15 +281,15 @@ On deployment `make buildCi` is run in the CI environment which creates bundles

### Bundle analysis reports

Every run of `npm run build` will update the bundle analysis files in the repo. To view a breakdown of the bundle size, open the generated html report in a browser `./reports/webpackBundleReport.html` This is generated via `webpack-bundle-analyzer`. The data is also available as json `./reports/webpackBundleReport.json`.
Every run of `yarn build` will update the bundle analysis files in the repo. To view a breakdown of the bundle size, open the generated html report in a browser `./reports/webpackBundleReport.html` This is generated via `webpack-bundle-analyzer`. The data is also available as json `./reports/webpackBundleReport.json`.

## Tests

### Linting and unit tests

We have linting with the [Airbnb styleguide](https://github.com/airbnb/javascript/tree/master/react) and we use [Prettier](https://github.com/prettier/prettier) as a code formatter. They can be run with `npm run test:lint`.
We have linting with the [Airbnb styleguide](https://github.com/airbnb/javascript/tree/master/react) and we use [Prettier](https://github.com/prettier/prettier) as a code formatter. They can be run with `yarn test:lint`.

We have [Jest](https://facebook.github.io/jest) unit tests that can be run with `npm run test:unit`.
We have [Jest](https://facebook.github.io/jest) unit tests that can be run with `yarn test:unit`.

`npm test` runs both sets of these.

Expand All @@ -292,14 +300,14 @@ We have [Jest](https://facebook.github.io/jest) unit tests that can be run with
We use [Cypress](https://www.cypress.io/) for our end-to-end tests. To run the [smoke tests](https://github.com/bbc/simorgh/tree/latest/cypress/integration#how-our-cypress-tests-work) locally, run this single command:

```
npm run test:e2e
yarn test:e2e
```

It will spin up a production server on port 7080 and run the Cypress tests against that.
To run the smoke tests interactively, run:

```
npm run test:e2e:interactive
yarn test:e2e:interactive
```

This loads a user interface which easily allows for individual tests to be run alongside a visual stream of the browser, as the tests run.
Expand All @@ -320,16 +328,16 @@ These commands can be run in combination.

#### Full suite of tests

The default way to run the e2e suite aka `npm run test:e2e` or `npm run test:e2e:interactive` runs a subset of our tests, otherwise know as _smoke tests_. To run the full suite:
The default way to run the e2e suite aka `yarn test:e2e` or `yarn test:e2e:interactive` runs a subset of our tests, otherwise know as _smoke tests_. To run the full suite:

`CYPRESS_SMOKE=false npm run test:e2e`
`CYPRESS_SMOKE=false yarn test:e2e`

#### Limiting scope of runs

Tests can be restricted to only run for a single service by specifying it using the `CYPRESS_ONLY_SERVICE` environment variable. For example:

```
CYPRESS_ONLY_SERVICE=urdu npm run test:e2e
CYPRESS_ONLY_SERVICE=urdu yarn test:e2e
```

To run only a particular spec it is necessary to invoke Cypress directly. First ensure Simorgh is already running in another tab and then run (for example, to only run article tests):
Expand All @@ -349,7 +357,7 @@ Cypress .visit() function is locked to visiting a single domain per test. This b
Here is an example command:

```
CYPRESS_APP_ENV=test CYPRESS_UK=true CYPRESS_SMOKE=true npm run cypress
CYPRESS_APP_ENV=test CYPRESS_UK=true CYPRESS_SMOKE=true yarn cypress
```

#### Running e2e outside EU
Expand All @@ -361,13 +369,13 @@ Running Cypress tests outside the EU will not show the EU consent banners on AMP
An example command will be:

```
CYPRESS_SKIP_EU=true npm run cypress:interactive
CYPRESS_SKIP_EU=true yarn cypress:interactive
```

The following command runs both simorgh and cypress:

```
CYPRESS_APP_ENV=local CYPRESS_UK=true CYPRESS_SMOKE=true npm run test:e2e
CYPRESS_APP_ENV=local CYPRESS_UK=true CYPRESS_SMOKE=true yarn test:e2e
```

CYPRESS_APP_ENV can also be set equal to 'test' and 'live'.
Expand Down