Skip to content

Commit

Permalink
chore: reduce flake in windows-run-app-integration-tests-chrome by no…
Browse files Browse the repository at this point in the history
…t testing certain tests that are flaky inside the pap-integration cy-in-cy pipeline. Because there are issues recording with --spec specified with cy-in-cy, we will not be recording the windows app integration tests. Additionally, this fixes the windows-test-binary-against-kitchensink-chrome where yarn is not installed on the windows-server executor by making sure yarn is installed first before any action happens when checking out a repo [run ci] (#29198)
  • Loading branch information
AtofStryker committed Apr 2, 2024
1 parent 4d97a0f commit 339f64b
Show file tree
Hide file tree
Showing 2 changed files with 35 additions and 11 deletions.
2 changes: 1 addition & 1 deletion .circleci/cache-version.txt
@@ -1,3 +1,3 @@
# Bump this version to force CI to re-create the cache from scratch.

03-12-24
03-25-24
44 changes: 34 additions & 10 deletions .circleci/workflows.yml
Expand Up @@ -84,7 +84,7 @@ windowsWorkflowFilters: &windows-workflow-filters
- equal: [ 'cacie/dep/electron-27', << pipeline.git.branch >> ]
- equal: [ 'feat/protocol_shadow_dom_support', << pipeline.git.branch >> ]
- equal: [ 'lerna-optimize-tasks', << pipeline.git.branch >> ]
- equal: [ 'mschile/mochaEvents_win_sep', << pipeline.git.branch >> ]
- equal: [ 'chore/reduce_windows_flake', << pipeline.git.branch >> ]
- matches:
pattern: /^release\/\d+\.\d+\.\d+$/
value: << pipeline.git.branch >>
Expand Down Expand Up @@ -125,7 +125,7 @@ executors:
# https://circleci.com/docs/2.0/hello-world-windows/#software-pre-installed-in-the-windows-image
windows: &windows-executor
machine:
image: windows-server-2019-vs2019:stable
image: windows-server-2022-gui:stable
shell: bash.exe -eo pipefail
resource_class: windows.large
environment:
Expand Down Expand Up @@ -666,6 +666,11 @@ commands:
browser: <<parameters.browser>>
- run:
command: |
if [[ $PLATFORM == 'windows' && '<<parameters.browser>>' == 'chrome' && '<<parameters.package>>' == 'app' && '<<parameters.type>>' == 'e2e' ]]; then
IS_WINDOWS_APP_INTEGRATION_TEST=true
else
IS_WINDOWS_APP_INTEGRATION_TEST=false
fi
echo Current working directory is $PWD
echo Total containers $CIRCLE_NODE_TOTAL
Expand All @@ -675,14 +680,23 @@ commands:
fi
# internal PR
cmd=$([[ <<parameters.percy>> == 'true' ]] && echo 'yarn percy exec --parallel -- --') || true
DEBUG=<<parameters.debug>> \
CYPRESS_CONFIG_ENV=production \
CYPRESS_RECORD_KEY=$MAIN_RECORD_KEY \
PERCY_PARALLEL_NONCE=$CIRCLE_WORKFLOW_WORKSPACE_ID \
PERCY_ENABLE=${PERCY_TOKEN:-0} \
PERCY_PARALLEL_TOTAL=-1 \
CYPRESS_INTERNAL_ENABLE_TELEMETRY="true" \
$cmd yarn workspace @packages/<<parameters.package>> cypress:run:<<parameters.type>> --browser <<parameters.browser>> --record --parallel --group <<parameters.package>>-<<parameters.type>>
if [[ "$IS_WINDOWS_APP_INTEGRATION_TEST" = true ]]; then
echo "on windows running app-integration tests. Skipping flaky tests"
# if windows app integration tests, skip some very flaky tests that fail to load for undetermined reasons
TESTFILES=$(cd packages/<<parameters.package>> && /usr/bin/find cypress/e2e -regextype posix-extended -name '*.cy.*' -not -regex '.*(experimentalRetries|reporter.command_errors|ct-framework-errors|reporter-ct-vite|reporter-ct-webpack|reporter.errors|reporter.hooks|cypress-in-cypress|runner.ui|specs|studio).*' | circleci tests split --total=$CIRCLE_NODE_TOTAL)
# Do NOT record on windows packages/app due to encryption issues
$cmd yarn workspace @packages/<<parameters.package>> cypress:run:<<parameters.type>> --browser <<parameters.browser>> --spec $TESTFILES
else
DEBUG=<<parameters.debug>> \
CYPRESS_CONFIG_ENV=production \
CYPRESS_RECORD_KEY=$MAIN_RECORD_KEY \
PERCY_PARALLEL_NONCE=$CIRCLE_WORKFLOW_WORKSPACE_ID \
PERCY_ENABLE=${PERCY_TOKEN:-0} \
PERCY_PARALLEL_TOTAL=-1 \
CYPRESS_INTERNAL_ENABLE_TELEMETRY="true" \
$cmd yarn workspace @packages/<<parameters.package>> cypress:run:<<parameters.type>> --browser <<parameters.browser>> --record --parallel --group <<parameters.package>>-<<parameters.type>>
fi
else
# external PR
Expand All @@ -703,6 +717,12 @@ commands:
# To run the `yarn` command, we need to walk out of the package folder.
cd ../..
if [[ "$IS_WINDOWS_APP_INTEGRATION_TEST" = true ]]; then
echo "on windows running app-integration tests. Skipping flaky tests"
# if windows app integration tests, skip some very flaky tests that fail to load for undetermined reasons
TESTFILES=$(cd packages/<<parameters.package>> && /usr/bin/find cypress/e2e -regextype posix-extended -name '*.cy.*' -not -regex '.*(experimentalRetries|reporter.command_errors|ct-framework-errors|reporter-ct-vite|reporter-ct-webpack|reporter.errors|reporter.hooks|cypress-in-cypress|runner.ui|specs|studio).*' | circleci tests split --total=$CIRCLE_NODE_TOTAL)
fi
DEBUG=<<parameters.debug>> \
CYPRESS_CONFIG_ENV=production \
PERCY_PARALLEL_NONCE=$CIRCLE_WORKFLOW_WORKSPACE_ID \
Expand Down Expand Up @@ -1010,6 +1030,10 @@ commands:
type: string
default: "npm start --if-present"
steps:
- run:
name: Install yarn if not already installed
command: |
yarn --version || npm i -g yarn
- clone-repo-and-checkout-branch:
repo: <<parameters.repo>>
pull_request_id: <<parameters.pull_request_id>>
Expand Down

4 comments on commit 339f64b

@cypress-bot
Copy link
Contributor

@cypress-bot cypress-bot bot commented on 339f64b Apr 2, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Circle has built the linux arm64 version of the Test Runner.

Learn more about this pre-release build at https://on.cypress.io/advanced-installation#Install-pre-release-version

Run this command to install the pre-release locally:

npm install https://cdn.cypress.io/beta/npm/13.7.2/linux-arm64/develop-339f64b15e6584e85050d40c4ce768ced2486d00/cypress.tgz

@cypress-bot
Copy link
Contributor

@cypress-bot cypress-bot bot commented on 339f64b Apr 2, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Circle has built the linux x64 version of the Test Runner.

Learn more about this pre-release build at https://on.cypress.io/advanced-installation#Install-pre-release-version

Run this command to install the pre-release locally:

npm install https://cdn.cypress.io/beta/npm/13.7.2/linux-x64/develop-339f64b15e6584e85050d40c4ce768ced2486d00/cypress.tgz

@cypress-bot
Copy link
Contributor

@cypress-bot cypress-bot bot commented on 339f64b Apr 2, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Circle has built the darwin arm64 version of the Test Runner.

Learn more about this pre-release build at https://on.cypress.io/advanced-installation#Install-pre-release-version

Run this command to install the pre-release locally:

npm install https://cdn.cypress.io/beta/npm/13.7.2/darwin-arm64/develop-339f64b15e6584e85050d40c4ce768ced2486d00/cypress.tgz

@cypress-bot
Copy link
Contributor

@cypress-bot cypress-bot bot commented on 339f64b Apr 2, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Circle has built the darwin x64 version of the Test Runner.

Learn more about this pre-release build at https://on.cypress.io/advanced-installation#Install-pre-release-version

Run this command to install the pre-release locally:

npm install https://cdn.cypress.io/beta/npm/13.7.2/darwin-x64/develop-339f64b15e6584e85050d40c4ce768ced2486d00/cypress.tgz

Please sign in to comment.