Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: convert projects to Playwright #54689

Merged

Conversation

Sembauke
Copy link
Member

@Sembauke Sembauke commented May 7, 2024

Checklist:

Closes #XXXXX

@github-actions github-actions bot added platform: learn UI side of the client application that needs familiarity with React, Gatsby etc. scope: tools/scripts Scripts for supporting dev work, generating config and build artifacts, etc. labels May 7, 2024
e2e/projects.spec.ts Outdated Show resolved Hide resolved
@Sembauke Sembauke force-pushed the feat/convert-projects-to-playwright branch 2 times, most recently from 3eb4aef to f8f736f Compare May 10, 2024 13:21
@Sembauke
Copy link
Member Author

It seems something weird is going on. When seeding development user in the test everything seems to work as intended in that particular test. This is not the case for other tests. Example the settings spec seems to receive development user instead of the certified user e.g.: Account Settings for developmentuser instead of Account Settings for certifieduser. So the database modifications seem to leak into the other tests.

Not only that, but in some cases the user is logged out entirely. Bummer.

@huyenltnguyen huyenltnguyen force-pushed the feat/convert-projects-to-playwright branch from b795563 to 5c4edb8 Compare May 10, 2024 18:26
@huyenltnguyen
Copy link
Member

I was able to make some progress (see 5c4edb8).

  • I removed the test.use() from the JS test suite, since it's already set globally. I was able to run the tests locally as a signed in user, but I'm not sure if that's because of this change
  • I created new util functions for the editor. The difference is, on project pages and in mobile view, the code editor is hidden in the Code tab, and the user would need to switch to the tab to access the editor. This isn't the case with practice challenge as the editor is available when the user accesses the page.
    • Note: We could either merge the utils into a single file (the existing editor.ts) or keep them separate in challenge-editor.ts and project-editor.ts.
  • I changed the editor.evaluate() part with just await editor.fill(contents)
  • The tests are still failing because there is a stray } in the editor, causing the code to be invalid and blocking the submission. The solution code in the JSON file looks correct to me, so I'm still (pulling my hair) trying to figure out where that character comes from (both editor.evaluate() and editor.fill() gave the same result).
    • Screenshot 2024-05-11 at 01 08 38

@Sembauke
Copy link
Member Author

Sembauke commented May 10, 2024

The tests are still failing because there is a stray } in the editor, causing the code to be invalid and blocking the submission. The solution code in the JSON file looks correct to me, so I'm still (pulling my hair) trying to figure out where that character comes from (both editor.evaluate() and editor.fill() gave the same result).

You deleted the parameter which disables bracket and quote completion in the editor.

image

When pasting or filling code, the editor will complete brackets or quotes. This function prevents that from happening, it looks if a certain parameter is present. Then disables them.

    // disable bracket completion when testing as it duplicates brackets when filling text
    const disableBracketCompletion =
      window.location.href.includes('testing=true');

    console.log(window.location.href, disableBracketCompletion);

    if (disableBracketCompletion) {
      editor?.updateOptions({
        autoClosingBrackets: 'never',
        autoClosingQuotes: 'never'
      });
    }

I should have left a comment about it.

@huyenltnguyen
Copy link
Member

I tried replicating the test flow in CI locally by seeding the DB with seed:certified-user first, then run the tests.

I got the following error in Playwright:

Screenshot 2024-05-13 at 15 45 36

Not sure if it has something to do with cookies since the test re-seeds the DB with dev user data.

@huyenltnguyen
Copy link
Member

I'll try using the .clearCookies() method to see if it could resolve the issue.

But maybe the last resort is setting up a new workflow that runs tests with the development user data (#52905 (comment)).

@ojeytonwilliams
Copy link
Contributor

@huyenltnguyen yep, it's the cookies. The issue is that the cookies for certifieduser are used even after the db is seeded with developmentuser. The fix/workaround is to create storageStates for both so that you can switch between users.

I'll create a quick PR for that, so you can use it here.

@ojeytonwilliams
Copy link
Contributor

#54752 should help.

Once it's in, switching users means both changing storage state and seeding the desired user.

@huyenltnguyen
Copy link
Member

huyenltnguyen commented May 15, 2024

I made some changes and cleaned up the tests a bit. The main changes are:

  • Use Meta+V for the paste action on MacOS (13d9e27ad77a4735cca6e25f2f46e241c365998a)
  • Call authedPost concurrently (618d892)
    • Technically the change should improve the speed, but our server is also local so the API calls are already fast and there isn't much to improve. Still, I think the API calls should be concurrent rather than sequential, so think of this as a refactoring change rather than a speed improvement change 😄
  • Replace .insertText() with copy-pasting in the test case for completion modal (0964b5b)
    • Filling the editor with .insertText() takes ~10s on my machine
      • Screenshot 2024-05-16 at 01 17 27
    • Side note: This test uses the Control+Enter keyboard combination, and it works fine on my machine, so I'm quite confused as to why Control+V isn't registered.
  • Move the test.use(), test.beforeAll(), and test.afterAll() blocks to the global scope (277d197be5680e6aedcf8bef6948a21a6cca1efc)
    • The test.use() relocation is to ensure the tests are run with an authenticated user
    • The test.beforeAll() relocation is to ensure all tests are run with the Development User data
    • The test.afterAll() relocation is to ensure the change to the seed data is reverted after the file is run (rather than after the "Should not be able to submit in quick succesion" test is run)
  • Remove a similar test Cypress case (0f79f41c195110bae6876560a5dcc6f9a4a27675)
    • The Cypress test case is quite similar to the Legacy JS case we're adding. What's difference is the Cypress one tests claim cert flow using the Front End Development Libraries cert.
    • I only removed a test case and not the entire file, since we need to ensure we have a corresponding Playwright test for the other case first (which is out of the scope of this PR)

Notes for morning me:

  • The utils/editor.ts file should be renamed to challenge-editor.ts, to prevent confusion
  • Update the page.fill query in the "should not be possible to submit twice in quick succession" case to use getByLabel

@Sembauke Sembauke marked this pull request as ready for review May 16, 2024 06:40
Copy link
Member

@huyenltnguyen huyenltnguyen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hookay, I think the PR is good to go.

This is one heck of a test suite to write / rewrite 😰

@huyenltnguyen huyenltnguyen added the status: waiting review To be applied to PR's that are ready for QA, especially when additional review is pending. label May 16, 2024
@huyenltnguyen huyenltnguyen mentioned this pull request May 20, 2024
57 tasks
huyenltnguyen and others added 3 commits May 21, 2024 14:05
It works for both projects and normal lessons, so there's no need to
handle them separately.
Copy link
Contributor

@ojeytonwilliams ojeytonwilliams left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was a struggle. Nice work, @Sembauke and @huyenltnguyen!

LGTM 👍

@ojeytonwilliams ojeytonwilliams merged commit c1e0494 into freeCodeCamp:main May 21, 2024
21 checks passed
@Sembauke Sembauke deleted the feat/convert-projects-to-playwright branch May 21, 2024 17:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
platform: learn UI side of the client application that needs familiarity with React, Gatsby etc. scope: tools/scripts Scripts for supporting dev work, generating config and build artifacts, etc. status: waiting review To be applied to PR's that are ready for QA, especially when additional review is pending.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants