Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Removing flakiness using .json() to check JSON string responses #96

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

akshathphillips
Copy link

What is the purpose of this PR

  • This PR fixes the error resulting from a flaky test: com.didispace.chapter21.Chapter21ApplicationTests
  • The mentioned test may fail or pass without changes made to the source code when it is run depending on the JSON's non-deterministic deserialization.

Why the tests fail

  • This test fails because when the JSON's response is deserialized to a String.class, the order of the fields inside the object may change to be in random positions. For example, {"id":1, "age":20}, or {"age":20, "id":1}

Reproduce the test failure

  • Run the tests with NonDex maven plugin. The commands to recreate the flaky test failures are:
    • Run the test with the NonDex maven plugin
      • mvn -pl 2.x/chapter2-1 test edu.illinois:nondex-maven-plugin:2.1.1:nondex -Dtest=com.didispace.chapter21.Chapter21ApplicationTests#testUserController -DnondexRuns=10
  • Fixing the flaky test now may prevent flaky test failures for users checking out this project.

Expected results

  • The test should run successfully when run with NonDex.

Actual Result

We get the following failure for test com.didispace.chapter21.Chapter21ApplicationTests

[ERROR] Failures: 
[ERROR]   Chapter21ApplicationTests.testUserController:52 Response content
Expected: "[{\"id\":1,\"name\":\"测试大师\",\"age\":20}]"
     but: was "[{\"age\":20,\"id\":1,\"name\":\"测试大师\"}]"

Fix

For test com.didispace.chapter21.Chapter21ApplicationTests - Parse the expected and actual strings as JSON and assert the two are "similar" - i.e. they contain the same attribute-value pairs regardless of formatting with a lenient checking. The .json(), does a lenient checking on the fields, which solves this issue.

…rt the two are "similar" - i.e. they contain the same attribute-value pairs regardless of formatting with a lenient checking
@xiaofeiahaha
Copy link

xiaofeiahaha commented May 2, 2023 via email

@liuhamglin
Copy link

liuhamglin commented May 2, 2023 via email

@loquitowen
Copy link

loquitowen commented May 2, 2023 via email

@akshathphillips
Copy link
Author

Hello. Any updates on this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants