Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Changed HashMap to LinkedHashMap for deterministic order (description,value,key) #1820

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

RugvedB
Copy link

@RugvedB RugvedB commented Nov 29, 2023

Note: This PR fixes all the tests failures (19) mentioned in PR: https://github.com/mock-server/mockserver/pull/1817/files in addition to the test failures(8) mentioned in this PR i.e total tests fixed = 27

Created this PR to fix 8 flaky tests listed as follows:

  1. shouldSerialiseForwardedRequests
  2. shouldSerialiseEventsWithRequestFilter
  3. shouldSerialiseRollUpEventsWithCorrelationId
  4. shouldSerialiseRecordedRequestsEventsWithoutFields
  5. shouldSerialiseEventsWithNoRequestFilter
  6. shouldSerialiseForwardedRequestsForEventsWithoutFields
  7. shouldSerialiseEventsAndIgnoreDeletedLogEvents
  8. shouldSerialiseRecordedRequests

  1. How were these test identified as flaky?
    These tests were identified as flaky by using an open-source research tool named NonDex which is responsible for finding and diagnosing non-deterministic runtime exceptions in Java programs.

  2. What was the error?
    The error was caused by non-deterministic ordering of keys in the entry map here and here. The tests were asserting the exact order of keys in the generated JSON in the following order: description, value, key. Since the implementation was using a HashMap to create the entry map, the order of keys was not guaranteed, leading to different orders during test execution.

Common Error among all the tests mentioned below - jumbled ordering of entry map's keys (i.e description, value, key) in the json:

Expected: is "{\n  \"logMessages\" : [ {\n    \"key\" : \"1a9d2b00-e37e-464f-91df-86c5167bf816_log\",\n    \"value\" : {\n      \"description\" : \"11-27 13:05:49.318 FORWARDED_REQUEST   \",\n      \"style\" : {\n        \"paddingBottom\" : \"4px\",\n        \"whiteSpace\" : \"nowrap\",\n        \"overflow\" : \"auto\",\n        \"color\" : \"rgb(152, 208, 255)\",\n        \"paddingTop\" : \"4px\"\n      },\n      \"messageParts\" : [ {\n        \"key\" : \"1a9d2b00-e37e-464f-91df-86c5167bf816_0msg\",\n        \"value\" : \"messageFormatFour\"\n      } ]\n    }\n  }, {\n    \"key\" : \"34773a6e-987a-488f-b4a7-60772b5d02e3_log\",\n    \"value\" : {\n      \"description\" : \"11-27 13:05:49.318 FORWARDED_REQUEST   \",\n      \"style\" : {\n        \"paddingBottom\" : \"4px\",\n        \"whiteSpace\" : \"nowrap\",\n        \"overflow\" : \"auto\",\n        \"color\" : \"rgb(152, 208, 255)\",\n        \"paddingTop\" : \"4px\"\n      },\n      \"messageParts\" : [ {\n        \"key\" : \"34773a6e-987a-488f-b4a7-60772b5d02e3_0msg\",\n        \"value\" : \"messageFormatThree\"\n      } ]\n    }\n  }, {\n    \"key\" : \"83b73925-6179-4a94-b703-22aca2677438_log\",\n    \"value\" : {\n      \"description\" : \"11-27 13:05:49.318 FORWARDED_REQUEST   \",\n      \"style\" : {\n        \"paddingBottom\" : \"4px\",\n        \"whiteSpace\" : \"nowrap\",\n        \"overflow\" : \"auto\",\n        \"color\" : \"rgb(152, 208, 255)\",\n        \"paddingTop\" : \"4px\"\n      },\n      \"messageParts\" : [ {\n        \"key\" : \"83b73925-6179-4a94-b703-22aca2677438_0msg\",\n        \"value\" : \"messageFormatTwo\"\n      } ]\n    }\n  }, {\n    \"key\" : \"76af75f7-12ac-49da-9790-f05f65b991b0_log\",\n    \"value\" : {\n      \"description\" : \"11-27 13:05:49.315 FORWARDED_REQUEST   \",\n      \"style\" : {\n        \"paddingBottom\" : \"4px\",\n        \"whiteSpace\" : \"nowrap\",\n        \"overflow\" : \"auto\",\n        \"color\" : \"rgb(152, 208, 255)\",\n        \"paddingTop\" : \"4px\"\n      },\n      \"messageParts\" : [ {\n        \"key\" : \"76af75f7-12ac-49da-9790-f05f65b991b0_0msg\",\n        \"value\" : \"messageFormatOne\"\n      } ]\n    }\n  } ],\n  \"proxiedRequests\" : [ {\n    \"description\" : \"   four\",\n    \"value\" : {\n      \"httpRequest\" : {\n        \"path\" : \"four\"\n      },\n      \"httpResponse\" : {\n        \"statusCode\" : 200,\n        \"reasonPhrase\" : \"OK\",\n        \"body\" : \"four\"\n      }\n    },\n    \"key\" : \"1a9d2b00-e37e-464f-91df-86c5167bf816_proxied\"\n  }, {\n    \"description\" : \"  three\",\n    \"value\" : {\n      \"httpRequest\" : {\n        \"path\" : \"three\"\n      },\n      \"httpResponse\" : {\n        \"statusCode\" : 200,\n        \"reasonPhrase\" : \"OK\",\n        \"body\" : \"three\"\n      }\n    },\n    \"key\" : \"34773a6e-987a-488f-b4a7-60772b5d02e3_proxied\"\n  }, {\n    \"description\" : \"    two\",\n    \"value\" : {\n      \"httpRequest\" : {\n        \"path\" : \"two\"\n      },\n      \"httpResponse\" : {\n        \"statusCode\" : 200,\n        \"reasonPhrase\" : \"OK\",\n        \"body\" : \"two\"\n      }\n    },\n    \"key\" : \"83b73925-6179-4a94-b703-22aca2677438_proxied\"\n  }, {\n    \"description\" : \"    one\",\n    \"value\" : {\n      \"httpRequest\" : {\n        \"path\" : \"one\"\n      },\n      \"httpResponse\" : {\n        \"statusCode\" : 200,\n        \"reasonPhrase\" : \"OK\",\n        \"body\" : \"one\"\n      }\n    },\n    \"key\" : \"76af75f7-12ac-49da-9790-f05f65b991b0_proxied\"\n  } ]\n}"
     but: was "{\n  \"logMessages\" : [ {\n    \"key\" : \"1a9d2b00-e37e-464f-91df-86c5167bf816_log\",\n    \"value\" : {\n      \"description\" : \"11-27 13:05:49.318 FORWARDED_REQUEST   \",\n      \"style\" : {\n        \"paddingBottom\" : \"4px\",\n        \"whiteSpace\" : \"nowrap\",\n        \"overflow\" : \"auto\",\n        \"color\" : \"rgb(152, 208, 255)\",\n        \"paddingTop\" : \"4px\"\n      },\n      \"messageParts\" : [ {\n        \"key\" : \"1a9d2b00-e37e-464f-91df-86c5167bf816_0msg\",\n        \"value\" : \"messageFormatFour\"\n      } ]\n    }\n  }, {\n    \"key\" : \"34773a6e-987a-488f-b4a7-60772b5d02e3_log\",\n    \"value\" : {\n      \"description\" : \"11-27 13:05:49.318 FORWARDED_REQUEST   \",\n      \"style\" : {\n        \"paddingBottom\" : \"4px\",\n        \"whiteSpace\" : \"nowrap\",\n        \"overflow\" : \"auto\",\n        \"color\" : \"rgb(152, 208, 255)\",\n        \"paddingTop\" : \"4px\"\n      },\n      \"messageParts\" : [ {\n        \"key\" : \"34773a6e-987a-488f-b4a7-60772b5d02e3_0msg\",\n        \"value\" : \"messageFormatThree\"\n      } ]\n    }\n  }, {\n    \"key\" : \"83b73925-6179-4a94-b703-22aca2677438_log\",\n    \"value\" : {\n      \"description\" : \"11-27 13:05:49.318 FORWARDED_REQUEST   \",\n      \"style\" : {\n        \"paddingBottom\" : \"4px\",\n        \"whiteSpace\" : \"nowrap\",\n        \"overflow\" : \"auto\",\n        \"color\" : \"rgb(152, 208, 255)\",\n        \"paddingTop\" : \"4px\"\n      },\n      \"messageParts\" : [ {\n        \"key\" : \"83b73925-6179-4a94-b703-22aca2677438_0msg\",\n        \"value\" : \"messageFormatTwo\"\n      } ]\n    }\n  }, {\n    \"key\" : \"76af75f7-12ac-49da-9790-f05f65b991b0_log\",\n    \"value\" : {\n      \"description\" : \"11-27 13:05:49.315 FORWARDED_REQUEST   \",\n      \"style\" : {\n        \"paddingBottom\" : \"4px\",\n        \"whiteSpace\" : \"nowrap\",\n        \"overflow\" : \"auto\",\n        \"color\" : \"rgb(152, 208, 255)\",\n        \"paddingTop\" : \"4px\"\n      },\n      \"messageParts\" : [ {\n        \"key\" : \"76af75f7-12ac-49da-9790-f05f65b991b0_0msg\",\n        \"value\" : \"messageFormatOne\"\n      } ]\n    }\n  } ],\n  \"proxiedRequests\" : [ {\n    \"key\" : \"1a9d2b00-e37e-464f-91df-86c5167bf816_proxied\",\n    \"description\" : \"   four\",\n    \"value\" : {\n      \"httpResponse\" : {\n        \"statusCode\" : 200,\n        \"reasonPhrase\" : \"OK\",\n        \"body\" : \"four\"\n      },\n      \"httpRequest\" : {\n        \"path\" : \"four\"\n      }\n    }\n  }, {\n    \"value\" : {\n      \"httpResponse\" : {\n        \"statusCode\" : 200,\n        \"reasonPhrase\" : \"OK\",\n        \"body\" : \"three\"\n      },\n      \"httpRequest\" : {\n        \"path\" : \"three\"\n      }\n    },\n    \"description\" : \"  three\",\n    \"key\" : \"34773a6e-987a-488f-b4a7-60772b5d02e3_proxied\"\n  }, {\n    \"description\" : \"    two\",\n    \"key\" : \"83b73925-6179-4a94-b703-22aca2677438_proxied\",\n    \"value\" : {\n      \"httpRequest\" : {\n        \"path\" : \"two\"\n      },\n      \"httpResponse\" : {\n        \"statusCode\" : 200,\n        \"reasonPhrase\" : \"OK\",\n        \"body\" : \"two\"\n      }\n    }\n  }, {\n    \"value\" : {\n      \"httpRequest\" : {\n        \"path\" : \"one\"\n      },\n      \"httpResponse\" : {\n        \"statusCode\" : 200,\n        \"reasonPhrase\" : \"OK\",\n        \"body\" : \"one\"\n      }\n    },\n    \"description\" : \"    one\",\n    \"key\" : \"76af75f7-12ac-49da-9790-f05f65b991b0_proxied\"\n  } ]\n}"
  1. What is the fix?
    To address the non-deterministic order of keys, I replaced the usage of HashMap with LinkedHashMap when creating the entry map. Unlike HashMap, LinkedHashMap preserves the order of keys based on their insertion order. By explicitly ordering the keys in the style map according to the expected order, we can ensure consistent ordering during each test run. This PR proposes changing HashMap to LinkedHashMap here and here and adding elements to style map in the expected order to fix all the listed tests.

You can run the following commands to run the 8 tests using NonDex tool respectively:

mvn -pl mockserver-netty edu.illinois:nondex-maven-plugin:2.1.1:nondex -Dtest=org.mockserver.dashboard.DashboardWebSocketHandlerTest#shouldSerialiseForwardedRequests
mvn -pl mockserver-netty edu.illinois:nondex-maven-plugin:2.1.1:nondex -Dtest=org.mockserver.dashboard.DashboardWebSocketHandlerTest#shouldSerialiseEventsWithRequestFilter
mvn -pl mockserver-netty edu.illinois:nondex-maven-plugin:2.1.1:nondex -Dtest=org.mockserver.dashboard.DashboardWebSocketHandlerTest#shouldSerialiseRollUpEventsWithCorrelationId
mvn -pl mockserver-netty edu.illinois:nondex-maven-plugin:2.1.1:nondex -Dtest=org.mockserver.dashboard.DashboardWebSocketHandlerTest#shouldSerialiseRecordedRequestsEventsWithoutFields
mvn -pl mockserver-netty edu.illinois:nondex-maven-plugin:2.1.1:nondex -Dtest=org.mockserver.dashboard.DashboardWebSocketHandlerTest#shouldSerialiseEventsWithNoRequestFilter
mvn -pl mockserver-netty edu.illinois:nondex-maven-plugin:2.1.1:nondex -Dtest=org.mockserver.dashboard.DashboardWebSocketHandlerTest#shouldSerialiseForwardedRequestsForEventsWithoutFields
mvn -pl mockserver-netty edu.illinois:nondex-maven-plugin:2.1.1:nondex -Dtest=org.mockserver.dashboard.DashboardWebSocketHandlerTest#shouldSerialiseEventsAndIgnoreDeletedLogEvents
mvn -pl mockserver-netty edu.illinois:nondex-maven-plugin:2.1.1:nondex -Dtest=org.mockserver.dashboard.DashboardWebSocketHandlerTest#shouldSerialiseRecordedRequests

Test Environment:

java version 11.0.19
Apache Maven 3.9.5

Kindly let me know if this fix is acceptable.
Thank you

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants