test(postman} flakey test: nonExistingURLshouldReturn404

[spbolton]

https://github.com/dotCMS/core/actions/runs/8790260744/job/24122370748#step:10:28026

[INFO] → nonExistingURLshouldReturn404
DOTCMS:21:18:48.772  WARN  struts.MultiMessageResources - Duplicate resource property definition (key=was ==> is now): contenttypes.content.copy=Copy ==> Copy
DOTCMS:21:18:48.774  WARN  struts.MultiMessageResources - Duplicate resource property definition (key=was ==> is now): message.category.import=Importing Categories.  This operation cannot be cancelled or undone. ==> IMPORT
DOTCMS:21:18:48.774  WARN  struts.MultiMessageResources - Duplicate resource property definition (key=was ==> is now): message.containers.create.content_type_code=Content Type Code ==> Content Type Code
DOTCMS:21:18:48.776  WARN  struts.MultiMessageResources - Duplicate resource property definition (key=was ==> is now): Save=Save ==> Save
DOTCMS:21:18:48.777  WARN  struts.MultiMessageResources - Duplicate resource property definition (key=was ==> is now): Move=Move ==> Move
DOTCMS:21:18:48.777  WARN  struts.MultiMessageResources - Duplicate resource property definition (key=was ==> is now): dot.common.remove=Remove ==> Remove
DOTCMS:21:18:48.777  WARN  struts.MultiMessageResources - Duplicate resource property definition (key=was ==> is now): archived=archived ==> Archived
DOTCMS:21:18:48.777  WARN  struts.MultiMessageResources - Duplicate resource property definition (key=was ==> is now): experiments.action.edit=Edit ==> Edit
DOTCMS:21:18:48.807  INFO  servlet.VelocityServlet - Initializing VelocityServlet @ Thread-6
DOTCMS:21:18:48.809  INFO  license.ServerDetector -
DOTCMS:21:18:48.822  INFO  license.ServerDetector -
DOTCMS:21:18:48.822  INFO  license.ServerDetector -
DOTCMS:21:18:48.823  INFO  license.ServerDetector -
DOTCMS:21:18:48.824  INFO  license.ServerDetector -
DOTCMS:21:18:48.824  INFO  license.ServerDetector -
DOTCMS:21:18:48.825  INFO  license.ServerDetector -
DOTCMS:21:18:48.825  INFO  license.ServerDetector -
DOTCMS:21:18:48.826  INFO  license.ServerDetector -
DOTCMS:21:18:48.827  WARN  business.PermissionBitAPIImpl - User 'anonymous' cannot verify READ permissions on Contentlet '8a7d5e23-da1e-420a-b4f0-471e7da8ea2d' because it is not live.
DOTCMS:21:18:48.827  ERROR web.HostWebAPIImpl - User anonymous does not have permission to host:default
DOTCMS:21:18:48.827  WARN  servlet.VelocityServlet - User anonymous does not have permission to host:default
DOTCMS:21:18:48.827  WARN  servlet.VelocityServlet - User anonymous does not have permission to host:default
DOTCMS:21:18:48.827  WARN  servlet.VelocityServlet - com.dotmarketing.business.web.HostWebAPIImpl.checkHostPermission(HostWebAPIImpl.java:116)
[INFO]   GET http://localhost:8080/thisDoesNotExist [401 Unauthorized, 355B, 125ms]
[INFO]   1. Should get 404 page
[INFO] 
[INFO] ┌─────────────────────────┬───────────────────┬───────────────────┐
[INFO] │                         │          executed │            failed │
[INFO] ├─────────────────────────┼───────────────────┼───────────────────┤
[INFO] │              iterations │                 1 │                 0 │
[INFO] ├─────────────────────────┼───────────────────┼───────────────────┤
[INFO] │                requests │                 4 │                 0 │
[INFO] ├─────────────────────────┼───────────────────┼───────────────────┤
[INFO] │            test-scripts │                 4 │                 0 │
[INFO] ├─────────────────────────┼───────────────────┼───────────────────┤
[INFO] │      prerequest-scripts │                 0 │                 0 │
[INFO] ├─────────────────────────┼───────────────────┼───────────────────┤
[INFO] │              assertions │                 4 │                 1 │
[INFO] ├─────────────────────────┴───────────────────┴───────────────────┤
[INFO] │ total run duration: 44s                                         │
[INFO] ├─────────────────────────────────────────────────────────────────┤
[INFO] │ total data received: 1.15kB (approx)                            │
[INFO] ├─────────────────────────────────────────────────────────────────┤
[INFO] │ average response time: 10.9s [min: 125ms, max: 25.3s, s.d.: 9s] │
[INFO] └─────────────────────────────────────────────────────────────────┘
[INFO] 
[INFO]   #  failure         detail                                                
[INFO]                                                                            
[INFO]  1.  AssertionError  Should get 404 page                                   
[INFO]                      expected response to have status code 404 but got 401 
[INFO]                      at assertion:0 in test-script                         
[INFO]                      inside "nonExistingURLshouldReturn404"                
[INFO] Collection VanityURL.postman_collection had failures. Details:
[INFO] Failure #1: SerialisedError {
[INFO]   name: 'AssertionError',
[INFO]   index: 0,
[INFO]   test: 'Should get 404 page',
[INFO]   message: 'expected response to have status code 404 but got 401',
[INFO]   stack: 'AssertionError: expected response to have status code 404 but got 401\n' +
[INFO]     '   at Object.eval sandbox-script.js:1:4)',
[INFO]   checksum: '7024325e2b389efff186e80b10aea092',
[INFO]   id: '1cbced96-a081-4193-85b8-02a14789a784',
[INFO]   timestamp: 1713820728842,
[INFO]   stacktrace: [
[INFO]     {
[INFO]       fileName: 'sandbox-script.js',
[INFO]       lineNumber: 1,
[INFO]       functionName: 'Object.eval',
[INFO]       typeName: 'Object',
[INFO]       methodName: 'eval',
[INFO]       columnNumber: 4,
[INFO]       native: false
[INFO]     }
[INFO]   ]
[INFO] }

Full Log

19_Merge Group Test _ Run Postman Tests - default.txt