How to Ignore certain test cases

[krishnan-techconative]

Hi @kerrykimbrough ,

Tcases is very good in terms of generating test cases for both positive and negative scenarios. I would like to know if there any possibilities are currently available to Ignore certain failures those are actually legitimate. I prefer to have @ignore annotations.

For instance, the delete endpoint is sending an id which is not exists in the server and tcases is expecting it to be passed.

@Test(timeOut=30000)
    public void deletePaymentControlsId_IdValue_Is_Gt_1() {
        given()
            .baseUri( forTestServer( "http://localhost:9000"))
            .relaxedHTTPSValidation()
            .header( "Authorization", tcasesApiBearerCredentials())
            .pathParam( "id", **"393756903"**)
        .when()
            .request( "DELETE", "/paymentControls/{id}")
        .then()
            .statusCode( isSuccess())
            ;
    }

[kerrykimbrough]

Your example demonstrates one of the limitations of Tcases for OpenAPI. A successful DELETE operation must identify a resource that is known to exist. But Tcases has no way to know this. For success cases, Tcases simply generates random input values that conform to the requirements of the OpenAPI definition.

In practice, a test for DELETE which needs to be repeatable will include some setup that automatically creates the resource to be deleted. But Tcases has no way to know how to do this or even if this is necessary. The only information available to Tcases is the OpenAPI definition.

This is an example of a situation where a test for one request requires a sequence of prior requests. But Tcases can only generate tests for a single request execution.

[krishnan-techconative]

Hi @kerrykimbrough

Thank you for your prompt reply. I concur that tcases is currently generating single request executions. The delete method mentioned above serves as an example of this behavior. While adding an example in the Open API Spec results in tcases generating a test case solely based on that example, my intention is to generate both negative and positive scenarios. Additionally, I aim to exclude certain negative scenarios that are considered legitimate.

Please feel free to know your suggestions / thoughts on this.

[kerrykimbrough]

I would not say that this is a "failure case that is considered legitimate". Instead, this is a case of Tcases generating what is intended to be a success case. But because of the limitations described, the test can't work. The solution is not to ignore the intended success case. Instead, we want Tcases to be smart enough to produce the test case in the right way. But how? I don't know.

[krishnan-techconative]

Agree, Can we have a Brainstorming session together to think through how we can address this issue, if possible?

[kerrykimbrough]

Let's continue the discussion here. I find it helps to clarify my thoughts when I write them out.