chore: Re-enable parallel showcase tests

[lqiu96]

Thank you for opening a Pull Request! For general contributing guidelines, please refer to contributing guide

Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

• Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
• Ensure the tests and linter pass
• Code coverage does not decrease (if any source code was changed)
• Appropriate docs were updated (if necessary)

Following up on the conversation at: #1669 (comment)

Seen a lot more stability with the showcase tests and I think we can re-enable them.

[sonarcloud]

[gapic-generator-java-root] Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
0.0% Duplication

[lqiu96]

Huh.. Seems like we're seeing a DEADLINE_EXCEEDED exception when running the test in parallel:

Error:  Tests run: 4, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 11.174 s <<< FAILURE! - in com.google.showcase.v1beta1.it.ITCrud
Error:  com.google.showcase.v1beta1.it.ITCrud.testHttpJson_Create  Time elapsed: 5.032 s  <<< ERROR!
com.google.api.gax.rpc.DeadlineExceededException: io.grpc.StatusRuntimeException: DEADLINE_EXCEEDED: deadline exceeded after 4.994418288s. [closed=[], open=[[remote_addr=localhost/127.0.0.1:7469]]]

[lqiu96]

@burkedavison @blakeli0 Adding parallel tests back for showcase re-introduced a flaky test (SonarCloud failed with the exception).

Perhaps we can reduce the forkCount value? Or keep it as sequential? WDYT?

[burkedavison]

Perhaps we can reduce the forkCount value? Or keep it as sequential? WDYT?

Do we know if it's our client or the Showcase server that is having trouble with the parallel tests?

If it's the server, I'd suggest a ticket in the gapic-showcase repo and keeping our tests sequential for now.

If it's our client, it'd be nice to have a better understanding of the issue.

[lqiu96]

Perhaps we can reduce the forkCount value? Or keep it as sequential? WDYT?

Do we know if it's our client or the Showcase server that is having trouble with the parallel tests?

If it's the server, I'd suggest a ticket in the gapic-showcase repo and keeping our tests sequential for now.

If it's our client, it'd be nice to have a better understanding of the issue.

I haven't been able to do a deep dive so I can't say this is the problem for sure, but I'm fairly certain it's a limitation from the server. The flaky test issues started to appear after adding a bunch of the unary deadline showcase tests which spammed the server with a ton of requests in a short amount of time. The past few weeks have been stable for the showcase tests and I think having sequential tests helped.

I've talked to Noah about this and he's mentioned that this may be a limitation of the server as it was never built for performance and was never tested to handle lots of requests.