Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: fix flakiness in subscriberIT integration test #476

Merged
Merged
Changes from all commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
19dd129
Modifying Publish example in README to match other examples given, and
hannahrogers-google Nov 25, 2019
4158529
fix: Modifying Publish example in README to match other examples, and
hannahrogers-google Nov 25, 2019
7d704f0
Merge branch 'master' of github.com:hannahrogers-google/java-pubsub
hannahrogers-google Jan 9, 2020
9e0ebc6
Merge branch 'master' of https://github.com/googleapis/java-pubsub
hannahrogers-google Jan 9, 2020
01d41b7
feat: Adding support for DLQs
hannahrogers-google Jan 13, 2020
ed3b1ee
Fix formatting
hannahrogers-google Jan 13, 2020
72f7996
fix: making changes requested in pull request
hannahrogers-google Jan 14, 2020
389cb86
Merge branch 'master' of https://github.com/googleapis/java-pubsub
hannahrogers-google Jan 14, 2020
c9e4bd2
Merge branch 'master' of https://github.com/googleapis/java-pubsub
hannahrogers-google Jan 29, 2020
3738ec8
fix: creating fix to not populate delivery attempt attribute when dead
hannahrogers-google Jan 29, 2020
4ce1d3b
Merge branch 'master' of https://github.com/googleapis/java-pubsub
hannahrogers-google Jan 29, 2020
aecd4ca
Adding unit test for case in which a received message has no delivery…
hannahrogers-google Jan 30, 2020
77fa3b3
Making MessageWaiter class more generic to also be used for outstanding
hannahrogers-google Jan 30, 2020
b87023c
Merge branch 'master' of https://github.com/googleapis/java-pubsub
hannahrogers-google Jan 30, 2020
7086b3c
Merge branch 'master' into fix-subscriber-stop
hannahrogers-google Jan 30, 2020
da0e355
Waiting for acks to complete before shutting down a streaming subscriber
hannahrogers-google Jan 30, 2020
dc55fc1
Fixing formatting error
hannahrogers-google Jan 31, 2020
45beee2
Merge branch 'master' of https://github.com/googleapis/java-pubsub
hannahrogers-google Jan 31, 2020
548b7a7
fix: making sure all publishes complete before shutting down the
hannahrogers-google Jan 31, 2020
76a1de9
Merge branch 'master' of https://github.com/googleapis/java-pubsub
hannahrogers-google Feb 20, 2020
a8989f1
Merge branch 'master' of https://github.com/googleapis/java-pubsub
hannahrogers-google Feb 24, 2020
b4439fd
adding default max outstanding request bytes
hannahrogers-google Feb 24, 2020
01f607c
Merge branch 'master' of https://github.com/googleapis/java-pubsub
hannahrogers-google Feb 27, 2020
c0cddb3
Merge branch 'master' of https://github.com/googleapis/java-pubsub
hannahrogers-google Apr 29, 2020
9a6381d
fix: make push endpoint valid https
hannahrogers-google Apr 29, 2020
0bd165f
Merge branch 'master' of https://github.com/googleapis/java-pubsub
hannahrogers-google May 24, 2020
5d47be8
Merge branch 'master' of https://github.com/googleapis/java-pubsub
hannahrogers-google May 25, 2020
e3b7f6d
Merge branch 'master' of https://github.com/googleapis/java-pubsub in…
hannahrogers-google Jul 8, 2020
6c1a26c
Merge branch 'master' of https://github.com/googleapis/java-pubsub in…
hannahrogers-google Jul 23, 2020
f2fa6b8
fix: use default zero value if a flow control setting is not provided
hannahrogers-google Jul 23, 2020
0cd08dc
fix lint issues
hannahrogers-google Jul 23, 2020
9ba9764
Merge branch 'master' of https://github.com/googleapis/java-pubsub in…
hannahrogers-google Aug 4, 2020
f52eb61
Merge branch 'master' of https://github.com/googleapis/java-pubsub
hannahrogers-google Aug 6, 2020
78d98e8
fix: better cleanup during publisher test
hannahrogers-google Aug 6, 2020
785c3ee
fix: format issues
hannahrogers-google Aug 7, 2020
dbe753f
Merge branch 'master' of https://github.com/googleapis/java-pubsub
hannahrogers-google Aug 20, 2020
35c2c6e
fix: test timeouts should be a minute
hannahrogers-google Aug 20, 2020
8fd6691
Merge branch 'master' of https://github.com/googleapis/java-pubsub
hannahrogers-google Sep 3, 2020
b76e387
fix: make subscriberIt less flaky
hannahrogers-google Sep 3, 2020
c855f24
Merge branch 'master' of https://github.com/googleapis/java-pubsub
hannahrogers-google Dec 23, 2020
d274df4
fix: add deprecated tag for deprecated methods
hannahrogers-google Dec 23, 2020
5663e12
Merge branch 'master' of https://github.com/googleapis/java-pubsub
hannahrogers-google Jan 6, 2021
fef9956
fix: retrying sync pulls in subscriberIT test
hannahrogers-google Jan 6, 2021
5bbaa7e
fix: reverting other branch
hannahrogers-google Jan 6, 2021
d185e92
fix: revert synth.py
hannahrogers-google Jan 6, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
43 changes: 35 additions & 8 deletions samples/snippets/src/test/java/pubsub/SubscriberIT.java
Expand Up @@ -37,6 +37,7 @@
import java.io.ByteArrayOutputStream;
import java.io.PrintStream;
import java.util.ArrayList;
import java.util.HashSet;
import java.util.List;
import java.util.UUID;
import org.junit.After;
Expand Down Expand Up @@ -84,6 +85,32 @@ private static void publishSomeMessages(Integer numOfMessages) throws Exception
ApiFutures.allAsList(messageIdFutures).get();
}

// Helper function to retry synchronous pull attempts until all outstanding messages are received.
private void syncPullWithRetries(
Integer numOfMessages, Integer maxRetries, CheckedRunnable syncPull) throws Exception {
HashSet<String> outstandingMessages = new HashSet<>();
for (int i = 0; i < numOfMessages; i++) {
outstandingMessages.add("Hello " + i);
}
int attempt = 1;
while ((outstandingMessages.size() > 0) && (attempt <= maxRetries)) {
syncPull.run();
HashSet<String> clone = (HashSet) outstandingMessages.clone();
for (String message : clone) {
if (bout.toString().contains(message)) {
outstandingMessages.remove(message);
}
}
attempt++;
}
assertThat(outstandingMessages).isEmpty();
}

@FunctionalInterface
public interface CheckedRunnable {
void run() throws Exception;
}

@Rule public Timeout globalTimeout = Timeout.seconds(600); // 10 minute timeout

@BeforeClass
Expand Down Expand Up @@ -168,17 +195,17 @@ public void testSubscriber() throws Exception {
publishSomeMessages(3);
bout.reset();
// Test subscribe synchronously.
SubscribeSyncExample.subscribeSyncExample(projectId, subscriptionId, 10);
for (int i = 0; i < 3; i++) {
assertThat(bout.toString()).contains("Hello " + i);
}
syncPullWithRetries(
3, 3, () -> SubscribeSyncExample.subscribeSyncExample(projectId, subscriptionId, 3));

publishSomeMessages(3);
bout.reset();
// Test subscribe synchronously with lease management.
SubscribeSyncWithLeaseExample.subscribeSyncWithLeaseExample(projectId, subscriptionId, 10);
for (int i = 0; i < 3; i++) {
assertThat(bout.toString()).contains("Hello " + i);
}
syncPullWithRetries(
3,
3,
() ->
SubscribeSyncWithLeaseExample.subscribeSyncWithLeaseExample(
projectId, subscriptionId, 10));
}
}