Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix(datastore): increase deferred key iter limit #2878

Merged
merged 3 commits into from Sep 17, 2020
Merged

fix(datastore): increase deferred key iter limit #2878

Changes from 2 commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
ac72c70
fix(datastore): increase deferred key iter limit
tritone Sep 17, 2020
7ecccf3
Merge branch 'master' into ds-iter-fix
tritone Sep 17, 2020
e1df3d5
Merge branch 'master' into ds-iter-fix
tritone Sep 17, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
12 changes: 4 additions & 8 deletions datastore/datastore.go
View file
Open in desktop
Expand Up @@ -421,15 +421,11 @@ func (c *Client) get(ctx context.Context, keys []*Key, dst interface{}, opts *pb
}
found := resp.Found
missing := resp.Missing
// Upper bound 100 iterations to prevent infinite loop.
// We choose 100 iterations somewhat logically:
// Max number of Entities you can request from Datastore is 1,000.
// Max size for a Datastore Entity is 1 MiB.
// Max request size is 10 MiB, so we assume max response size is also 10 MiB.
// 1,000 / 10 = 100.
// Upper bound 1000 iterations to prevent infinite loop. This matches the max
// number of Entities you can request from Datastore.
// Note that if ctx has a deadline, the deadline will probably
// be hit before we reach 100 iterations.
for i := 0; len(resp.Deferred) > 0 && i < 100; i++ {
// be hit before we reach 1000 iterations.
for i := 0; len(resp.Deferred) > 0 && i < 1000; i++ {
req.Keys = resp.Deferred
resp, err = c.client.Lookup(ctx, req)
if err != nil {
Expand Down