Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

pubsub: unit tests cause emulator to use excessive CPU #697

Closed
cortesi opened this issue Jul 13, 2017 · 4 comments
Closed

pubsub: unit tests cause emulator to use excessive CPU #697

cortesi opened this issue Jul 13, 2017 · 4 comments
Assignees
@pongad
Labels
api: pubsub Issues related to the Pub/Sub API.

Comments

@cortesi
Copy link

cortesi commented Jul 13, 2017

The pubsub emulator recently started using large amounts of CPU on OSX during our unit tests (other platforms untested). High CPU usage persists after the test suite has completed and all clients disconnected, until the process is restarted. I am able to reproduce this behaviour with simple clients and with google-cloud-go's own test suite.

Steps to reproduce:

  • Start the emulator:
    gcloud beta emulators pubsub start --host-port=localhost:8822
  • In the directory cloud.google.com/go/pubsub, run the test suite:
    env PUBSUB_EMULATOR_HOST=localhost:8822 go test -v

On my system, the pubsub emulator Java process CPU usage is at 45% after one run, 145% after the second run, and so forth.

This is undoubtedly a bug in the emulator, but git bisect shows that google-cloud-go only started triggering the issue with commit 927c812. This commit is included in the v0.10.0 release.

Software versions used:

  • google-cloud-go master
  • pubsub-emulator 2017.03.24
  • OSX 10.12.5
@pongad pongad self-assigned this Jul 14, 2017
@pongad pongad added the api: pubsub Issues related to the Pub/Sub API. label Jul 14, 2017
@pongad
Copy link
Contributor

pongad commented Jul 14, 2017 •
edited

Thank you for the detailed analysis. That commit increases the number of worker goroutines from 1 to 10*CPU. So this makes a lot of sense.

In your test, could you try adding sub.ReceiveSettings.NumGoroutines = 1? This should bring CPU consumption back to the level before the commit you linked.

@cortesi
Copy link
Author

cortesi commented Jul 14, 2017

Hi there - that's the workaround that we have in place for now. We still see a slow growth in CPU usage of 1-2% with each run.

@pongad
Copy link
Contributor

pongad commented Jul 14, 2017

Ah, I'm sorry. I misread the part where you said "High CPU usage persists after the test suite has completed". I'll loop in the emulator folks.

@pongad
Copy link
Contributor

pongad commented Aug 8, 2017

The fix for this has been made to the emulator and going to be published soon.
We've also scaled back NumGoroutines, so this should alleviate the problem in the mean time.
I'll close this for now.

@pongad pongad closed this as completed Aug 8, 2017
semistrict pushed a commit to semistrict/google-cloud-go that referenced this issue Jun 11, 2018
@rakyll
Fix the improperly formatted link trace ID (googleapis#697)
c7be888 
Fixes googleapis#690.
@yoshi-automation yoshi-automation mentioned this issue Jan 12, 2022
Merged
yoshi-automation added a commit that referenced this issue Jan 12, 2022
@yoshi-automation
chore(all): auto-regenerate gapics
547df08 
This is an auto-generated regeneration of the gapic clients by
cloud.google.com/go/internal/gapicgen. Once the corresponding genproto PR is
submitted, genbot will update this PR with a newer dependency to the newer
version of genproto and assign reviewers to this PR.

If you have been assigned to review this PR, please:

- Ensure that the version of genproto in go.mod has been updated.
- Ensure that CI is passing. If it's failing, it requires your manual attention.
- Approve and submit this PR if you believe it's ready to ship.

Corresponding genproto PR: googleapis/go-genproto#742

Changes:

feat(automl): publish updated protos for cloud/automl/v1 service fix!: One of the fields now have field_behavior as REQUIRED in cloud/automl/v1 service definition.
  Committer: @hkdevandla
  PiperOrigin-RevId: 421156406
  Source-Link: googleapis/googleapis@05b24d8

fix(talent): add ancillary service bindings to service_yaml
  PiperOrigin-RevId: 421134169
  Source-Link: googleapis/googleapis@ef417dc

fix(workflows): add ancillary service bindings to service_yaml
  PiperOrigin-RevId: 421098450
  Source-Link: googleapis/googleapis@6706d9b

fix(managedidentities): add ancillary service bindings to service_yaml
  PiperOrigin-RevId: 421098407
  Source-Link: googleapis/googleapis@a635f71

fix(documentai): add ancillary service bindings to service_yaml
  PiperOrigin-RevId: 421098374
  Source-Link: googleapis/googleapis@706ac2b

fix(datalabeling): add ancillary service bindings to service_yaml
  PiperOrigin-RevId: 421098303
  Source-Link: googleapis/googleapis@ac12f33

fix(apigateway): add ancillary service bindings to service_yaml
  PiperOrigin-RevId: 421098261
  Source-Link: googleapis/googleapis@4c13e8f

fix(asset): add service_yaml to php_gapic_library targets
  PiperOrigin-RevId: 421075193
  Source-Link: googleapis/googleapis@dc15873

feat(compute): remove BETA language on Compute V1 (#697)

  Source-Link: googleapis/googleapis@b4b8eda

fix(artifactregistry): add missing HTTP rules to service config
  PiperOrigin-RevId: 420790332
  Source-Link: googleapis/googleapis@3965406
yoshi-automation added a commit that referenced this issue Jan 12, 2022
@yoshi-automation
chore(all): auto-regenerate gapics
e32cfc4 
This is an auto-generated regeneration of the gapic clients by
cloud.google.com/go/internal/gapicgen. Once the corresponding genproto PR is
submitted, genbot will update this PR with a newer dependency to the newer
version of genproto and assign reviewers to this PR.

If you have been assigned to review this PR, please:

- Ensure that the version of genproto in go.mod has been updated.
- Ensure that CI is passing. If it's failing, it requires your manual attention.
- Approve and submit this PR if you believe it's ready to ship.

Corresponding genproto PR: googleapis/go-genproto#742

Changes:

feat(automl): publish updated protos for cloud/automl/v1 service fix!: One of the fields now have field_behavior as REQUIRED in cloud/automl/v1 service definition.
  Committer: @hkdevandla
  PiperOrigin-RevId: 421156406
  Source-Link: googleapis/googleapis@05b24d8

fix(talent): add ancillary service bindings to service_yaml
  PiperOrigin-RevId: 421134169
  Source-Link: googleapis/googleapis@ef417dc

fix(workflows): add ancillary service bindings to service_yaml
  PiperOrigin-RevId: 421098450
  Source-Link: googleapis/googleapis@6706d9b

fix(managedidentities): add ancillary service bindings to service_yaml
  PiperOrigin-RevId: 421098407
  Source-Link: googleapis/googleapis@a635f71

fix(documentai): add ancillary service bindings to service_yaml
  PiperOrigin-RevId: 421098374
  Source-Link: googleapis/googleapis@706ac2b

fix(datalabeling): add ancillary service bindings to service_yaml
  PiperOrigin-RevId: 421098303
  Source-Link: googleapis/googleapis@ac12f33

fix(apigateway): add ancillary service bindings to service_yaml
  PiperOrigin-RevId: 421098261
  Source-Link: googleapis/googleapis@4c13e8f

fix(asset): add service_yaml to php_gapic_library targets
  PiperOrigin-RevId: 421075193
  Source-Link: googleapis/googleapis@dc15873

feat(compute): remove BETA language on Compute V1 (#697)

  Source-Link: googleapis/googleapis@b4b8eda

fix(artifactregistry): add missing HTTP rules to service config
  PiperOrigin-RevId: 420790332
  Source-Link: googleapis/googleapis@3965406
codyoss pushed a commit that referenced this issue Jan 12, 2022
@yoshi-automation
chore(all): auto-regenerate gapics (#5347)
3bbe8c0 
This is an auto-generated regeneration of the gapic clients by
cloud.google.com/go/internal/gapicgen. Once the corresponding genproto PR is
submitted, genbot will update this PR with a newer dependency to the newer
version of genproto and assign reviewers to this PR.

If you have been assigned to review this PR, please:

- Ensure that the version of genproto in go.mod has been updated.
- Ensure that CI is passing. If it's failing, it requires your manual attention.
- Approve and submit this PR if you believe it's ready to ship.

Corresponding genproto PR: googleapis/go-genproto#742

Changes:

feat(automl): publish updated protos for cloud/automl/v1 service fix!: One of the fields now have field_behavior as REQUIRED in cloud/automl/v1 service definition.
  Committer: @hkdevandla
  PiperOrigin-RevId: 421156406
  Source-Link: googleapis/googleapis@05b24d8

fix(talent): add ancillary service bindings to service_yaml
  PiperOrigin-RevId: 421134169
  Source-Link: googleapis/googleapis@ef417dc

fix(workflows): add ancillary service bindings to service_yaml
  PiperOrigin-RevId: 421098450
  Source-Link: googleapis/googleapis@6706d9b

fix(managedidentities): add ancillary service bindings to service_yaml
  PiperOrigin-RevId: 421098407
  Source-Link: googleapis/googleapis@a635f71

fix(documentai): add ancillary service bindings to service_yaml
  PiperOrigin-RevId: 421098374
  Source-Link: googleapis/googleapis@706ac2b

fix(datalabeling): add ancillary service bindings to service_yaml
  PiperOrigin-RevId: 421098303
  Source-Link: googleapis/googleapis@ac12f33

fix(apigateway): add ancillary service bindings to service_yaml
  PiperOrigin-RevId: 421098261
  Source-Link: googleapis/googleapis@4c13e8f

fix(asset): add service_yaml to php_gapic_library targets
  PiperOrigin-RevId: 421075193
  Source-Link: googleapis/googleapis@dc15873

feat(compute): remove BETA language on Compute V1 (#697)

  Source-Link: googleapis/googleapis@b4b8eda

fix(artifactregistry): add missing HTTP rules to service config
  PiperOrigin-RevId: 420790332
  Source-Link: googleapis/googleapis@3965406
This was referenced Feb 9, 2022
Closed
Closed
Closed
@release-please release-please bot mentioned this issue Feb 17, 2022
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@pongad pongad

Labels
api: pubsub Issues related to the Pub/Sub API.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@cortesi @pongad