ListJobsRequest is not using the 'location' filter

[PlugaruT]

Thanks for stopping by to let us know something could be better!

PLEASE READ: If you have a support contract with Google, please create an issue in the support console instead of filing on GitHub. This will ensure a timely response.

Please run down the following list and make sure you've tried the usual "quick fixes":

• Search the issues already opened: https://github.com/googleapis/python-dataflow-client/issues
• Search StackOverflow: https://stackoverflow.com/questions/tagged/google-cloud-platform+python

If you are still having issues, please be sure to include as much information as possible:

Environment details

• OS type and version: MacOS local and Debian in CI
• Python version: 3.8.7
• pip version: 21.2.3
• google-cloud-dataflow-client version: 0.1.3

Steps to reproduce

• Try and get the running dataflow jobs by setting another location besides us-central1, say us-east1. With the default one, it will get the jobs correctly, one you specify another region, it will return empty result, even tho there are jobs running in that region.

Code example

from google.cloud.dataflow_v1beta3.services.jobs_v1_beta3 import JobsV1Beta3Client
from google.cloud.dataflow_v1beta3.types.jobs import ListJobsRequest

jobs_client = JobsV1Beta3Client()

active_jobs = jobs_client.list_jobs(
            ListJobsRequest(project_id="project-id-here", filter=ListJobsRequest.Filter.ACTIVE, location="us-east1")
        )

Stack trace

No stacktrace, it just returns empty list.

Making sure to follow these steps will guarantee the quickest resolution possible.

Thanks!

[meredithslota]

Thanks for filing this! It appears there are (possibly) related issues in other clients which makes me think it might be a service-side problem: googleapis/nodejs-dataflow#19, #25 (previous issue in this repo). Pinging @busunkim96 to investigate.

[meredithslota]

We've identified similar issues in other repos and have escalated this to the API service team since it's not a language-specific library issue. Here are the related issues:

• ListJobsRequest is not using the 'location' filter #31: ListJobsRequest is not using the 'location' filter
• Support for locations in the launch_template method #25: Support for locations in the launch_template method
• The location option of launchFlexTemplate is not working nodejs-dataflow#19: The location option of launchFlexTemplate is not working
• https://github.com/googleapis/java-dataflow/issues/64: Dataflow: The location field of LaunchFlexTemplateRequest building is not working
• ListJobsAsync() returns only us-central1 jobs google-cloud-dotnet#7038: ListJobsAsync() returns only us-central1 jobs (context from issue is that the location parameter isn't working)

Marking this as a P1 bug w/ external label.

[bbhoss]

Just to be clear @meredithslota this is "external" in the sense that it's not an issue with this library itself, but an issue with the Dataflow service API (another team internal to Google?).

@PlugaruT did you find any workaround? This seems possibly related to GRPC as I've got a REST client service that's not having this problem.

[bbhoss]

I couldn't find an existing issue in the Google Issue Tracker so I created one here. I suggest starring this issue to anyone else that runs across this problem, hopefully it will get some attention from Google soon.

[davidcavazos]

The fix has been merged on PR googleapis/gapic-generator-python#1284 and it should be available on release 0.65.2 or later.

I'm closing this, but please re-open if the issue still persists.