Debug dataproc failed presubmits

[bcheena]

Do not merge this PR. Only for debugging purposes.

[bcheena]

/gcbrun

[bcheena]

This is the error after debug logs:

Step #4 - "dataproc-1.5-ubuntu18-tests": 2023-12-05T08:20:54.220014553Z ==================== Test output for //knox:test_knox (shard 6 of 6):
Step #4 - "dataproc-1.5-ubuntu18-tests": 2023-12-05T08:20:54.220021987Z Running tests under Python 3.8.10: /usr/bin/python3
Step #4 - "dataproc-1.5-ubuntu18-tests": 2023-12-05T08:20:54.220045910Z W1205 08:20:47.712818 140201952642880 dataproc_test_case.py:57] Printing debug logs for project here
Step #4 - "dataproc-1.5-ubuntu18-tests": 2023-12-05T08:20:54.220085387Z W1205 08:20:49.442378 140201952642880 dataproc_test_case.py:58] (1, '', 'ERROR: gcloud failed to load. You are running gcloud with Python 3.11, which is not currently supported by gcloud.\nInstall a compatible version of Python 3.8-3.10 and set the CLOUDSDK_PYTHON environment variable to point to it.\n\nIf you are still experiencing problems, please reinstall the Google Cloud CLI using the instructions here:\n    https://cloud.google.com/sdk/docs/install\n')
Step #4 - "dataproc-1.5-ubuntu18-tests": 2023-12-05T08:20:54.220094003Z W1205 08:20:49.442851 140201952642880 dataproc_test_case.py:59] 
Step #4 - "dataproc-1.5-ubuntu18-tests": 2023-12-05T08:20:54.220101885Z W1205 08:20:51.024891 140201952642880 dataproc_test_case.py:64] Printing debug logs for region here
Step #4 - "dataproc-1.5-ubuntu18-tests": 2023-12-05T08:20:54.220109438Z W1205 08:20:52.446720 140201952642880 dataproc_test_case.py:65] (1, '', 'ERROR: gcloud failed to load. You are running gcloud with Python 3.11, which is not currently supported by gcloud.\nInstall a compatible version of Python 3.8-3.10 and set the CLOUDSDK_PYTHON environment variable to point to it.\n\nIf you are still experiencing problems, please reinstall the Google Cloud CLI using the instructions here:\n    https://cloud.google.com/sdk/docs/install\n')
Step #4 - "dataproc-1.5-ubuntu18-tests": 2023-12-05T08:20:54.220116263Z W1205 08:20:52.447159 140201952642880 dataproc_test_case.py:66] 
Step #4 - "dataproc-1.5-ubuntu18-tests": 2023-12-05T08:20:54.220130123Z [  FAILED  ] setUpClass (__main__.KnoxTestCase)
Step #4 - "dataproc-1.5-ubuntu18-tests": 2023-12-05T08:20:54.220140215Z ======================================================================
Step #4 - "dataproc-1.5-ubuntu18-tests": 2023-12-05T08:20:54.220162364Z ERROR: setUpClass (__main__.KnoxTestCase)
Step #4 - "dataproc-1.5-ubuntu18-tests": 2023-12-05T08:20:54.220170769Z ----------------------------------------------------------------------
Step #4 - "dataproc-1.5-ubuntu18-tests": 2023-12-05T08:20:54.220178225Z Traceback (most recent call last):
Step #4 - "dataproc-1.5-ubuntu18-tests": 2023-12-05T08:20:54.220186764Z   File "/home/ia-tests/.cache/bazel/_bazel_ia-tests/83b1ae36bb04ea5432b9efccee83c25f/execroot/__main__/bazel-out/k8-fastbuild/bin/knox/test_knox.runfiles/__main__/integration_tests/dataproc_test_case.py", line 68, in setUpClass
Step #4 - "dataproc-1.5-ubuntu18-tests": 2023-12-05T08:20:54.220194241Z     assert cls.PROJECT
Step #4 - "dataproc-1.5-ubuntu18-tests": 2023-12-05T08:20:54.220201750Z AssertionError

Specifically,

ERROR: gcloud failed to load. You are running gcloud with Python 3.11, which is not currently supported by gcloud.\nInstall a compatible version of Python 3.8-3.10 and set the CLOUDSDK_PYTHON environment variable to point to it.\n\nIf you are still experiencing problems, please reinstall the Google Cloud CLI using the instructions here:\n    https://cloud.google.com/sdk/docs/install\n'

[bcheena]

The latest version of gcloud mentions python support extended upto 3.11.6 in the release notes.

gcloud somehow picks up its own bundled 3.11 and claims to have the wrong version. I was able to check this internally where in one of our frontend checks, we had not modified MAX_SUPPORTED_PY3_VERSION from 3.10 to 3.11. This has been fixed in a change raised on Dec 1st and should be available in prod soon.

No action from our side should be needed to fix this presubmit.

[bcheena]

/gcbrun

[bcheena]

/gcbrun

[bcheena]

/gcbrun

[bcheena]

/gcbrun

[bcheena]

/gcbrun

[bcheena]

/gcbrun

[bcheena]

/gcbrun