fix: issue1460 intermediate files existing (or not)

[vsoch]

Description

Just testing different ideas for #1460 - feel free to ignore!

QC

• The PR contains a test case for the changes or the changes are already covered by an existing test case.
• The documentation (docs/) is updated to reflect the changes or this is not necessary (e.g. if the change does neither modify the language nor the behavior or functionalities of Snakemake).

[github-actions]

Please format your code with black: black snakemake tests/*.py.

[vsoch]

okay doing a little testing - I've gotten down to the io.py function to wait for files, and looking at the files and attributes, according to this, one of them does not exist:

indeed it does exist!

It comes down to checking if a remote object exists, e.g., self.blob.exists() erroneously returns False:

Type 'copyright', 'credits' or 'license' for more information
IPython 7.23.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: self.blob
Out[1]: <Blob: snakemake-testing-d924es45, test_github_issue1460/preblob.txt, 1648855265000563>

In [2]: self.blob.exists()
Out[2]: False

but there it is!

self.blob.reload() does not work

self.blob.reload()
...
NotFound: 404 GET https://storage.googleapis.com/storage/v1/b/snakemake-testing-d924es45/o/test_github_issue1460%2Fpreblob.txt?generation=1648855265000563&projection=noAcl&prettyPrint=false: No such object: snakemake-testing-d924es45/test_github_issue1460/preblob.txt

But indeed if we "ask again" for the key, we see it exists:

In [11]: self.bucket.get_blob(self.key)
Out[11]: <Blob: snakemake-testing-d924es45, test_github_issue1460/preblob.txt, 1648856699412397>

In [12]: blob = self.bucket.get_blob(self.key)

In [13]: blob.exists()
Out[13]: True

so that's the change I did for the PR here - looks like the reproducing test of the bug passes, so we fixed it! There might be some ways to further optimize (e.g., only do the update_blob under some conditions) but someone else can look into that if desired.

Thanks for letting me play on GCP this evening @johanneskoester it was SO fun I missed it a lot <3

[sonarcloud]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
0.0% Duplication

[johanneskoester]

Awesome work @vsoch, thanks a lot!