Unable to catch (release API) errors without reconnection

[phlax]

im using gidgethub to push releases/artefacts.

if an artefact exists already it throws a gidgethub.InvalidField: Validation Failed for 'name'

if i catch the error, i have to reconnect (ie delete/create new) the session/GitHubAPI objects

im wondering if there is either a way to catch errors without the connection hosing, or a good/canonical way to reconnect if it does

[Mariatta]

Would you mind sharing a code snippet of how you're doing it currently?

[phlax]

Would you mind sharing a code snippet of how you're doing it currently?

sure its v WIP, but the connect to github is here:

https://github.com/envoyproxy/envoy/pull/17519/files#diff-be35bc20c4da421238b7b104b38b9605db9cbfeecc7095ea26e2a715db3a5c57R88

the session is created here:

https://github.com/envoyproxy/envoy/pull/17519/files#diff-be35bc20c4da421238b7b104b38b9605db9cbfeecc7095ea26e2a715db3a5c57R117

and anything around here, can trigger an error if there is an existing tag/artefact etc:

https://github.com/envoyproxy/envoy/pull/17519/files#diff-be35bc20c4da421238b7b104b38b9605db9cbfeecc7095ea26e2a715db3a5c57R219-R220

i can look before i leap, which i probs would anyway, but it would be good to catch gidgethub errors as known and handlable anyway, without having to reconnect

[Mariatta]

Thanks for sharing.

I saw these snippets:

try:
            response = await self.github.post(
                f"{await self.upload_url}?name={release_file.name}",
                data=release_file.read_bytes(),
                content_type="application/octet-stream")
        except gidgethub.InvalidField:
            self.log.error("Something went wrong uploading")
            await self.session.close()
            del self.__dict__["session"]
            del self.__dict__["github"]
            return

(from https://github.com/envoyproxy/envoy/pull/17519/files#diff-be35bc20c4da421238b7b104b38b9605db9cbfeecc7095ea26e2a715db3a5c57R191-R201
)

I think it was not necessary to close the session in the except clause. You can retry the API call and re-use the same session. At least that's how I've been doing it. I wonder if you had other reasons for closing the session when you catch the exception?

[phlax]

I think it was not necessary to close the session in the except clause. You can retry the API call and re-use the same session. At least that's how I've been doing it. I wonder if you had other reasons for closing the session when you catch the exception?

it was because without doing so i hit this error

...
ip3/pypi__aiohttp/aiohttp/client_reqrep.py", line 890, in start
    message, payload = await self._protocol.read()  # type: ignore
  File "/root/.cache/bazel/_bazel_root/f704bab1b165ed1368cb88f9f49e7532/execroot/envoy/bazel-out/k8-fastbuild/bin/tools/distribution/release.runfiles/github_pip3/pypi__aiohttp/aiohttp/streams.py", line 604, in read
    await self._waiter
aiohttp.client_exceptions.ClientOSError: [Errno 32] Broken pipe

when trying to make further accesses to the api

[phlax]

experimenting with this further, it seems the issue i hit is only happening with release artefact uploads

i can catch and carry on in these situations:

• gidgethub.BadRequest when trying to get a release that doesnt exist
• gidgethub.InvalidField when trying to create a release that exists already

the problem seems to be limited to the post/data/upload trying to send artefacts that exist already - which also raises a gidgethub.InvalidField

in that situation, catching and reusing the client connection results in aiohttp.client_exceptions.ClientOSError: [Errno 32] Broken pipe if i dont recreate/connect it

[phlax]

and experimenting a bit further...

i stepped through the code - afaict the aiohttp response is all good (well failing but with correct info etc)

if i add await asyncio.sleep(.4) (anything less doesnt work) then the connect recovers and i can carry on

not sure if this is something to do with github's api or an aiohttp issue with failed posts

[phlax]

not sure if this is something to do with github's api or an aiohttp issue with failed posts

i think the latter as if i reconnect immediately it works

[Mariatta]

It sounds more like a bug with aiohttp, and not with gidgethub.