awswrangler==2.14.0 in AWS glue python shell within VPC ,private subnet- athena results s3 connection timeout

[sunil-ygroup]

Hi All,

I am getting very weird exception , most of the times am getting connection timeout exception to s3 athena results bucket when i tries to read the data from Athena. Sometimes the job ran successfully .

The glue job is running within a private subnet , but we have private endpoint to S3, Athena , Glue.

dataset= wr.athena.read_sql_query("<query>", database=raw_zone_database,boto3_session=my_session,s3_output=athena_s3_output)

[AdrianoNicolucci]

Ok, I think I can answer why it might sometimes work for you and sometimes it doesn't. It's possible that one of your subnets is missing the vpc endpoints so when the glue job uses that specific one it fails but then it works the other times when using the other subnet that has the vpc endpoints