Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Network is unreachable #760

Open
HuifangYeo opened this issue Feb 27, 2023 · 2 comments
Open

Network is unreachable #760

HuifangYeo opened this issue Feb 27, 2023 · 2 comments
Assignees
@tibdex
Labels
🐛 bug unexpected or wrong behavior

Comments

@HuifangYeo
Copy link
Contributor

Steps to reproduce

Hi,

since upgrading to v0.7.3, I have seen a lot of these exception in the server.log:

2023-02-27 10:17:59.757  WARN 38316 --- [Thread-1] i.InstanceMetadataServiceResourceFetcher : Fail to retrieve token 

com.amazonaws.SdkClientException: Failed to connect to service endpoint: 
	at com.amazonaws.internal.EC2ResourceFetcher.doReadResource(EC2ResourceFetcher.java:100) ~[atoti-aws.jar:na]
	at com.amazonaws.internal.InstanceMetadataServiceResourceFetcher.getToken(InstanceMetadataServiceResourceFetcher.java:91) ~[atoti-aws.jar:na]
	at com.amazonaws.internal.InstanceMetadataServiceResourceFetcher.readResource(InstanceMetadataServiceResourceFetcher.java:69) ~[atoti-aws.jar:na]
	at com.amazonaws.internal.EC2ResourceFetcher.readResource(EC2ResourceFetcher.java:66) ~[atoti-aws.jar:na]
	at com.amazonaws.auth.InstanceMetadataServiceCredentialsFetcher.getCredentialsEndpoint(InstanceMetadataServiceCredentialsFetcher.java:60) ~[atoti-aws.jar:na]
	at com.amazonaws.auth.InstanceMetadataServiceCredentialsFetcher.getCredentialsResponse(InstanceMetadataServiceCredentialsFetcher.java:48) ~[atoti-aws.jar:na]
	at com.amazonaws.auth.BaseCredentialsFetcher.fetchCredentials(BaseCredentialsFetcher.java:124) ~[atoti-aws.jar:na]
	at com.amazonaws.auth.BaseCredentialsFetcher.getCredentials(BaseCredentialsFetcher.java:80) ~[atoti-aws.jar:na]
	at com.amazonaws.auth.InstanceProfileCredentialsProvider.getCredentials(InstanceProfileCredentialsProvider.java:166) ~[atoti-aws.jar:na]
	at com.amazonaws.auth.EC2ContainerCredentialsProviderWrapper.getCredentials(EC2ContainerCredentialsProviderWrapper.java:75) ~[atoti-aws.jar:na]
	at com.amazonaws.auth.AWSCredentialsProviderChain.getCredentials(AWSCredentialsProviderChain.java:117) ~[atoti-aws.jar:na]
	at io.atoti.loading.s3.impl.S3CredentialsProviderChain.getCredentials(S3CredentialsProviderChain.java:29) ~[atoti-aws.jar:na]
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.getCredentialsFromContext(AmazonHttpClient.java:1269) ~[atoti-aws.jar:na]
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.runBeforeRequestHandlers(AmazonHttpClient.java:845) ~[atoti-aws.jar:na]
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:794) ~[atoti-aws.jar:na]
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:781) ~[atoti-aws.jar:na]
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:755) ~[atoti-aws.jar:na]
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:715) ~[atoti-aws.jar:na]
	at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:697) ~[atoti-aws.jar:na]
	at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:561) ~[atoti-aws.jar:na]
	at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:541) ~[atoti-aws.jar:na]
	at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5456) ~[atoti-aws.jar:na]
	at com.amazonaws.services.s3.AmazonS3Client.getBucketRegionViaHeadRequest(AmazonS3Client.java:6431) ~[atoti-aws.jar:na]
	at com.amazonaws.services.s3.AmazonS3Client.fetchRegionFromCache(AmazonS3Client.java:6404) ~[atoti-aws.jar:na]
	at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5441) ~[atoti-aws.jar:na]
	at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5403) ~[atoti-aws.jar:na]
	at com.amazonaws.services.s3.AmazonS3Client.getObjectMetadata(AmazonS3Client.java:1372) ~[atoti-aws.jar:na]
	at com.amazonaws.services.s3.AmazonS3Client.getObjectMetadata(AmazonS3Client.java:1346) ~[atoti-aws.jar:na]
	at com.amazonaws.services.s3.AmazonS3Client.doesObjectExist(AmazonS3Client.java:1427) ~[atoti-aws.jar:na]
	at io.atoti.loading.s3.impl.S3Path.exist(S3Path.java:244) ~[atoti-aws.jar:na]
	at io.atoti.loading.impl.AFileBasedDataTable.<init>(AFileBasedDataTable.java:28) ~[patachou-core-6.0.3-20230222-192541-08c2aaa0.jar!/:na]
	at io.atoti.loading.csv.impl.CsvDataTable.<init>(CsvDataTable.java:73) ~[patachou-core-6.0.3-20230222-192541-08c2aaa0.jar!/:na]
	at io.atoti.loading.csv.impl.CsvDataTableFactory.createTable(CsvDataTableFactory.java:33) ~[patachou-core-6.0.3-20230222-192541-08c2aaa0.jar!/:na]
	at io.atoti.loading.csv.impl.CsvDataTableFactory.createTable(CsvDataTableFactory.java:10) ~[patachou-core-6.0.3-20230222-192541-08c2aaa0.jar!/:na]
	at io.atoti.api.impl.OutsideTransactionDataApiImpl.discoverCsvFileFormat(OutsideTransactionDataApiImpl.java:92) ~[patachou-core-6.0.3-20230222-192541-08c2aaa0.jar!/:na]
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) ~[na:na]
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[na:na]
	at java.base/java.lang.reflect.Method.invoke(Unknown Source) ~[na:na]
	at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244) ~[py4j-0.10.9.jar!/:na]
	at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357) ~[py4j-0.10.9.jar!/:na]
	at py4j.Gateway.invoke(Gateway.java:282) ~[py4j-0.10.9.jar!/:na]
	at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132) ~[py4j-0.10.9.jar!/:na]
	at py4j.commands.CallCommand.execute(CallCommand.java:79) ~[py4j-0.10.9.jar!/:na]
	at py4j.ClientServerConnection.waitForCommands(ClientServerConnection.java:182) ~[py4j-0.10.9.jar!/:na]
	at py4j.ClientServerConnection.run(ClientServerConnection.java:106) ~[py4j-0.10.9.jar!/:na]
	at java.base/java.lang.Thread.run(Unknown Source) ~[na:na]
Caused by: java.net.SocketException: Network is unreachable: no further information
	at java.base/sun.nio.ch.Net.pollConnect(Native Method) ~[na:na]
	at java.base/sun.nio.ch.Net.pollConnectNow(Unknown Source) ~[na:na]
	at java.base/sun.nio.ch.NioSocketImpl.timedFinishConnect(Unknown Source) ~[na:na]
	at java.base/sun.nio.ch.NioSocketImpl.connect(Unknown Source) ~[na:na]
	at java.base/java.net.Socket.connect(Unknown Source) ~[na:na]
	at java.base/sun.net.NetworkClient.doConnect(Unknown Source) ~[na:na]
	at java.base/sun.net.www.http.HttpClient.openServer(Unknown Source) ~[na:na]
	at java.base/sun.net.www.http.HttpClient.openServer(Unknown Source) ~[na:na]
	at java.base/sun.net.www.http.HttpClient.<init>(Unknown Source) ~[na:na]
	at java.base/sun.net.www.http.HttpClient.New(Unknown Source) ~[na:na]
	at java.base/sun.net.www.http.HttpClient.New(Unknown Source) ~[na:na]
	at java.base/sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(Unknown Source) ~[na:na]
	at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect0(Unknown Source) ~[na:na]
	at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect(Unknown Source) ~[na:na]
	at java.base/sun.net.www.protocol.http.HttpURLConnection.connect(Unknown Source) ~[na:na]
	at com.amazonaws.internal.ConnectionUtils.connectToEndpoint(ConnectionUtils.java:95) ~[atoti-aws.jar:na]
	at com.amazonaws.internal.EC2ResourceFetcher.doReadResource(EC2ResourceFetcher.java:80) ~[atoti-aws.jar:na]
	... 46 common frames omitted

Environment

  • atoti: 0.7.3
  • Python: 3.9.13
  • Operating system: win32

Logs (if relevant)

server.log
@HuifangYeo HuifangYeo added the 🐛 bug unexpected or wrong behavior label Feb 27, 2023
@tibdex
Copy link
Member

tibdex commented Feb 27, 2023

Hi Hui Fang, could you please share a minimal reproducible example to help the server team investigate the issue?

@HuifangYeo
Copy link
Contributor Author

HuifangYeo commented Mar 2, 2023 •
edited

Hi @tibdex,
any notebook from the gallery that loads file from the s3 server has this exception in the log.
Please see the attached notebook for instance:
Notebook: atoti.zip
server log: server.log

@HuifangYeo HuifangYeo assigned tibdex and unassigned HuifangYeo Mar 2, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@tibdex tibdex

Labels
🐛 bug unexpected or wrong behavior
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@tibdex @HuifangYeo