Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Unable to start Trino after using PrestoSQL-TiDB-Connector #110

Open
myheartsgoon opened this issue Sep 28, 2021 · 6 comments
Open

Unable to start Trino after using PrestoSQL-TiDB-Connector #110

myheartsgoon opened this issue Sep 28, 2021 · 6 comments

Comments

@myheartsgoon
Copy link

I followed this doc to build the connector and the build finished successfully.
https://github.com/tidb-incubator/TiBigData/tree/master/prestosql

my tidb plugin folder content as blew:

root@trino-worker01:/home/trino/trino-server-361/etc/catalog# ls -l /home/trino/trino-server-361/plugin/tidb/
total 38940
-rw-r--r-- 1 root root     4467 9月  28 15:03 aopalliance-1.0.jar
-rw-r--r-- 1 root root    44329 9月  28 15:03 bigdata-core-0.0.5-SNAPSHOT.jar
-rw-r--r-- 1 root root    26353 9月  28 15:03 bootstrap-202.jar
-rw-r--r-- 1 root root   805301 9月  28 15:03 bval-jsr-2.0.0.jar
-rw-r--r-- 1 root root   415348 9月  28 15:03 cglib-nodep-3.3.0.jar
-rw-r--r-- 1 root root   230905 9月  28 15:03 checker-qual-3.8.0.jar
-rw-r--r-- 1 root root    83762 9月  28 15:03 configuration-202.jar
-rw-r--r-- 1 root root    13854 9月  28 15:03 error_prone_annotations-2.5.1.jar
-rw-r--r-- 1 root root     4617 9月  28 15:03 failureaccess-1.0.1.jar
-rw-r--r-- 1 root root  2874025 9月  28 15:03 guava-30.1.1-jre.jar
-rw-r--r-- 1 root root   856934 9月  28 15:03 guice-4.2.3.jar
-rw-r--r-- 1 root root   156005 9月  28 15:03 HikariCP-3.4.5.jar
-rw-r--r-- 1 root root     8781 9月  28 15:03 j2objc-annotations-1.3.jar
-rw-r--r-- 1 root root    68167 9月  28 15:03 jackson-annotations-2.11.1.jar
-rw-r--r-- 1 root root   349108 9月  28 15:03 jackson-core-2.10.3.jar
-rw-r--r-- 1 root root  1404171 9月  28 15:03 jackson-databind-2.10.3.jar
-rw-r--r-- 1 root root    83893 9月  28 15:03 jackson-datatype-guava-2.10.3.jar
-rw-r--r-- 1 root root    34402 9月  28 15:03 jackson-datatype-jdk8-2.10.3.jar
-rw-r--r-- 1 root root    72117 9月  28 15:03 jackson-datatype-joda-2.10.3.jar
-rw-r--r-- 1 root root   105898 9月  28 15:03 jackson-datatype-jsr310-2.10.3.jar
-rw-r--r-- 1 root root     9328 9月  28 15:03 jackson-module-parameter-names-2.10.3.jar
-rw-r--r-- 1 root root    26586 9月  28 15:03 javax.annotation-api-1.3.2.jar
-rw-r--r-- 1 root root     2497 9月  28 15:03 javax.inject-1.jar
-rw-r--r-- 1 root root    16537 9月  28 15:03 jcl-over-slf4j-1.7.30.jar
-rw-r--r-- 1 root root  2920901 9月  28 15:03 jmxutils-1.21.jar
-rw-r--r-- 1 root root    32601 9月  28 15:03 json-202.jar
-rw-r--r-- 1 root root    19936 9月  28 15:03 jsr305-3.0.2.jar
-rw-r--r-- 1 root root     2199 9月  28 15:03 listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar
-rw-r--r-- 1 root root     3908 9月  28 15:03 log-202.jar
-rw-r--r-- 1 root root    23702 9月  28 15:03 log4j-over-slf4j-1.7.30.jar
-rw-r--r-- 1 root root   471901 9月  28 15:03 logback-core-1.2.3.jar
-rw-r--r-- 1 root root    19470 9月  28 15:03 log-manager-202.jar
-rw-r--r-- 1 root root  1006956 9月  28 15:45 mysql-connector-java-5.1.48.jar
-rw-r--r-- 1 root root    48203 9月  28 15:03 prestosql-connector-0.0.5-SNAPSHOT.jar
-rw-r--r-- 1 root root    41203 9月  28 15:03 slf4j-api-1.7.25.jar
-rw-r--r-- 1 root root     8510 9月  28 15:03 slf4j-jdk14-1.7.30.jar
-rw-r--r-- 1 root root   103273 9月  28 15:03 tidb-jdbc-0.0.5-SNAPSHOT.jar
-rw-r--r-- 1 root root 27276468 9月  28 15:03 tikv-client-java-3.2.0-SNAPSHOT.jar
-rw-r--r-- 1 root root    19555 9月  28 15:03 units-1.6.jar
-rw-r--r-- 1 root root    93107 9月  28 15:03 validation-api-2.0.1.Final.jar

And I add the catalog for tidb, content as blow.

connector.name=tidb
tidb.database.url=jdbc:mysql://xxx:3306/xxx
tidb.username=xxx
tidb.password=xxx
tidb.filter-push-down=true

But after I restart the trino, it errored as below.

2021-09-28T15:49:51.601+0800    ERROR   main    io.trino.server.Server  No service providers of type io.trino.spi.Plugin
java.lang.IllegalStateException: No service providers of type io.trino.spi.Plugin
        at com.google.common.base.Preconditions.checkState(Preconditions.java:591)
        at io.trino.server.PluginManager.loadPlugin(PluginManager.java:139)
        at io.trino.server.PluginManager.loadPlugin(PluginManager.java:129)
        at io.trino.server.ServerPluginsProvider.loadPlugins(ServerPluginsProvider.java:48)
        at io.trino.server.PluginManager.loadPlugins(PluginManager.java:110)
        at io.trino.server.Server.doStart(Server.java:122)
        at io.trino.server.Server.lambda$start$0(Server.java:77)
        at io.trino.$gen.Trino_361____20210928_074943_1.run(Unknown Source)
        at io.trino.server.Server.start(Server.java:77)
        at io.trino.server.TrinoServer.main(TrinoServer.java:38)

My trino version: 361
mysql connector I use: mysql-connector-java-5.1.48.jar

Please help if anything wrong with my setup?
@myheartsgoon
Copy link
Author

myheartsgoon commented Sep 28, 2021 •
edited

I manage to fixe the above issue by change prestosql to trino in build process. Now I am encountering below issue when start the trino server.

error when adding tidb.filter-push-down=true in the catalog file

If I add tidb.filter-push-down=true in the catalog file, below error will occurr.

2021-09-28T17:11:03.612+0800	ERROR	main	io.trino.server.Server	Configuration errors:

1) Error: Configuration property 'tidb.filter-push-down' was not used

1 error
io.airlift.bootstrap.ApplicationConfigurationException: Configuration errors:

1) Error: Configuration property 'tidb.filter-push-down' was not used

1 error
	at io.airlift.bootstrap.Bootstrap.initialize(Bootstrap.java:233)
	at io.tidb.bigdata.trino.tidb.TiDBConnectorFactory.create(TiDBConnectorFactory.java:57)
	at io.trino.connector.ConnectorManager.createConnector(ConnectorManager.java:360)
	at io.trino.connector.ConnectorManager.createCatalog(ConnectorManager.java:215)
	at io.trino.connector.ConnectorManager.createCatalog(ConnectorManager.java:207)
	at io.trino.connector.ConnectorManager.createCatalog(ConnectorManager.java:193)
	at io.trino.metadata.StaticCatalogStore.loadCatalog(StaticCatalogStore.java:88)
	at io.trino.metadata.StaticCatalogStore.loadCatalogs(StaticCatalogStore.java:68)
	at io.trino.server.Server.doStart(Server.java:124)
	at io.trino.server.Server.lambda$start$0(Server.java:77)
	at io.trino.$gen.Trino_361____20210928_091053_1.run(Unknown Source)
	at io.trino.server.Server.start(Server.java:77)
	at io.trino.server.TrinoServer.main(TrinoServer.java:38)


2021-09-28T17:11:03.614+0800	INFO	Thread-81	io.airlift.bootstrap.LifeCycleManager	JVM is shutting down, cleaning up
2021-09-28T17:11:03.614+0800	INFO	Thread-81	io.airlift.bootstrap.LifeCycleManager	Life cycle stopping...
2021-09-28T17:11:03.615+0800	INFO	Thread-81	io.airlift.bootstrap.LifeCycleManager	Life cycle stopped

error WITHOUT adding tidb.filter-push-down=true in the catalog file

If remove tidb.filter-push-down=true, then server can be started successfully, but when I run query in cli, Internal error will occurred.

trino:skyline_test> select * from test1;
Query 20210928_101552_00008_hcyiy failed: Internal error

In the stack trace log it shows as below:

java.lang.UnsupportedOperationException: undefined
	at io.trino.spi.connector.ConnectorSplitManager.getSplits(ConnectorSplitManager.java:35)
	at io.trino.split.SplitManager.getSplits(SplitManager.java:86)
	at io.trino.sql.planner.DistributedExecutionPlanner$Visitor.visitScanAndFilter(DistributedExecutionPlanner.java:206)
	at io.trino.sql.planner.DistributedExecutionPlanner$Visitor.visitTableScan(DistributedExecutionPlanner.java:188)
	at io.trino.sql.planner.DistributedExecutionPlanner$Visitor.visitTableScan(DistributedExecutionPlanner.java:159)
	at io.trino.sql.planner.plan.TableScanNode.accept(TableScanNode.java:222)
	at io.trino.sql.planner.DistributedExecutionPlanner.doPlan(DistributedExecutionPlanner.java:127)
	at io.trino.sql.planner.DistributedExecutionPlanner.doPlan(DistributedExecutionPlanner.java:134)
	at io.trino.sql.planner.DistributedExecutionPlanner.plan(DistributedExecutionPlanner.java:104)
	at io.trino.execution.SqlQueryExecution.planDistribution(SqlQueryExecution.java:508)
	at io.trino.execution.SqlQueryExecution.start(SqlQueryExecution.java:415)
	at io.trino.execution.SqlQueryManager.createQuery(SqlQueryManager.java:237)
	at io.trino.dispatcher.LocalDispatchQuery.lambda$startExecution$7(LocalDispatchQuery.java:143)
	at io.trino.$gen.Trino_361____20210928_100845_2.run(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)

@sunxiaoguang
Copy link
Collaborator

tidb.filter-push-down

Hi @myheartsgoon, thanks for reporting issue. Could you please try running the same query with trino version 359?

@myheartsgoon
Copy link
Author

tidb.filter-push-down

Hi @myheartsgoon, thanks for reporting issue. Could you please try running the same query with trino version 359?

Thanks for your prompt response, I just tested with version 359, when not specifying tidb.filter-push-down=true, it is working as expected. (I guess it is due to this change since 360? https://trino.io/docs/current/release/release-360.html#spi)

When I adding tidb.filter-push-down=true in catalog file, it is still not working, same error message as I describe above.

2021-09-29T11:47:43.085+0800	ERROR	main	io.trino.server.Server	Configuration errors:

1) Error: Configuration property 'tidb.filter-push-down' was not used

1 error
io.airlift.bootstrap.ApplicationConfigurationException: Configuration errors:

1) Error: Configuration property 'tidb.filter-push-down' was not used

1 error
	at io.airlift.bootstrap.Bootstrap.initialize(Bootstrap.java:233)
	at io.tidb.bigdata.trino.tidb.TiDBConnectorFactory.create(TiDBConnectorFactory.java:57)
	at io.trino.connector.ConnectorManager.createConnector(ConnectorManager.java:360)
	at io.trino.connector.ConnectorManager.createCatalog(ConnectorManager.java:215)
	at io.trino.connector.ConnectorManager.createCatalog(ConnectorManager.java:207)
	at io.trino.connector.ConnectorManager.createCatalog(ConnectorManager.java:193)
	at io.trino.metadata.StaticCatalogStore.loadCatalog(StaticCatalogStore.java:88)
	at io.trino.metadata.StaticCatalogStore.loadCatalogs(StaticCatalogStore.java:68)
	at io.trino.server.Server.doStart(Server.java:124)
	at io.trino.server.Server.lambda$start$0(Server.java:77)
	at io.trino.$gen.Trino_359____20210929_034733_1.run(Unknown Source)
	at io.trino.server.Server.start(Server.java:77)
	at io.trino.server.TrinoServer.main(TrinoServer.java:38)

But for now I am good as at least I can query tidb data with presto, although without filter pushdown feature.

@sunxiaoguang
Copy link
Collaborator

sunxiaoguang commented Sep 29, 2021 •
edited

Thanks for your prompt response, I just tested with version 359, when not specifying tidb.filter-push-down=true, it is working as expected. (I guess it is due to this change since 360? https://trino.io/docs/current/release/release-360.html#spi)

Trino / PrestoSQL / PrestoDB are evolving pretty fast, it's usually required to match version used to compile and run connector.

@humengyu2012 Can you take a look at configuration not used error?

@humengyu2012
Copy link
Collaborator

humengyu2012 commented Sep 29, 2021 •
edited by sunxiaoguang

Presto will always enable filter-push-down and can not be disabled, so you don't need to configure it.

@humengyu2012
Copy link
Collaborator

This is our fault, we will change the documentation later.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@sunxiaoguang @myheartsgoon @humengyu2012