ADBC connection standard feature

[simonaubertbd]

Is your feature request related to a problem? Please describe.
Hello all,

ADBC is a database connection standard (like ODBC or JDBC) but specifically designed for columnar storage (so database like DuckDB, Clickhouse, MonetDB, Vertica...). This is typically the kind of stuff that can make MonetDB way faster in user experience.

more info in https://arrow.apache.org/blog/2023/01/05/introducing-arrow-adbc/

Describe the solution you'd like
Implementation of ADBC for monetdb

Describe alternatives you've considered
N/A

Additional context
Here the benchmark with DuckDB
https://duckdb.org/2023/08/04/adbc.html

[njnes]

That column orientation helps when ingesting large tables is known (thats basically why the ongoing projects on copy into and binary data shipping are done). Having yet another interface could for some cases help. But in many cases (output through odbc/adbc) is small, which should reduce the difference a lot. This benchmark your looking at isn't that fair in that sense. For which application/use case are you looking into ADBC.

[simonaubertbd]

Hello @njnes
Well, you answered that fast :)
So, I have two use cases in mind :
-first one would be an etl/data preparation tools that have to insert huge amount of data. ADBC can improve the performance without having to implement a bulk load logic which seems more specific.
-second one is when you have to retrieve thousands of rows with a select in your tool, especially on datavis software. On Tableau, the user experience is awful. (and yes, I perfectly know as of today Tableau does not support ADBC yet)

Am I wrong with that?

Best regards,

Simon

[njnes]

Both indeed require ADBC on both ends, for the later (visualisation) I'd expect the data transfer to improve from this (allthough I'm not so agreeing on the huge volumes going over the line). But on the etl I'd expect you to transfer from an row store, which I guess will put more strain on your production system, which your dba's will not like.
Still its a nice api, still cleaner etc than the odbc, looking for people to take this up ;-)

[arjenderijke]

For Go there is a package available, that helps with starting an implementation. I had a quick look and there is a part that maps easily on the existing Go driver. But that only part of the adbc spec. But there is a lot of functionality that is different than the existing drivers. Trying to make a Go implementation might be a nice way to figure out how ADBC really works. But first i will need to implement the some additional features in the MonetDB-Go package, such as the binary resultset. That would be required anyway, for adbc. I cannot promise anything, but it is an interesting challenge