Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Batch insert in \copy to DB destination #458

Open
murfffi opened this issue Apr 12, 2024 · 1 comment
Open

Batch insert in \copy to DB destination #458

murfffi opened this issue Apr 12, 2024 · 1 comment

Comments

@murfffi
Copy link
Contributor

murfffi commented Apr 12, 2024

Most databases support multi-row INSERT statements so
it is possible to speed up \copy by using batches. This issue is discussed in the PR for Clickhouse \copy implementation -
#457 .

Specifically, Clickhouse doesn't benefit from multi-row inserts within a Go SQL transaction, because the Clickhouse driver implements the Go transactions as batches anyway. All INSERTs in a transactions are written as a single batch always. Furthermore, the Clickhouse Go driver doesn't seem to support including multiple rows in an Exec on a prepared statement within a transaction/batch.

Still this feature can be implemented for other DBs.

@murfffi
Copy link
Contributor Author

murfffi commented Apr 12, 2024

I volunteer to do this after #457 .

@murfffi murfffi changed the title Batch insert in \copy to Clickhouse DB destination Batch insert in \copy to DB destination Apr 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant