New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix: possible race condition on EXCHANGE TABLES
query
#61135
base: master
Are you sure you want to change the base?
Changes from 15 commits
558d764
f8b4ef0
2de26fa
2f051c7
067a8c8
2825656
064eca6
c66d8e1
792a298
b24629d
5955cc5
e121665
190823f
4eeff80
c67a898
0eae42a
242dc60
57627ec
856cb21
1bd25d0
c84b702
f0d55ea
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -912,6 +912,23 @@ String Context::getFilesystemCacheUser() const | |
return shared->filesystem_cache_user; | ||
} | ||
|
||
DatabaseAndTable Context::getOrCacheStorage(const StorageID & id, std::function<DatabaseAndTable()> f, std::optional<Exception> * exception) const | ||
{ | ||
if (auto storage_id = storage_cache.find(id); storage_id != storage_cache.end()) | ||
{ | ||
DatabaseAndTable storage = DatabaseCatalog::instance().tryGetByUUID(storage_id->uuid); | ||
if (exception && !storage.second) | ||
exception->emplace(Exception(ErrorCodes::UNKNOWN_TABLE, "Table {} does not exist anymore - maybe it was dropped", id.getNameForLogs())); | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Currently, a query will fail if it requests a table the second time, but the table was dropped after the first get. But we can improve it later There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't think we need to improve it - if table was dropped concurrently then query is better to be interrupted than trying to reposes the table from drop queue - at this point we don't even know who is holding it from physical deletion... There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We do know, see |
||
return storage; | ||
} | ||
|
||
auto storage = f(); | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Consider a better name |
||
if (storage.second) | ||
if (const auto & new_id = storage.second->getStorageID(); new_id.hasUUID()) | ||
storage_cache.insert(new_id); | ||
return storage; | ||
} | ||
|
||
Strings Context::getWarnings() const | ||
{ | ||
Strings common_warnings; | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,27 @@ | ||
#!/usr/bin/env bash | ||
# Tags: no-ordinary-database | ||
|
||
set -e | ||
|
||
CURDIR=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd) | ||
# shellcheck source=../shell_config.sh | ||
. "$CURDIR"/../shell_config.sh | ||
|
||
${CLICKHOUSE_CLIENT} --multiquery <<EOF | ||
DROP TABLE IF EXISTS tbl_03007_1; | ||
DROP TABLE IF EXISTS tbl_03007_2; | ||
CREATE TABLE tbl_03007_1 (n Float64) ENGINE=Memory; | ||
CREATE TABLE tbl_03007_2 (n Int256) ENGINE=Memory; | ||
EOF | ||
|
||
for _ in {1..50}; do | ||
(! ${CLICKHOUSE_CLIENT} --query "SELECT n * 0.123 FROM (SELECT * FROM tbl_03007_1)" 2>&1 | grep LOGICAL_ERROR) & | ||
${CLICKHOUSE_CLIENT} --query "EXCHANGE TABLES tbl_03007_1 AND tbl_03007_2" & | ||
done | ||
|
||
wait | ||
|
||
${CLICKHOUSE_CLIENT} --multiquery <<EOF | ||
DROP TABLE IF EXISTS tbl_03007_1; | ||
DROP TABLE IF EXISTS tbl_03007_2; | ||
EOF |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We could cache it in
Context::resolveStorageID
, but a new method is fine tooThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think we want to enforce this cache everywhere where we call to
Context::resolveStorageID
- at least having a query context is a condition we need to use this cache.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We can check this condition in resolveStorageID