Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Citus v10.x to v11.x or Newer Major Version Upgrade Error in Non-Transactional Metadata Sync Mode #7238

Open
gokhangulbiz opened this issue Sep 27, 2023 · 0 comments · May be fixed by #7239
Open

Citus v10.x to v11.x or Newer Major Version Upgrade Error in Non-Transactional Metadata Sync Mode #7238

gokhangulbiz opened this issue Sep 27, 2023 · 0 comments · May be fixed by #7239

Comments

@gokhangulbiz
Copy link
Contributor

When attempting to upgrade from Citus v10.x to newer major versions with a non-transactional metadata sync mode, an issue arises during the execution of the citus_finalize_upgrade_to_citus11() function by the following error message:

PL/pgSQL function citus_finish_citus_upgrade() line 25 at PERFORM
NOTICE: preparing all the existing partitioned table indexes
NOTICE: preparing to sync the metadata to all nodes

> Error: do not sync metadata in transaction block when the sync mode is nontransactional

HINT: resync after SET citus.metadata_sync_mode to 'transactional'
PL/pgSQL function citus_finalize_upgrade_to_citus11(boolean) line 174 at PERFORM
SQL statement "SELECT citus_finalize_upgrade_to_citus11()"

The problem specifically occurs when the transaction mode is set to non-transactional. It originates from the fact that metadata synchronization is called within a transaction block without considering the metadata sync mode. This behavior triggers the error message shown above. The relevant code section causing this issue can be found here:

citus/src/backend/distributed/sql/udfs/citus_finalize_upgrade_to_citus11/latest.sql

Lines 209 to 216 in 27ac44e

DECLARE
BEGIN
-- this might take long depending on the number of tables & objects ...
RAISE NOTICE 'Preparing to sync the metadata to all nodes';
PERFORM start_metadata_sync_to_all_nodes();
END;

The issue can be traced to the guard condition defined in the following code section:

citus/src/backend/distributed/metadata/node_metadata.c

Lines 1057 to 1072 in 27ac44e

void
ActivateNodeList(MetadataSyncContext *context)
{
if (context->transactionMode == METADATA_SYNC_NON_TRANSACTIONAL &&
IsMultiStatementTransaction())
{
/*
* prevent inside transaction block as we use bare connections which can
* lead deadlock
*/
ereport(ERROR, (errmsg("do not sync metadata in transaction block "
"when the sync mode is nontransactional"),
errhint("resync after SET citus.metadata_sync_mode "
"TO 'transactional'")));
}

I think we need to adjust the logic in citus_finalize_upgrade_to_citus11() to ensure that metadata synchronization is performed correctly based on the sync mode in use.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Resolve upgrade problem related to non-transactional metadata synchronization. citusdata/citus
1 participant
@gokhangulbiz