-
Notifications
You must be signed in to change notification settings - Fork 1k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[#22257] YSQL: Build and run pg_cron on YB
Summary:
Enable the use of pg_cron in YugabyteDB for testing purposes.
This change allows us to schedule and test the cron APIS. There is no cron leader or worker node yet, so every tserver node will behave as its own leader. This means that each cron job will run N (number of tserver nodes) times instead of running only once.
- Feature is guarded under NON_RUNTIME gFlag `TEST_enable_pg_cron`.
- Fixed `YBCDeleteSysCatalogTuple` to use `relfileNodeId` to create the `delete_stmt`.
- Use `YBCDeleteHeapTuple` in pg_cron instead of `simple_heap_delete`.
- Set the default cron database to `yugabyte`.
- Changes the default for `use_background_workers` to `true` since yb will only use remote execution in the future.
- Cron runner will always call `RefreshTaskHash` once every `cron.yb_job_list_refresh_interval` seconds. Since jobs are scheduled on remote nodes which cannot send invalidation messages, we need to reload the table regularly. This also means that tasks will continue to get scheduled for upto 1 minute after they are removed.
- Added `YBCPgResetCatalogReadTime()` to the launcher loop. Without this we cache the catalog read time and never update it resulting is stale job table reads.
```
CREATE EXTENSION pg_cron;
CREATE TABLE tbl1(a INT, i_time TIMESTAMP DEFAULT CURRENT_TIMESTAMP);
SELECT cron.schedule('tbl1', '* * * * *', 'INSERT INTO tbl1 VALUES(1);');
SELECT * from cron.job_run_details ORDER BY runid;
\watch 60
```
Fixes #22257
Jira: DB-11175
Test Plan: TestPgRegressThirdPartyExtensionsPgCron
Reviewers: fizaa, tnayak
Reviewed By: fizaa, tnayak
Subscribers: yql, ybase
Differential Revision: https://phorge.dev.yugabyte.com/D34750- Loading branch information
Showing
7 changed files
with
122 additions
and
13 deletions.
There are no files selected for viewing
42 changes: 42 additions & 0 deletions
42
java/yb-pgsql/src/test/java/org/yb/pgsql/TestPgRegressThirdPartyExtensionsPgCron.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,42 @@ | ||
| // Copyright (c) YugabyteDB, Inc. | ||
| // | ||
| // Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except | ||
| // in compliance with the License. You may obtain a copy of the License at | ||
| // | ||
| // http://www.apache.org/licenses/LICENSE-2.0 | ||
| // | ||
| // Unless required by applicable law or agreed to in writing, software distributed under the License | ||
| // is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express | ||
| // or implied. See the License for the specific language governing permissions and limitations | ||
| // under the License. | ||
| // | ||
| package org.yb.pgsql; | ||
|
|
||
| import java.io.File; | ||
| import java.util.Map; | ||
| import org.junit.Test; | ||
| import org.junit.runner.RunWith; | ||
| import org.yb.YBTestRunner; | ||
| import org.yb.client.TestUtils; | ||
|
|
||
| @RunWith(value = YBTestRunner.class) | ||
| public class TestPgRegressThirdPartyExtensionsPgCron extends BasePgSQLTest { | ||
| @Override | ||
| public int getTestMethodTimeoutSec() { | ||
| return 1800; | ||
| } | ||
|
|
||
| @Override | ||
| protected Map<String, String> getTServerFlags() { | ||
| Map<String, String> flagMap = super.getTServerFlags(); | ||
| flagMap.put("TEST_enable_pg_cron", "true"); | ||
| return flagMap; | ||
| } | ||
|
|
||
| @Test | ||
| public void schedule() throws Exception { | ||
| runPgRegressTest( | ||
| new File(TestUtils.getBuildRootDir(), "postgres_build/third-party-extensions/pg_cron"), | ||
| "yb_schedule"); | ||
| } | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,3 @@ | ||
| # third-party-extensions/pg_cron/yb_schedule | ||
|
|
||
| test: pg_cron-test |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters