Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: crud users synchronously #3928

Merged
merged 17 commits into from
May 20, 2024

Conversation

oh2024
Copy link
Collaborator

@oh2024 oh2024 commented May 15, 2024

  • What kind of change does this PR introduce?
    feature

  • What is the current behavior? (You can also link to an open issue here)

  1. users cruds are performed from sdk
  2. tablets and nameservers asynchronously sync user table every 100ms
  • What is the new behavior (if this is a feature change)?
  1. user cruds go through nameserver
  2. nameserver synchronously flushes latest user table to all healthy tablets
  3. tablets and nameservers asynchronously sync user table every 15 minutes

@github-actions github-actions bot added the storage-engine openmldb storage engine. nameserver & tablet label May 15, 2024
Copy link
Contributor

github-actions bot commented May 15, 2024

SDK Test Report

102 files  ±0  102 suites  ±0   2m 22s ⏱️ +6s
359 tests ±0  345 ✅ ±0  14 💤 ±0  0 ❌ ±0 
487 runs  ±0  473 ✅ ±0  14 💤 ±0  0 ❌ ±0 

Results for commit 021d867. ± Comparison against base commit ba817e4.

This pull request removes 30 and adds 9 tests. Note that renamed tests count towards both.
  PARTITION BY db1.t1.col2 ORDER BY db1.t1.col1
  PARTITION BY t1.col2 ORDER BY t1.col1
  ROWS_RANGE BETWEEN 3 PRECEDING AND CURRENT ROW
 ) limit 10;](1)
 ) limit 10;](2)
 ) limit 10;](3)
 FROM db1.t1
 FROM t1
 WINDOW w1 AS (
 last join db2.t2 order by db2.t2.col1
…
com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlLastJoinWithMultipleDB[,  SELECT sum(db1.t1.col1) over w1 as sum_t1_col1, db2.t2.str1 as t2_str1
 FROM db1.t1
 last join db2.t2 order by db2.t2.col1
 on db1.t1.col1 = db2.t2.col1 and db1.t1.col2 = db2.t2.col0
 WINDOW w1 AS (
  PARTITION BY db1.t1.col2 ORDER BY db1.t1.col1
  ROWS_RANGE BETWEEN 3 PRECEDING AND CURRENT ROW
 ) limit 10;](2)
com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlLastJoinWithMultipleDB[db1,  SELECT sum(t1.col1) over w1 as sum_t1_col1, db2.t2.str1 as t2_str1
 FROM t1
 last join db2.t2 order by db2.t2.col1
 on t1.col1 = db2.t2.col1 and t1.col2 = db2.t2.col0
 WINDOW w1 AS (
  PARTITION BY t1.col2 ORDER BY t1.col1
  ROWS_RANGE BETWEEN 3 PRECEDING AND CURRENT ROW
 ) limit 10;](1)
com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlLastJoinWithMultipleDB[null,  SELECT sum(db1.t1.col1) over w1 as sum_t1_col1, db2.t2.str1 as t2_str1
 FROM db1.t1
 last join db2.t2 order by db2.t2.col1
 on db1.t1.col1 = db2.t2.col1 and db1.t1.col2 = db2.t2.col0
 WINDOW w1 AS (
  PARTITION BY db1.t1.col2 ORDER BY db1.t1.col1
  ROWS_RANGE BETWEEN 3 PRECEDING AND CURRENT ROW
 ) limit 10;](3)
com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlMultipleDBErrorTest[, SELECT db2.t2.str1 as t2_str1
 FROM t1
 last join db2.t2 order by db2.t2.col1
 on t1.col1 = db2.t2.col1 and t1.col2 = db2.t2.col0;
, SQL parse error: Fail to transform data provider op: table t1 not exists in database []](4)
com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlMultipleDBErrorTest[db1, SELECT db1.t2.str1 as t2_str1
 FROM t1
 last join db2.t2 order by db2.t2.col1
 on t1.col1 = db2.t2.col1 and t1.col2 = db2.t2.col0;
, SQL parse error: Column Not found: db1.t2.str1](2)
com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlMultipleDBErrorTest[db1, SELECT db2.t2.str1 as t2_str1
 FROM t1
 last join db2.t2 order by db2.t2.col1
 on t1.col1 = t2.col1 and t1.col2 = db2.t2.col0;
, SQL parse error: Column Not found: .t2.col1](3)
com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlMultipleDBErrorTest[db1, SELECT t2.str1 as t2_str1
 FROM t1
 last join db2.t2 order by db2.t2.col1
 on t1.col1 = db2.t2.col1 and t1.col2 = db2.t2.col0;
, SQL parse error: Column Not found: .t2.str1](1)
com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlMultipleDBErrorTest[null, SELECT db2.t2.str1 as t2_str1
 FROM t1
 last join db2.t2 order by db2.t2.col1
 on t1.col1 = db2.t2.col1 and t1.col2 = db2.t2.col0;
, SQL parse error: Fail to transform data provider op: table t1 not exists in database []](5)
com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlWindowLastJoin[ SELECT sum(t1.col1) over w1 as sum_t1_col1, t2.str1 as t2_str1
 FROM t1
 last join t2 order by t2.col1
 on t1.col1 = t2.col1 and t1.col2 = t2.col0
 WINDOW w1 AS (
  PARTITION BY t1.col2 ORDER BY t1.col1
  ROWS_RANGE BETWEEN 3 PRECEDING AND CURRENT ROW
 ) limit 10;](1)

♻️ This comment has been updated with latest results.

Copy link

codecov bot commented May 15, 2024

Codecov Report

Attention: Patch coverage is 87.50000% with 14 lines in your changes are missing coverage. Please review.

Project coverage is 75.21%. Comparing base (ba817e4) to head (021d867).

Files Patch % Lines
src/sdk/sql_cluster_router.cc 62.50% 6 Missing ⚠️
src/nameserver/name_server_impl.cc 90.56% 5 Missing ⚠️
src/client/ns_client.cc 89.47% 2 Missing ⚠️
src/client/tablet_client.cc 85.71% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               main    #3928      +/-   ##
============================================
+ Coverage     75.18%   75.21%   +0.02%     
  Complexity      711      711              
============================================
  Files           752      752              
  Lines        135230   135261      +31     
  Branches       1474     2013     +539     
============================================
+ Hits         101676   101732      +56     
+ Misses        33251    33226      -25     
  Partials        303      303              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Contributor

github-actions bot commented May 15, 2024

Linux Test Report

    59 files  +     5     252 suites  +193   1h 42m 51s ⏱️ + 1h 16m 1s
13 520 tests +12 962  13 513 ✅ +12 959  7 💤 +3  0 ❌ ±0 
19 197 runs  +18 638  19 190 ✅ +18 635  7 💤 +3  0 ❌ ±0 

Results for commit 021d867. ± Comparison against base commit ba817e4.

♻️ This comment has been updated with latest results.

Copy link
Collaborator

@tobegit3hub tobegit3hub left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@oh2024 oh2024 merged commit 21184d5 into 4paradigm:main May 20, 2024
23 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
storage-engine openmldb storage engine. nameserver & tablet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants