Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: support tidb as one of the storage databases #5808

Open
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

Icemap
Copy link

@Icemap Icemap commented Apr 27, 2024

What changed?

Supported TiDB as one of the storage databases for Temporal.
Originally, copied from MySQL support, but changed some points followed:

  1. TiDB doesn't support the read lock or shared lock, so I upgraded it to write lock.
  2. TiDB doesn't Full Text Index, but TiDB has a component named TiFlash, it's column-based storage to boost the analytic queries. So I deleted the Full Text Index and used column_name [not] like '%XXX%' instead.

Why?

TiDB (/’taɪdiːbi:/, "Ti" stands for Titanium) is an open-source distributed SQL database that supports Hybrid Transactional and Analytical Processing (HTAP) workloads. It is MySQL compatible and features horizontal scalability, strong consistency, and high availability. The goal of TiDB is to provide users with a one-stop database solution that covers OLTP (Online Transactional Processing), OLAP (Online Analytical Processing), and HTAP services. TiDB is suitable for various use cases that require high availability and strong consistency with large-scale data.

So I think TiDB and Temporal will have bilateral benefits if we integrate them together.

How did you test it?

Adopted all MySQL's unit tests to TiDB. And all pasted.
You can follow this way to test it.

  1. Edit config file, the name will be config.toml:
port = 4000
lease = "0"
path = "/tmp/tidb"
socket = ""

# When create table, split a separated region for it. It is recommended to
# turn off this option if there will be a large number of tables created.
split-table = false

# Whether new collations are enabled, as indicated by its name, this configuration entry take effect ONLY when a TiDB cluster bootstraps for the first time.
new_collations_enabled_on_first_bootstrap = true

[log]
level = "error"

[status]
status-port = 9082

[performance]
stats-lease = "0"

[prepared-plan-cache]
enabled = true

# for temporal test
[experimental]
allow-expression-index = true
  1. Start the TiDB instance with this config file:
./tidb-server -config config.toml -store unistore -path "" -lease 0s > tidb.log 2>&1 &
  1. Log in and create a new test account:
mysql -h 127.0.0.1 -P 4000 -u root

Within mysql shell, create user and password:

CREATE USER 'temporal'@'%' IDENTIFIED BY 'temporal';
GRANT ALL PRIVILEGES ON *.* TO 'temporal'@'%';
FLUSH PRIVILEGES;
  1. Shift to the Temporal path, and run the test cases of TiDB:
go test -run ^TestTiDB go.temporal.io/server/common/persistence/tests -test.v

Here is a capture that shows the tested result from my side:
image

Potential risks

Nope

Documentation

Yes, I added a new document in docs/development/macos/tidb.md and changed the docs/development/run-dependencies-host.md.

Is hotfix candidate?

No

@Icemap Icemap requested a review from a team as a code owner April 27, 2024 13:30
@CLAassistant
Copy link

CLAassistant commented Apr 27, 2024

CLA assistant check
All committers have signed the CLA.

@Icemap
Copy link
Author

Icemap commented May 6, 2024

Hi @stephanos, @pdoerner. Could you PTAL? Or redirect it to who can help. THX a lot.

@sicheng-db
Copy link

sicheng-db commented May 7, 2024

Looking forward for supporting TiDB in Temporal. Super excited as MySQL would likely become the performance bottleneck.

Copy link
Contributor

@rodrigozhou rodrigozhou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately, we currently don't have expertise to maintain support for TiDB as we develop new features to add this to the codebase.

We have plans to make persistence layer more pluggable, so you are able to maintain this as separate project from Temporal.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants