Query task management plan design discussion and Q&A

[shilinlee]

feature list:

1. support for assigning a unique ID to each query

ID range example:

sql0: [0,         100000000)
sql1: [100000000, 200000000)
sql2: [200000000, 300000000)
sql3: [300000000, 400000000)
...

2. support show queries

> show queries

qid      query      database  duration   status  host
---      -----      -------   --------   -----   -----
100000001  select...  db0      123us       running 127.0.0.1:8088 

3. support show queries

> kill query 100000001

[shilinlee]

Query Task Manage Design

[TOC]

Attach Query

ts-sql

• When ts-sql starts, it interacts with ts-meta to let ts-meta allocate a sufficient range of ID segments to the current ts-sql. The recommended range is 10w.

sql0: [0, 100000000)
sql1: [100000000, 200000000)
sql2: [200000000, 300000000)
sql3: [300000000, 400000000)
...

- ts-sql assigns a unique ID(qid) to each query, and saves the `SQL` and `qid` in ts-sql memory.

- ts-sql also passes the `qid` to the SQL that needs to be executed on the ts-store.

### ts-meta

- ts-meta needs to save the relationship between ts-sql's identity (IP) and ID range.

```go
const QueryIDSpan 100000000

type Data struct {
     ......
    // Query ID range segment allocated by all sql nodes
    QueryIDInit map[string]uint64 // {"127.0.0.1:8086": 0, "127.0.0.2:8086": 100000000, "127.0.0.3:8086": 200000000}, span is QueryIDSpan
}

QueryIDInit: If it is a cloud-native scenario, the key in the map needs to be the domain:port of sql.

why: In this way, when ts-meta restarts or the leader changes, the QueryIDInit information will not be lost.

ts-store

• ts-store saves the SQL and qid in ts-store memory, like ts-sql TaskManager.AttachQuery.

Detach Query

ts-store

When the query task ends, remove the current query from the taskManager.

ts-sql

When the query task ends, remove the current query from the taskManager.

Pay attention:

if any errors occur in the query, you still need to remove the query while logging the cause of the error.

Show Queries

Returns all queries for the current ts-sql node and all ts-store nodes.

Merge the queries of all ts-store nodes according to qid.

> show queries

qid      query      database  duration   status   host
---      -----      -------   --------   -----    -----
100000001  select...  db0       10m        running  all                            -> all ts-store nodes
200000001  select...  db0       20m        running  127.0.0.1:8086                 -> partion ts-store nodes
200000001  select...  db0       20m        killed   127.0.0.2:8086,127.0.0.3:8086  -> partion ts-store nodes                                                

• status: running/killed/unknow

• host: If the query status of all ts-stores is running, the host value is all.

Kill Query

> kill query 100000001

• Send kill request to current ts-sql and all ts-store nodes.
• If there is a corresponding qid, mark the query as killed.