What's expected in RisingLight v0.2?

[skyzh]

Before outlining the full roadmap for RisingLight v0.2, I'd like to draft some ideas.

The main objective for RisingLight v0.2 is to support sorting everywhere -- enable MergeIterator on storage, add executors like SortMergeJoin and SortAgg, add optimization rules for sort, etc.

• Refactor catalog to support composite primary key.
• Add optimization rules about primary keys. e.g., use sort merge join and sorted table scan for them.
• Implement executors like sort merge join and sort agg.
• (?) Support clustered index in storage. e.g., one user table can corresponds to multiple storage tables with different primary keys, so as to accelerate query.

At the same time, we should also resolve some long-lasting issues in RisingLight and try refactoring them.

• Refactor catalog. There should be no locks in catalog. It should be simply an MVCC hashmap.
• Support snapshot isolation in storage. Currently, we record rowset ids in every table, and the transaction-related APIs are over a single table. We should move it to database-level.
• Full support of column pruning framework, and resolve the issue of count(*).
• Refactor binder. We should come up with a plan to refactor it (or create a new one), move some functionalities to planner and optimizer, and make it context-aware, so that we can implement query un-nesting easier.

If you have any ideas, welcome to comment! Also, we should create tracking issues about these big features.

[wangrunji0408]

I can think of 2 big features in SQL: transaction and nested query support.
Besides, we should setup benchmarks for TPC-H queries ASAP, then start turning RisingLight into a high performance OLAP system!

[arkbriar]

Besides, I think we can support the "explain analyze" statement for quick evaluation of the performance of each executor and as the guidance to the further optimization.

[skyzh]

One more thing, the expression framework. We should support arithmetic operations between different integer types, SQL functions, etc.