5.1.0

New feature

1. Support SQL hint
2. New DistSQL syntax: SHOW AUTHORITY RULE
3. New DistSQL syntax: SHOW TRANSACTION RULE
4. New DistSQL syntax: ALTER TRANSACTION RULE
5. New DistSQL syntax: SHOW SQL_PARSER RULE
6. New DistSQL syntax: ALTER SQL_PARSER RULE
7. New DistSQL syntax: ALTER DEFAULT SHARDING STRATEGY
8. New DistSQL syntax: DROP DEFAULT SHARDING STRATEGY
9. New DistSQL syntax: CREATE DEFAULT SINGLE TABLE RULE
10. New DistSQL syntax: SHOW SINGLE TABLES
11. New DistSQL syntax: SHOW SINGLE TABLE RULES
12. New DistSQL syntax: SHOW SHARDING TABLE NODES
13. New DistSQL syntax: CREATE/ALTER/DROP SHARDING KEY GENERATOR
14. New DistSQL syntax: SHOW SHARDING KEY GENERATORS
15. New DistSQL syntax: REFRESH TABLE METADATA
16. New DistSQL syntax: PARSE SQL, Output the abstract syntax tree obtained by parsing SQL
17. New DistSQL syntax: SHOW UNUSED SHARDING ALGORITHMS
18. New DistSQL syntax: SHOW UNUSED SHARDING KEY GENERATORS
19. New DistSQL syntax: CREATE/DROP SHARDING SCALING RULE
20. New DistSQL syntax: ENABLE/DISABLE SHARDING SCALING RULE
21. New DistSQL syntax: SHOW SHARDING SCALING RULES
22. New DistSQL syntax: SHOW INSTANCE MODE
23. New DistSQL syntax: COUNT SCHEMA RULES
24. Scaling: Add rateLimiter configuration and QPS TPS implementation
25. Scaling: Add DATA_MATCH data consistency check
26. Scaling: Add batchSize configuration to avoid possible OOME
27. Scaling: Add streamChannel configuration and MEMORY implementation
28. Scaling: Support MySQL BINARY data type
29. Scaling: Support MySQL YEAR data type
30. Scaling: Support PostgreSQL BIT data type
31. Scaling: Support PostgreSQL MONEY data type
32. Database discovery adds support for JDBC Spring Boot
33. Database discovery adds support for JDBC Spring Namespace
34. Database discovery adds support for openGauss
35. Shadow DB adds support for logical data source transfer
36. Add data type validator for column matching shadow algorithm
37. Add support for xa start/end/prepare/commit/recover in encrypt case with only one data source

API Change

1. Redesign the database discovery related DistSQL syntax
2. In DistSQL, the keyword GENERATED_KEY is adjusted to KEY_GENERATE_STRATEGY
3. Native authority provider is marked as deprecated and will be removed in a future version
4. Scaling: Move scaling configuration from server.yaml to config-sharding.yaml
5. Scaling: Rename clusterAutoSwitchAlgorithm SPI to completionDetector and refactor method parameter
6. Scaling: Data consistency check API method rename and return type change
7. Database discovery module API refactoring
8. Readwrite-splitting supports static and dynamic configuration
9. Shadow DB remove the enable configuration
10. Shadow algorithm type modified

Enhancement

1. Improve load multi single table performance
2. Remove automatically added order by primary key clause
3. Optimize binding table route logic without sharding column in join condition
4. Support update sharding key when the sharding routing result keep the same
5. Optimize rewrite engine performance
6. Support select union/union all ... statements by federation engine
7. Support insert on duplicate key update sharding column when route context keep same
8. Use union all to merge sql route units for simple select to improve performance
9. Supports autocommit in ShardingSphere-Proxy
10. ShardingSphere openGauss Proxy supports sha256 authentication method
11. Remove property java.net.preferIPv4Stack=true from Proxy startup script
12. Remove the verification of null rules for JDBC
13. Optimize performance of executing openGauss batch bind
14. Disable Netty resource leak detector by default
15. Supports describe prepared statement in PostgreSQL / openGauss Proxy
16. Optimize performance of executing PostgreSQL batched inserts
17. Add instance_id to the result of SHOW INSTANCE LIST
18. Support to use instance_id to perform operations when enable/disable a proxy instance
19. Support auto creative algorithm when CREATE SHARDING TABLE RULE, reducing the steps of creating rule
20. Support specifying an existing KeyGenerator when CREATE SHARDING TABLE RULE
21. DROP DATABASE supports IF EXISTS option
22. DATANODES in SHARDING TABLE RULE supports enumerated inline expressions
23. CREATE/ALTER SHARDING TABLE RULE supports complex sharding algorithm
24. SHOW SHARDING TABLE NODES supports non-inline scenarios (range, time, etc.)
25. When there is only one read data source in the readwrite-splitting rule, it is not allowed to be disabled
26. Scaling: Add basic support of chunked streaming data consistency check
27. Shadow algorithm decision logic optimization to improve performance

Refactor

1. Refactor federation engine scan table logic
2. Avoid duplicated TCL SQL parsing when executing prepared statement in Proxy
3. Scaling: Add pipeline modules to redesign scaling
4. Scaling: Refactor several job configuration structure
5. Scaling: Precalculate tasks splitting and persist in job configuration
6. Scaling: Add basic support of pipeline-core code reuse for encryption job
7. Scaling: Add basic support of scaling job and encryption job combined running
8. Scaling: Add input and output configuration, including workerThread and rateLimiter
9. Scaling: Move blockQueueSize into streamChannel
10. Scaling: Change jobId type from integer to text
11. Optimize JDBC to load only the specified schema
12. Optimize meta data structure of the registry center
13. Rename Note shadow algorithm to HINT shadow algorithm

Bug Fix

1. Support parsing function
2. Fix alter table drop constrain
3. Fix optimize table route
4. Support Route resource group
5. Support parsing binlog
6. Support postgreSql/openGauss '&' and '|' operator
7. Support parsing openGauss insert on duplicate key
8. Support parse postgreSql/openGauss union
9. Support query which table has column contains keyword
10. Fix missing parameter in function
11. Fix sub query table with no alias
12. Fix utc timestamp function
13. Fix alter encrypt column
14. Support alter column with position encrypt column
15. Fix delete with schema for postgresql
16. Fix wrong route result caused by oracle parser ambiguity
17. Fix projection count error when use sharding and encrypt
18. Fix npe when using shadow and readwrite_splitting
19. Fix wrong metadata when actual table is case insensitive
20. Fix encrypt rewrite exception when execute multiple table join query
21. Fix encrypt rewrite wrong result with table level queryWithCipherColumn
22. Fix parsing chinese
23. Fix encrypt exists sub query
24. Fix full route caused by the MySQL BINARY keyword in the sharding condition
25. Fix getResultSet method empty result exception when using JDBCMemoryQueryResult processing statement
26. Fix incorrect shard table validation logic when creating store function/procedure
27. Fix null charset exception occurs when connecting Proxy with some PostgreSQL client
28. Fix executing commit in prepared statement cause transaction status incorrect in MySQL Proxy
29. Fix client connected to Proxy may stuck if error occurred in PostgreSQL with non English locale
30. Fix file not found when path of configurations contains blank character
31. Fix transaction status may be incorrect cause by early flush
32. Fix the unsigned datatype problem when query with PrepareStatement
33. Fix protocol violation in implementations of prepared statement in MySQL Proxy
34. Fix caching too many connections in openGauss batch bind
35. Fix the problem of missing data in SHOW READWRITE_SPLITTING RULES when db-discovery and readwrite-splitting are used together
36. Fix the problem of missing data in SHOW READWRITE_SPLITTING READ RESOURCES when db-discovery and readwrite-splitting are used together
37. Fix the NPE when the CREATE SHARDING TABLE RULE statement does not specify the sub-database and sub-table strategy
38. Fix NPE when PREVIEW SQL by schema.table
39. Fix DISABLE statement could disable readwrite-splitting write data source in some cases
40. Fix DIABLE INSTANCE could disable the current instance in some cases
41. Fix the issue that user may query the unauthorized logic schema when the provider is SCHEMA_PRIVILEGES_PERMITTED
42. Fix NPE when authority provider is not configured
43. Scaling: Fix DB connection leak on XA initialization which triggered by data consistency check
44. Scaling: Fix PostgreSQL replication stream exception on multiple data sources
45. Scaling: Fix migrating updated record exception on PostgreSQL incremental phase
46. Scaling: Fix MySQL 5.5 check BINLOG_ROW_IMAGE option failure
47. Scaling: Fix PostgreSQL xml data type consistency check
48. Fix database discovery failed to modify cron configuration
49. Fix single read data source use weight loadbalance algorithm error
50. Fix create redundant data souce without memory mode
51. Fix column value matching shadow algorithm data type conversion exception

Change Log

1. MILESTONE

5.0.0

New feature

1. Support parsing SQL comment
2. New DistSQL syntax: shadow rule management
3. New DistSQL syntax: scaling job management
4. New DistSQL syntax: disable proxy instance
5. New DistSQL syntax: disable readwrite-splitting read data source
6. New DistSQL syntax: DROP SHARDING ALGORITHM
7. New DistSQL syntax: ALTER RESOURCE
8. New DistSQL syntax: CREATE SHARDING ALGORITHM
9. New DistSQL syntax: CREATE DEFAULT SHARDING [TABLE | DATABASE] STRATEGY
10. New DistSQL syntax: SHOW ALL VARIABLE
11. New DistSQL syntax：SHOW VARIABLE variableName;
12. Support 3 modes, including Memory, Standalone and Cluster mode
13. Proxy supports for openGauss
14. Scaling: Add basic support for openGauss
15. Scaling: Add incremental task completion detect algorithm SPI interface
16. Scaling: Add data consistency check algorithm SPI interface
17. Scaling: Basic support of create table on target automatically for MySQL and openGauss
18. Scaling: Support auto switch cluster configuration when job finished
19. Scaling: Add more DistSQL support such as data consistency check, etc

API Change

1. Add schema name configuration for ShardingSphere-JDBC
2. Add default sharding column configuration
3. Change the default authority provider from NATIVE to ALL_PRIVILEGES_PERMITTED
4. SCTL syntax adjustment, merged with DistSQL RAL syntax
5. SHOW RESOURCES DistSQL is adjusted to SHOW SCHEMA RESOURCES
6. Remove shadow logical field, support shadow algorithm

Enhancement

1. Support parsing MySQL union/union all statement
2. Support PostgreSQL ABORT statement
3. Refactor and improve metadata loading process
4. Support PostgreSQL CREATE INDEX statement to generate index automatically when no index is specified
5. Support SQL statement execution with logical schema
6. Support binding tables configuration with different sharding columns
7. Optimize kernel performance
8. Proxy supports queries to part of information_schema tables to optimize client connection experience
9. DistSQL supports using quotation marks to use keywords as parameter names
10. The password in the ADD RESOURCE statement supports special characters
11. ADD RESOURCE supports custom JDBC parameters and connection pool properties
12. DROP RESOURCE supports optional parameter ignore single tables, used to ignore single table rule restrictions
13. Support the use of DistSQL to create sharding table rule based on readwrite-splitting rules
14. SHOW DATABASES statement supports like syntax
15. CREATE SHARDING TABLE RULE supports the use of inline expressions to specify resources
16. CREATE SHARDING TABLE RULE supports configuration using dataNodes
17. CREATE SHARDING TABLE RULE supports reuse of existing algorithms
18. SET VARIABLE, support to modify proxy configuration
19. PostgreSQL's protocol enhancements (Such as supports Portal, unspecified type)
20. Using Netty executor to optimize Proxy performance in specified scenarios
21. Make memory strictly fetch size configurable in Proxy
22. Scaling: Improve support for PostgreSQL
23. Scaling: Support concurrent data consistency check of source and target side

Refactor

1. Refactor the SingleTable feature to support Encrypt multiple data sources
2. Adjust the persistent data structure of the registry center state node
3. Remove the SQL rewrite of DML for Shadow
4. Support the SQL routing of DDL for Shadow
5. Scaling: Refactor default implementation of incremental task completion detect algorithm
6. Scaling: Refactor default implementation of data consistency check algorithm
7. Scaling: Remove HTTP API and binary distribution

Bug Fix

1. Fix sharding interval algorithm
2. Fix SHOW INDEX FROM TABLE FROM TABLE statement rewrite exception
3. Fix Encrypt multi tables join query rewrite exception
4. Fix subquery index out of range exception
5. Fix wrong result of Oracle paging query
6. Fix the rewrite exception when KeyGenerateStrategy is not configured in the Sharding feature
7. Fix federation executor engine exception caused by Oracle dialect case
8. Fix Sharding and Encrypt integration usage rewrite exception
9. Fix Oracle metadata loading exception
10. Fix the issue that SHOW RESOURCES statement cannot display custom attributes
11. Fix the issue that SQL execution exception is not thrown
12. Fix Etcd can not send node added event
13. Fix PostgreSQL rows contains null value may be missing in query result
14. Fix PostgreSQL metadata columns are out-of-order
15. Fix client character set may be incorrect in Proxy

Change Log

1. MILESTONE

5.0.0-beta

New feature

1. New DistSQL to load and show all the ShardingSphere configuration rules
2. Support join SQL from different database instances
3. Support multiple backend database types for new feature of database gateway
4. Support creating and updating the authentication online
5. Add a new automated agent module

API Change

1. QueryReplica configuration item is replaced by read-write-splitting
2. Authentication configuration item of ShardingProxy is replaced by AUTHORITY
3. Optimize the dataSource configuration for ShardingJDBC with dataSourceClassName
4. New API for automated sharding table configuration to provide standard and automated sharding tables
5. Remove configuration item acceptor-size from ShardingProxy
6. Create a built-in sharding algorithm SPI which allows users to directly configure the class name as in 4.x

Enhancement

1. Improve metadata loading process distinctly
2. Greatly enhance the SQL parsing for Oracle, SQLServer and PostgreSQL
3. Support loading privileges from MySQL/PostgreSQL/SQLServer/Oracle
4. Support DDL statement for encryption feature
5. Support rewrite owner table name of projection if using sharding and encrypt together
6. When using SELECT * from encrypted SQL, add quote char for the rewritten column to avoid conflict with keyword
7. Support PostgreSQL JSON/JSONB/pattern matching operator parse
8. Support MySQL/PostgreSQL CREATE/ALTER/DROP TABLESPACE statement
9. Support PostgreSQL PREPARE, EXECUTE, DEALLOCATE statement
10. Support PostgreSQL EXPLAIN statement
11. Support PostgreSQL START/END TRANSACTION statement
12. Support PostgreSQL ALTER/DROP INDEX statement
13. Support CREATE TABLESPACE of PostgreSQL
14. Support MySQL CREATE LOADABLE FUNCTION statement
15. Support MySQL/PostgreSQL ALTER TABLE RENAME statement
16. Support PostgreSQL Close command

Refactor

1. New schema structure in registry center
2. Remove Nacos and Apollo config center support
3. ShardingScaling leverages elasticJob as its job distribution
4. Refactor the metadata content and its update process

Bug Fix

1. Bugfix for cannot use SELECT * wildcard when readwrite-splitting only
2. Fix instance error when the custom sharding algorithm does not match the configuration type.
3. Fix NoSuchTableException when executing drop table if exists statement
4. Fix the wrong table rewrite in UPDATE ... SET ... statement
5. Fix wrong table rewrite in CREATE/ALTER TABLE statement foreign key reference table
6. Fix projection owner check exception when existing subquery temporary table
7. Fix Oracle/SQL92 SELECT ... WHERE ... LIKE statement class cast exception
8. Fix MySQL SELECT EXISTS ... FROM ... statement parse error
9. Fix the wrong result of SHOW INDEX statement
10. Fix SELECT ... GROUP BY ... statement rewrite and merge result error
11. Fix CREATE TABLE statement rewrite error for encrypt
12. Fix exception occurs in PostgreSQL Proxy when reading text format parameter values
13. Enhance the support of array object for PostgreSQL Proxy
14. Fix the bug of Datatype casting for ShardingProxy
15. PostgreSQL Proxy supports using numeric type
16. Fix PostgreSQL Proxy transaction command complete packet's tag incorrect
17. Fix PostgreSQL Proxy may return packet which is not expected by the client

Change Log

1. MILESTONE

5.0.0-alpha

5.0.0-alpha

New Feature

1. The pluggable architecture is available and support function extension through the SPI.
2. Independent SQL parsing engine is available to get SQL parsed AST for the different database dialects.
3. New RDL feature for ShardingSphere Proxy supports creating sharding rules and sharding tables.
4. ShardingSphere-Scaling supports resuming data migration from break-point.
5. ShardingSphere-Scaling supports using ShardingSphere JDBC to migrate data to new cluster.
6. ShardingSphere shadow database is available.

API Changes

1. New sharding/encryption/shadow/queryReplica API.
2. New sharding algorithm and strategy API.
3. New API for ShardingSphere Scaling to create task.
4. Remove DefaultDataSourceName setting item.
5. The separator of the parameter configuration item is changed from - the dot ‘.’ to minus sign ‘-’.
6. Change parameter allow.range.query.with.inline.sharding from global parameter to sharding algorithm parameter.

Refactor

1. Refactor the architecture of SQL parsing module base on the database dialects.
2. Use SPI mechanism to reconstruct online metadata change processing.
3. Rename Orchestration module to Governance module.
4. Rename MasterSlave module to QueryReplica module.
5. Refactor the metadata structure in the governance registration center.
6. Refactor GovernmentShardingSphereDataSource.
7. ShardingSphere UI merges configuration center and registration center display layout.

Enhancement

1. Upgrade the minimum supported version of JDK to Java8.
2. Update Zookeeper curator to version 5.1.0.
3. Update Google Guava to version 29.0-jre.
4. The enhancement for MySQL SQL syntax definition and parsing process.
5. The enhancement for PostgreSQL syntax definition and parsing process.
6. The enhancement for sub-queries in different database dialects.
7. Support MySQL view operations for non-sharding tables
8. Support MySQL stored functions and procedure operations for non-sharding tables.
9. Support SQLServer Top syntax.
10. Optimize the metadata loading to improve the startup speed.
11. Optimize batch insert performance.
12. Supports the use of Oracle RAC JDBC URL.
13. XA transaction manager adds support for Oracle database.
14. ShardingSphere Proxy supports the use of p6sy driver.
15. Add help information to the ShardingSphere Proxy startup script.

Bug fixes

1. Fix alias rewriting error when processing OrderBy condition.
2. Fix SQL rewriting error when MYSQL Insert statement contains expression.
3. Fix parameter calculation error in update on duplicate SQL.
4. Fix generatedKeys gets wrong when batch inserting.
5. Fix the abnormal issue of multi-table verification in DML statement update operation.
6. Fix the NPE problem caused by executing SQL when the table does not exist.
7. Fix the exception when using the Show table command for an unconfigured table.
8. Fix metadata loading error when Oracle database has multiple qualified users.
9. Fix the issue that replica node cannot be enabled online.
10. Fix the problem that ShardingSphere-JDBC does not support PostgreSQL array type.
11. Fix the problem that ShardingSphere-Proxy does not respond when querying long blob data.

shardingsphere-ui-4.1.1

4.1.1

Enhancement

1. Add Sharding-UI dockerfile
2. Enhancement for Sharding-UI docker build

Bug Fixes

1. Fix faild to disable slave database through ui

4.1.1

4.1.1

Enhancement

1. Add Sharding-Scaling & Sharding-UI dockerfile
2. update MySQL & PostgreSQL version for proxy

Bug Fixes

1. Fix parser syntax rule of SUBSTRING and CONVERT
2. Fix parser ColumnSegment ClassCastException
3. Fix TableMetaData load error when use Sharding-JDBC with oracle
4. Fix getSchema NPE when use Sharding-JDBC with oracle
5. Fix Sharding-JDBC parse sql NPE in PostgreSQL
6. Fix Sharding-Proxy receive error response for PostgreSQL JDBC client
7. Fix Sharding-Proxy response number of update is 0 for PostgreSQL JDBC client
8. Fix Sharding-Proxy receive null for PostgreSQL column meta data
9. Fix Sharding-Scaling NPE in MySQL incremental task

Change Logs

1. MILESTONE

4.1.0

4.1.0

New Features

1. Support scaling for ShardingSphere (alpha version)
2. Move etcd registry center from experiment repo to apache repo
3. Upgrade to Java 8

Enhancement

1. Optimization for Sharing Parser with ANTLR Visitor improving the parsing performance of long SQL by 100%~1000%
2. Use multiple threads to load metadata for different data sources
3. Support allow.range.query.with.inline.sharding option
4. The docker of Sharding-Proxy supports loading external lib packages
5. Support integration with Spring using @ShardingTransactionType
6. Enhance ShardingDataSource to compatiable with Seata in micro-sevice distribution transaction

Refactor

1. Remove leaf key generator

Bug Fixes

1. Fix an exception caused by using a CHAR/VARCHAR type column as a order by item
2. Refine DataTypeName syntax rules of all database dialects
3. Fix an exception caused by executing BEGIN using prepared statement of MySQL C API
4. Fix the problem that ALTER TABLE fails to execute when the field type of the table contains Integer/Double/BigDecimal
5. Fix the problem of the stop index dislocation of segment with alias
6. Fix the problem that overwriting SQL SELECT * FROM tbl WHERE encrypt_col = ? AND (sharding_key = ? OR sharding_key = ?) when using sharding + encrypt would throw StringIndexOutOfBoundsException
7. Fix the problem of incorrect decoding after AES encoding when using Sharding-Proxy in Spring Boot
8. Fix a long-time problem of adding schema dynamically in Sharding-UI

Change Logs

1. MILESTONE

shardingsphere-ui-4.1.0

4.1.0

New Features

1. Add a start.bat to launch for shardingsphere-ui distribution module in Windows
2. The work flow and roadmap of sharding-scaling

Enhancements

1. sharding-ui-frontend config file problem
2. The enhancement for the first version sharding-scaling-ui
3. Fix link of build status
4. Fix ui login Chinese text
5. Add sharding scaling API document

Bug Fixes

1. Fix request error page interactive presentation problem
2. Fix ui module for new center api
3. Fix spelling error
4. UI details modification
5. Fixes ui errors when loading datasources config

4.0.1

4.0.1

Bug Fixes

1. Using guava cache to fix parsing deadlock.
2. Oracle insert SQL could not work in encrypt mode.
3. Proxy for PostgreSQL decode parameters error in all types except String.
4. COM_STM_EXECUTE of proxy for MySQL could not support sysbench.
5. None sharding strategy could not config in spring-boot.
6. Plain column could not get from resultSet in sharding-encrypt mode.
7. WasNull field was wrong in GroupByStreamMergeResult.
8. Metadata.getColumns could not work in JDBC.
9. IN operator contains space and \n \t \r could not supported by parser.

Enhancement

1. Optimize antlr performance using two-stage parsing strategy.
2. Add class filter constructor to restrict the illegal class from YAML.

Change Logs

1. MILESTONE

4.0.0

4.0.0

API Changes

1. Change package and maven groupId form io.shardingsphere to org.apache.shardingsphere.
2. Adjust Sharding-JDBC configuration API.
3. Adjust persist structure for registry center.

New Features

1. SQL92 Syntax available.
2. Sharding-Proxy for PostgreSQL protocol available.
3. SQL 100% compatible if route to single data node.
4. Less-than(<), greater-than(>) and Less-than-equal(<=), greater-than-equal(>=) for sharding key operator available.
5. DISTINCT SQL syntax available.
6. Broadcast table available.
7. LEAF key generator available.
8. XA Transaction available, Atomikos, Narayana and Bitronix integrated.
9. BASE Transaction available, Seata integrated.
10. Data encrypt available.
11. Skywalking plugin available.
12. Sharding-UI available, an orchestration management platform.

Enhancement

1. MariaDB supported.
2. Improve the compatibility of SQL parsing.
3. SELECT FOR UPDATE route to master data source only.
4. Hint in Sharding-Proxy available.
5. Make configuration of orchestration consistent between Sharding-JDBC and Sharding-Proxy.
6. Renew modified data sources only, not renew all the data sources.
7. Vibrate configurable for Snowflake key generator.

Bug Fixes

1. Improve the compatibility of JDBC Driver URL.
2. Delete statement with alias available.
3. Check and disable updating sharding column.
4. Fix wrong type of TINYINT and SMALLINT as INTEGER.

Change Logs

1. MILESTONE #3
2. MILESTONE #4
3. MILESTONE #5
4. MILESTONE #6
5. MILESTONE #7
6. MILESTONE #8
7. MILESTONE #9