Amoro Optimizer Support Flink 1.18

[czy006]

Amoro Optimizer Now Just Supports Flink 1.14 to Flink 1.16, Maybe We can upgrade it to support Flink 1.15/1.16/1.17/1.18, and remove Flink 1.14 support.

[baiyangtx]

I agree with that. From a technical perspective, the flink-optimizer should be able to achieve engine versions independence.

[zhoujinsong]

I agree with the idea, but currently, using a fixed set of profiles to expose the Flink optimizer's dependency on Flink versions seems to limit the use of more Flink versions. Perhaps we should directly expose the ability to set the Flink version used through parameters, which would be more scalable. For example, like this：

mvn clean package -pl 'ams/optimizer/flink-optimizer' -Doptimizer-flink.version=1.18.0 -Doptimizer-flink.scala-version=2.13

[czy006]

Does this mean that we need to implement flink-optimizer version-independent optimization capabilities? For example, a jar supports all versions

[zhoujinsong]

Actually, the Flink optimizer has the same implementation across all Flink versions. As it only uses some basic Flink API, this enables it to automatically apply to most Flink versions, although we may need to specify the specific range of supported versions.