Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Legacy Flink Support #19

Open
ranchodeluxe opened this issue Feb 1, 2024 · 1 comment
Open

Legacy Flink Support #19

ranchodeluxe opened this issue Feb 1, 2024 · 1 comment
Assignees
Labels
enhancement New feature or request

Comments

@ranchodeluxe
Copy link
Collaborator

Problem

We are currently on Flink Operator version 1.6.1 which spins up Flink version 1.17.1 in the cluster. This is important for two reasons:

  • in the past we've seen older Flink Operator versions (and therefore different Flink versions) not work with our recipes -- things just stall out at runtime or half finish
  • for some reason the Apache Beam team which makes the portable Flink runner jars seems to be behind (and currently doesn't have a runner for even 1.17)

The Flink Operator team moves fast and only keeps the two most recent versions around (more deets in this PR).

Goal

We want to be able to run Flink Operator 1.6.1 for some time b/c it works even if it and the dependency images and jars disappear

Solution

AC:

  • [ ]: currently we see older versions of the Flink Operator on artifacthub.io but not 1.6.1 or 1.7.
  • [ ]: get in touch with the developers and ask that they publish the most recent operators to artifacthub.io
  • [ ]: crosswalk image deps in the 1.6.1 operator and make sure the container registry has legacy versions sticking around
  • [ ]: crosswalk jar deps in the 1.6.1 operator and make sure legacy versions sticking around
  • [ ]: if either of these last two points isn't true then we need to figure out where to store these for our use
@ranchodeluxe ranchodeluxe added the enhancement New feature or request label Feb 1, 2024
@ranchodeluxe
Copy link
Collaborator Author

@moradology is saying we might be able to use the Flink Operator GH and tags to go back in time and reconstruct things we need

@ranchodeluxe ranchodeluxe self-assigned this Feb 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant