chore: migrate from kuttl to chainsaw

[eddycharly]

Chainsaw was created because kuttl stopped to be actively maintained (it's actually a single maintainer now).

We are also solving most of the kuttl limitations that exist today and wrote a command to automatically migrate from kuttl to chainsaw. It supercharges e2e tests with better logs, config maps assertions, assertions trees and many more things 💪

This is a relatively young project but it has already been adopted by a couple of e2e tests heavy projects (kyverno, keptn, fairwinds rbac-manager, grafana-operator, redis-operator and others).

To give more context, you can see similar PRs at:

• chore: replace kuttl tests with chainsaw keptn/lifecycle-toolkit#3000
• chore: migrate from kuttl to chainsaw open-telemetry/opentelemetry-operator#2630
• feat: add chainsaw scorecard test support operator-framework/operator-sdk#6676
• chore: migrate from kuttl to chainsaw grafana/tempo-operator#786 in this PR the tempo-operator maintainers asked some good questions and I explained the current status, that I suggest taking a look at.

Sadly, kuttl haven't done a release since jan 2023.

Here you can find a good video showcasing chainsaw: https://www.youtube.com/watch?v=Ejof-wtAdQM&ab_channel=CamilaMacedo

If you need more infos 👇

• GitHub: https://github.com/kyverno/chainsaw
• Docs: https://kyverno.github.io/chainsaw
• Slack: https://kubernetes.slack.com/archives/C067LUFL43U

[eddycharly]

This kuttl-in-kuttl thing... first time i see that 🙈

[eljohnson92]

Please open an issue describing what this PR solves

[eddycharly]

@eljohnson92 i will do, let me at least update the pr description :)

[AshleyDumaine]

Looks like this closes #103 (removing kuttl-in-kuttl)

[eddycharly]

@AshleyDumaine yes it could fix that (and more)

[eddycharly]

The tricky one is byovpc i'll check deeper, maybe i'll need to add something in chainsaw for that.

[eljohnson92]

@eddycharly we have done a pretty major refactor of our kuttl testing including removing the kuttl-in-kuttl and complications in the byoc. I suspect a migration path to chainsaw would now be much simpler. is there any tooling around transitioning from kuttl to chainsaw?

[eddycharly]

You can use chainsaw migrate kuttl tests ... to convert them automatically. It's not 100% reliable though.

I think the only test that is not easy to migrate to chainsaw yet is the byovpc one, i will investigate next week, it sounds like a good use case.

Another important point is that you probably don't want to migrate as is, there's a lot of features in chainsaw that will make your tests better. Automatic conversion is not going to do that, it's just going to produce a chainsaw equivalent of your kuttl tests.

[eljohnson92]

It seems like one of the main usecases we still have to farm out to a Makefile right now is when we curl the LInode API to validate backend resources are created, is there any built in chainsaw functionality to cover this usecase?

[eddycharly]

@eljohnson92 nope, you can still do it with curl though.

I'll look at your tests again today, upcoming v0.1.8 could solve the byovpc issues.

[eddycharly]

@eljohnson92 i started over in #162

Can you let the workflow run ?

[eddycharly]

Closing, obsolete