How do we make LambdaCD more approachable? The structure of the ecosystem

[flosell]

Sparked by #190:

And I also think that it should be at the core of the project because as a new user I want to take a solution and immediately start using it.
And it very important to have it in the core of the project. Because people don't really want to spend time searching for third-party libraries, which are often incompatible etc. Like it was with me when I took the last version of lambdacd, then added lambdacd-ui, which was out of date and I was in one step of leaving it and switching to Jenkins :)

My initial plan was to build LambdaCD in a way that it is easy to extend or swap things out and build most functionality that's not necessary in the core as separate extensions. That was motivated both by general design concerns (Unix Philosophy of one piece doing exactly one thing well, Libraries, not frameworks, ...) as well as the hope of sparking a community with different pieces being maintained by different people. This had mixed results:

Clearly (as you have observed), separating stuff into many different libraries makes LambdaCD much harder to handle for users (need to find the right libraries, make sure the versions match up, ...) and LambdaCD-Developers (interfaces between libraries, keeping stuff in sync, where does something go,...).
Also, the idea of a larger community owning parts of the ecosystem never really materialised. There are a couple of libraries out there built by others but often not very actively maintained. The rest is owned by me with the occasional PR.

On the other hand, I have seen some successes of the general design and it might be LambdaCDs biggest strength: Some people don't use LambdaCD as a monolithic tool they just deploy onto a machine. They use it as building blocks to build their own internal, custom delivery tool. To be fair, this audience is probably not that big.

Long story short, I wouldn't want to fundamentally change the design ideas but there's definitely room for improvement in making it more approachable for newcomers.

Here are some ideas:

• Re-centralisation: Keep the general design and APIs in place (for extensibility and for people who want to just use building blocks to build their own thing) but merge most of the code into a single project and add all major improvements (e.g. multi pipeline support) there
• LambdaCD distribution/generator: Keep the structure as is and add improvements that don't affect the core (e.g. multi-pipeline support) in separate projects. Invest in lein new lambdacd to become a true, batteries included "LambdaCD distribution". Also invest in the infrastructure around it (e.g. proper pipelines to make sure the different pieces fit together, maybe a common github organisation).

[isagalaev]

As one evaluating the project right now I would say simply pointing out recommended relevant libraries in the docs would go a long way towards making it approachable. For example right now I'm stuck with things like:

• how do I do docker stuff in my build steps? Is there a good Clojure library or should I shell out?
• is there a library to deploy containers to AWS ECS, K8s, etc.?

The Getting Started guide already assumes people would come to LambdaCD from outside the Clojure ecosystem, so we should be forgiving not knowing how things are done here :-)

[zerg000000]

My little thinking about this topic is what make lambdacd better than others?

Solving outstanding CI/CD big issue would be a good way to make lambdacd become unique.

Something like testable ci/cd pipeline would be awesome!

As a Devops Engineer, I want a CI/CD pipeline could be tested easier and faster so that I could spend less time on waiting and make my pipeline more stable.