Plugins 2.0 2nd round

[fortmarek]

Tuist Plugins 2.0

There have been some great suggestion in the latest RFC here. I'd like to run another round of reviews (in a similar fashion as Apple runs Swift evolution proposals) as adding new comments to the current RFC would become way too messy.

The response to the last round have been mostly positive for taks specifically, there were also some unclear areas such as tuist edit.

Tasks

Definition

Reusing the capabilities of Package.swift for tasks has a lot of benefits - mainly easy reusing of third party packages. Creating our own solution would incur a big maintenance burden for us and for now, the tradeoff does not seem to be worth it.

Invocation

How will we invoke tuist tasks? As mentioned in the last review, we will invoke any command that has tuist- prefix. To add to that, we will first try to see if a command name is defined as a command in tuist core and only then try to find alternatives (so that users would not be able to replace our own commands like tuist generate). I can see some value in being able to override tuist core commands but it could lead to confusion (commands doing different things based on where they would be run from).

Secondly, tasks imported as plugins should be possible to run as well but they should not be installed globally. That's because users can have multiple versions of the same plugin. We should therefore install them to a location such as ~/.tuist/Cache/Tasks/plugin-name/version/. For the first iteration, I'd ignore plugins that don't have tasks pre-built. We should, however, add support for that at some point.

Location in a plugin

We still want developers of tasks share them as a plugin - and keeping the rest of the structure as-is. I propose two alternatives for how new tasks should be structured in the plugins repo:

• Option 1: Project root
  For this option, Package.swift would be at the root along with Plugin.swift. If plugin developers would keep the original Package.swift location for sources, they would be in root/Sources. The benefit of this approach is that Package.swift at root can be easily extendable to be used for things other than Tasks. What I don't really like is among ProjectDescriptionHelpers, Templates, etc. we would have not really descriptive Sources. We could nudge the people to rename it to something else but there would be no strict limits in that. (we can "re-enforce" that nudge with tuist plugins create)

• Option 2: Package.swift in Tasks
Package.swift and Sources directory would be completely hidden in Tasks repository. This would make the structure of the code in the root a bit easier to read but it is less flexible in the future.

Of these options, I am leaning to having a Package.swift in the root and nudging people to replace the default location of Sources to Tasks.

Local tasks

One thing not addressed in the original proposal were local tasks - either in the form of a local plugin or a simple local task located in Tuist/Tasks. While we could support both, I'm not sure if there is enough value in supporting Tuist/Tasks. What we could maybe rather do is to let users define the tasks via plugins only. This will make the implementation users while users will not lose any functionality.

tuist plugins archive

I'd like to detail a little bit of what this command might do and how that fits in the bigger picture of Plugins 2.0.

The primary purpose of this command will be to build all the products in Package.swift and create a zip of the executables. The final zip would be called tasks.zip and creators of plugins would attach that to a release of a plugin as an artifact. When fetching a plugin, we'd download this zip and extract its contents to ~/.tuist/Cache/Tasks/plugin-name/version/.

tuist plugins create

We could either wire this command to swift package init. We can add our own template down the line if we want to have tigher control over this.

tuist plugins {test, build, run} name-of-task

Apart from test and build, I'd also add tuist plugins run. All of them would just run swift xxx, though, so there's not much to expand on this.

Project Description Helpers

For now, we will keep project description helpers as-is and we will revisit this topic once we progress with new plugins architecture for tasks. This is because having third-party code in helpers does not bring a lot of value and therefore the currently planned changes might not be as beneficial for them as for tasks.

[pepicrft]

I can see some value in being able to override tuist core commands but it could lead to confusion (commands doing different things based on where they would be run from).

Agree. If someone tries to override Tuist commands, our commands will take preference. We can make that clear in the documentation that there are some command names that are reserved and that the list of names might grow in the future as we add more features to the project.

Secondly, tasks imported as plugins should be possible to run as well but they should not be installed globally.

I guess you mean here global but managed by Tuist in a tuist-scoped directory?

Of these options, I am leaning to having a Package.swift in the root and nudging people to replace the default location of Sources to Tasks.

Same here. I think it aligns with how Swift Package Manager expects packages to be structured: a repository with a Package.swift* at the root.
One suggestion that I'd make is that we should make things conventional. For example:

• Helpers: Those are represented by static library products whose name is suffixed by DescriptionHelpers.
• Tasks: Those are represented by executable products whose name is prefixed by tuist-.

I'd check if we can maintain this alongside the Plugin.swift manifest by leveraging symbolic links. For example, Sources/XXDescriptionHelpers is a symbolic link to the directory where the helpers are located.

One thing also I'd like to call out, is that maintaining both, Plugin.swift and Package.swift can be cumbersome for both users and us. Therefore, if we come up with a new plugins architecture that significantly improves the experience, we can make this a breaking change and release a new major version. I know this might be too close to 2.0, but considering this is an important milestone for the project, I think it justifies doing something like that.

Local tasks

I lean towards having Tuist/Plugins, and each folder there represents a plugin with a Package.swift in it.

tuist plugins archive

The way you described it is the way I envisioned it. I wouldn't name it tasks.zip though. Because it's a plugin, whose first iteration will only support tasks, we can name it name-of-the-plugin.tuist-plugin.zip. That contains:

• Helpers libraries that statically link ProjectDescription.
• Static executables that have been compiled for arm64 and x86_64

I'd check what happens in the first case to make sure it doesn't cause duplicated symbols. Otherwise the helpers will have to dynamically link ProjectDescription.

tuist plugins create

I'd instead have a template where we codify the conventions described above.

tuist plugins {test, build, run} name-of-task

Agree. Because we have a Swift package we can proxy the calls to the Swift Package Manager CLI

[luispadron]

Thanks for the write up @fortmarek
I like the over-all design and also echo some of @pepibumur suggestions, particularly around making plugins basically just SPM packages as much as possible

We may be able to remove the Plugin.swift in favor of just the name of the swift package? Think this would simplify how users create plugins.

Also echo the comments made about using the generic "plugin" name vs. tasks.

In terms of the tuist plugin archive, I wonder if this is something we need? As in, should tuist just do this instead during fetching a plugin?

Requiring a CI job which runs this archive in order to release plugins is certainly a bit complex and loses on some of the simplicity the current implementation offers. I'd really love for us to consider just fetching the package, running through the archive internally, and caching that result.

[fortmarek]

We may be able to remove the Plugin.swift in favor of just the name of the swift package? Think this would simplify how users create plugins.

Yes, that's a good idea, I'll try to explore this.

In terms of the tuist plugin archive, I wonder if this is something we need? As in, should tuist just do this instead during fetching a plugin?

We value developer's productivity and especially since plugins will be able to include third-party dependencies, the compilation can become time-intensive.

Requiring a CI job which runs this archive in order to release plugins is certainly a bit complex and loses on some of the simplicity the current implementation offers.

For the reference, most of the executable command line tools for Mac are managed via brew which is much harder to actually manage. We will provide something that is distributed but still fast. It's just a simple step and then an upload to Github releases, I'm not sure if this is such a big burden to developers of plugins? We probably will have to do archive internally at some point but I'd like the first iteration to rather rely on releases.

I understand the sentiment about the simplicity of the feature but I don't think trying to make it as simple as possible is the right trade-off here.

[luispadron]

Gotcha, think it makes sense to add this with the expectation of plugin developers archiving manually for now until we stabilize plugins 2.0 and we can revisit this decision! I agree compilation sucks, but if were able to make it easier for everyone to integrate it may be a good trade off, but for now I agree we can just expect an archived plugin.

[pepicrft]

Share the same sentiment as Marek with regards to archiving and distributing binaries. We can make the process more convenient by providing CI-primitives like a GitHub action that wraps the CLI and takes care of archiving and creating the release with the artifacts for you. It'll be a drag & drop type of experience.