RFC: bundle scoping

[charleskorn]

This is a request for comment and feedback on the idea in proposal.md.

The RFC process is intended to allow anyone with an interest in Batect to get involved and help shape it to suit their needs, so please don't be shy! Any comments, suggestions, questions or constructive criticism are more than welcome.

Please comment directly on the file itself or add general comments in this PR.

[theramis]

Suggestion: I love the idea of private containers. Thoughts on having private tasks too?

[charleskorn]

I considered it, but couldn't think of any concrete use cases for private tasks. Is there a use case you have in mind that would benefit from private tasks?

[theramis]

The use case I was thinking of was where you might have prerequisites tasks for your own task.

The exact use case is:

1. Publish DB Task
   - Start sql container + run command to deploy latest schema (binary) to the sql container
2. Run DB Task
   - PreReq = Publish DB Task
   - Start sql container + run command "Press any key to continue command"

In this example I don't really want to expose the first task. Since the functionality this bundle provides is to take a binary and run it on a DB container.

(not sure if this can be done without multiple tasks)

[charleskorn]

Right, makes sense. I think that would definitely fall into a "stage 3" in terms of implementation priority.

[theramis]

Sounds good to me.

To be completely honest it's not something that I "need". Just a nice to have so feel free to deprioritize it 🙂

[binkley]

This looks like cleaner syntax for bundles and additional features / use cases come along for the ride.

[codecov]

Codecov Report

Merging #991 (5696fcb) into main (f936697) will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##               main     #991   +/-   ##
=========================================
  Coverage     83.86%   83.86%           
  Complexity     2686     2686           
=========================================
  Files           353      353           
  Lines         11773    11773           
  Branches       1412     1412           
=========================================
  Hits           9874     9874           
  Misses         1598     1598           
  Partials        301      301           

Flag	Coverage Δ
linux	82.17% <ø> (ø)
mac	82.24% <ø> (ø)
windows	82.88% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f936697...5696fcb. Read the comment docs.

[stale]

This issue has been automatically marked as stale because it has not had any activity in the last 60 days. It will automatically be closed if no further activity occurs in the next seven days to enable maintainers to focus on the most important issues.
If this issue is still affecting you, please comment below within the next seven days.
Thank you for your contributions.

[charleskorn]

📢 Update on implementing this RFC

I realise there's been radio silence on this RFC since I posted it and so I wanted to give an update on what's going on.

The short version is that while I'm very, very keen to implement this, I'm currently prioritising some other work to improve the experience of working on Batect itself.

Batect has grown quite a bit since it first began a few years ago, and unfortunately that is starting to show - builds take longer to run than I'd like (both locally and on CI), flaky tests are going unfixed and I'm finding more and more of my precious time diverted away to fixing things that are invisible to most users, like edge cases in the Docker client. All of these things make it less pleasant to work on Batect, and make it much harder for others to contribute as well.

I have two major things on my backlog to address these issues:

• The biggest contributor to the slow builds and flaky tests are the journey tests, where Batect is started from the command line and used to verify a variety of scenarios. This group of tests has grown to the point where they take up to ten minutes to run as part of the CI build.

  I'm going to spend some time trimming back the test scenarios here, either removing them altogether or covering the functionality with other kinds of tests, and also looking into other ways to improve their performance.

• The second biggest contributor to this is the Docker API client. It comprises nearly 40% of the production code in the codebase and has a similarly large proportion of the unit tests. Not only does the API client carry a significant maintenance burden, it consumes a significant amount of time in CI, it is often the source of bugs and instability, and having to implement new Docker client features myself means that there is always a lag between new features being added to Docker and being able to provide them in Batect.

  I'm currently working on a new Docker client which is currently very much in the experimental stage but is already showing promise:

  • Separating the client out of the main project means it does not need to be rebuilt and retested on every commit to the main project, saving a significant amount of time
  • Embedding the official Docker Golang client as a library inside the client (rather than implementing these features myself) means adding support for new features is far simpler and faster, and also means that the behaviour of Batect should more closely match that of the docker CLI, which in turn makes supporting alternatives to Docker for Desktop easier as well
  • This new approach also removes a significant impediment to migrating to Kotlin/Native, which would remove the need for users to install a JVM to use Batect

Once I've finished addressing these issues, I will come back to this feature as a priority. I'm most likely to tackle the new expression syntax (#990) first, then bundle scoping.

[charleskorn]

📢 Update

Apologies for the continuing radio silence on this RFC.

Replacing the Docker client in Batect has taken much longer than I planned / hoped / expected, however, I can see the light at the end of the tunnel.

As soon as this is complete, I'll start work on #990, then pick up the features described in this RFC.

[binkley]

I wasn't vocal enough previously. I'm a 👍 on this.