[README]: Contributing new modules

[HofmeisterAn]

Problem

Hi 👋, in the past few weeks, I've encountered several issues with the GitHub-hosted runners. Due to the increasing number of modules, pulling all the images occupies too much disk space. I've implemented a temporary fix to free up some disk space, but this won't be enough as we add more new modules.

Until we find a suitable fix and implement a mechanism to handle the growing number of modules and image sizes, I prefer not to merge pull requests that involve very simple module configurations. Which can be accomplished with just a few lines using the generic container builder API. If you believe the module is valuable for the community, please consider creating an issue first and discussing it there to avoid unnecessary work on creating a pull request. OC, I will review and try to merge the remaining and outstanding module PRs.

Solution

-

Benefit

-

Alternatives

-

Would you like to help contributing this enhancement?

Yes

[mrudat]

Would it be worthwhile to refactor modules so that each image or class of images (e.g. Web/Database/... Servers that might want to share a common interface) has its own repository?

That would mean that exactly one docker image should be created for each image-specific repository, presuming that the same configuration used from multiple implementations in different languages results in an identical container being tested.

We could also describe the image-specific configuration in a common configuration format and use that to generate the language-specific code.

For example, username/password/... is often configured via environment variables, and the specific environment variable names are specific to individual images. However, the code for capturing the username/password is shared between multiple images across multiple languages, which suggests we have too much nearly identical custom-maintained code and are waiting for a refactoring pass to extract the common functionality.

[eddumelendez]

Hi, I think this can help to free some more space for linux workers https://github.com/jlumbroso/free-disk-space

[HofmeisterAn]

Hi Eddú, we are doing this already:

testcontainers-dotnet/.github/workflows/cicd.yml

Lines 48 to 64 in 734fb52

	# Our modules occupy too much disk space. The GitHub-hosted runners ran into the
	# error: "no space left on device." The pulled images are not cleaned up between
	# the test runs. One obvious approach is splitting the tests and running them on
	# multiple runners. However, we need to keep in mind that running too many
	# simultaneous builds has an impact on others as well. We observed that scheduled
	# Dependabot builds blocked others in the Testcontainers organization.
	- name: Free Disk Space
	uses: jlumbroso/free-disk-space@v1.3.1
	if: runner.os == 'Linux'
	with:
	tool-cache: true
	android: true
	dotnet: true
	haskell: true
	large-packages: true
	docker-images: true
	swap-storage: false

[kiview]

Would it be worthwhile to refactor modules so that each image or class of images (e.g. Web/Database/... Servers that might want to share a common interface) has its own repository?

From past experiences with tc-java, this leads to a lot of operational overhead (e.g. releases), unless we have good automation in place.

Splitting in parallelizing the CI workflow on a per-module or module-sets level can help and it is what we essentially do in tc-java as well.