Add python packages to dockerfile

[Tim-Tscheppe]

Fixing issue
#14847

[github-actions]

Thank you for your contribution to the Solidity compiler! A team member will follow up shortly.

If you haven't read our contributing guidelines and our review checklist before, please do it now, this makes the reviewing process and accepting your contribution smoother.

If you have any questions or need our help, feel free to post them in the PR or talk to us directly on the #solidity-dev channel on Matrix.

[nikola-matic]

@r0qs can you please check the rest.

[r0qs]

Hi @Tim-Tscheppe thanks for your contribution. However, there are still some additional tasks that require attention. You can find further details in the comments section of the issue you referenced: #14847 (comment)

Basically, we also need to identify what packages are commonly used in our CI and can be moved to our Linux docker images (I believe they are requests, parsec and tabulate or maybe even all the packages installed by chk_pylint job as suggested by @cameel, but we need to double check that). Conversely, certain packages, particularly those utilized by the Windows/MacOS images, must still be installed during runtime in our CI, since those are Convenience images provided by CircleCI.

For instance, see: https://github.com/ethereum/solidity/blob/develop/.circleci/config.yml#L936 and https://github.com/ethereum/solidity/blob/develop/.circleci/config.yml#L999.

[Tim-Tscheppe]

Hi @Tim-Tscheppe thanks for your contribution. However, there are still some additional tasks that require attention. You can find further details in the comments section of the issue you referenced: #14847 (comment)

Basically, we also need to identify what packages are commonly used in our CI and can be moved to our Linux docker images (I believe they are requests, parsec and tabulate or maybe even all the packages installed by chk_pylint job as suggested by @cameel, but we need to double check that). Conversely, certain packages, particularly those utilized by the Windows/MacOS images, must still be installed during runtime in our CI, since those are Convenience images provided by CircleCI.

For instance, see: https://github.com/ethereum/solidity/blob/develop/.circleci/config.yml#L936 and https://github.com/ethereum/solidity/blob/develop/.circleci/config.yml#L999.

Hi @r0qs ,
I took a look at all of the packages in the chk_pylint job and noted the following:

• Deefdiff - used on windows in lsp.py, keeping in ci
• Colorama - used in windows in same location as Deepdiff
• Requests / Parsec / Tabulate - can be moved to Linux docker images (already done in this MR)
• The rest I do not see being used anywhere in the project, so I will not touch them for now.

Note that I kept them in for the windows pyscripts config.yml file. I only removed from the linux based versions that I added to the images.

Please let me know if there is any other work needed.

Also, is there some guides to ramp on this project? I am interested in getting more involved with open source and specifically Ethereum

Edit: It looks like removing the packages from the config.yml file is causing the pipeline to fail

[r0qs]

parsec and tabulate should be removed from t_ubu_pyscripts since now they are installed in the images.

Suggested change

	name: Install gas_diff_stats.py dependencies
	command: python3 -m pip install --user parsec tabulate

Also, there still some things that we could move to the ubuntu image. For instance, the test_lsp command installs deepdiff and colorama as well. So we could remove that step (i.e. Install dependencies), since test_lsp is only used on the Ubuntu machines.

And indeed @cameel was right, and the same applies for chk_pylint, so we can remove the install_python3 step from chk_pylint and install all those packages in the Ubuntu images.

[r0qs]

Hi @r0qs , I took a look at all of the packages in the chk_pylint job and noted the following:

* Deefdiff - used on windows in lsp.py, keeping in ci

* Colorama - used in windows in same location as Deepdiff

* Requests / Parsec / Tabulate - can be moved to Linux docker images (already done in this MR)

* The rest I do not see being used anywhere in the project, so I will not touch them for now.

Note that I kept them in for the windows pyscripts config.yml file. I only removed from the linux based versions that I added to the images.

Please let me know if there is any other work needed.

Hi @Tim-Tscheppe, thank you for your contribution. I've taken a look at our CircleCI config and left a comment regarding what still needs to be done.

Also, is there some guides to ramp on this project? I am interested in getting more involved with open source and specifically Ethereum

You can check our contribution guide at https://docs.soliditylang.org/en/latest/contributing.html

Edit: It looks like removing the packages from the config.yml file is causing the pipeline to fail

yes, the pipeline will fail because we will need to rebuild the images and it cannot be done from a fork. I will copy your branch and see if we can still reuse this PR or if we will need to create a new one ourselves. Usually, when we update the docker images, we have one PR changing only the images and another changing the CI configs. For instance see: #14964 and #14962.

[r0qs]

@Tim-Tscheppe I just created other two PRs with the changes, since it was much faster to do that in that way. For future reference, kindly refrain from merging new features directly into the develop branch of your fork; creating a separate branch simplifies the PR workflow when we need to fix something. So I'm closing this in favor of #15060.

Thanks for your contribution anyway :)

See this section of our contribution guide: https://docs.soliditylang.org/en/latest/contributing.html#workflow-for-pull-requests for further details.