Skip to content
This repository has been archived by the owner on Oct 13, 2023. It is now read-only.

Move parallel_exec() to exectools; keep only one version #791

Merged
merged 1 commit into from
Jul 6, 2023

Conversation

locriandev
Copy link
Contributor

@locriandev locriandev commented Jul 4, 2023

We have 2 similar functions that are used to execute a given function in a thread pool, with a predefined set of params. These functions are both defined in runtime.Runtime. This PR aims to clean up the code by keeping just one version of parallel_exec; moreover, the natural place for such a function is probably within exectools module.

Tested in a hack space with ocp4-scan, that calls doozer config:scan-sources and parallel_exec: https://saml.buildvm.hosts.prod.psi.bos.redhat.com:8888/job/hack/job/dpaolell/job/ocp4-scan/94/console

@locriandev locriandev requested a review from vfreex July 4, 2023 13:59
@openshift-bot
Copy link

Build #1

GLOB sdist-make: /mnt/workspace/jenkins/working/art-tools_doozer_PR-791/setup.py
py38 create: /mnt/workspace/jenkins/working/art-tools_doozer_PR-791/.tox/py38
py38 installdeps: -rrequirements-dev.txt, -rrequirements.txt
py38 inst: /mnt/workspace/jenkins/working/art-tools_doozer_PR-791/.tox/.tmp/package/1/rh-doozer-2.1.1.dev26+g1c0f838.zip
py38 installed: aiofiles==23.1.0,aiohttp==3.8.4,aiosignal==1.3.1,astroid==2.15.5,async-timeout==4.0.2,attrs==23.1.0,autopep8==2.0.2,bashlex==0.18,bcrypt==4.0.1,cachetools==5.3.1,certifi==2023.5.7,cffi==1.15.1,chardet==5.1.0,charset-normalizer==3.1.0,click==8.1.3,colorama==0.4.6,coverage==7.2.7,cryptography==41.0.1,decorator==5.1.1,defusedxml==0.7.1,Deprecated==1.2.14,dill==0.3.6,distlib==0.3.6,dockerfile-parse==2.0.0,exceptiongroup==1.1.2,fastcore==1.5.29,filelock==3.12.2,flake8==6.0.0,flexmock==0.11.3,frozenlist==1.3.3,future==0.18.3,ghapi==1.0.4,gssapi==1.8.2,idna==3.4,iniconfig==2.0.0,isort==5.12.0,jira==3.4.1,koji==1.33.0,krb5==0.5.0,lazy-object-proxy==1.9.0,mccabe==0.7.0,multidict==6.0.4,mysql-connector-python==8.0.33,oauthlib==3.2.2,openshift-client==1.0.19,packaging==23.1,paramiko==3.2.0,platformdirs==3.8.0,pluggy==1.2.0,protobuf==3.20.3,pycodestyle==2.10.0,pycparser==2.21,pydantic==1.10.10,pyflakes==3.0.1,pygit2==1.10.1,PyGithub==1.59.0,PyJWT==2.7.0,pylint==2.17.4,PyNaCl==1.5.0,pyproject-api==1.5.2,pyspnego==0.9.1,pytest==7.4.0,python-dateutil==2.8.2,PyYAML==6.0,requests==2.31.0,requests-gssapi==1.2.3,requests-kerberos==0.14.0,requests-oauthlib==1.3.1,requests-toolbelt==1.0.0,rh-doozer @ file:///mnt/workspace/jenkins/working/art-tools_doozer_PR-791/.tox/.tmp/package/1/rh-doozer-2.1.1.dev26%2Bg1c0f838.zip,semver==3.0.1,setuptools-scm==7.1.0,six==1.16.0,tenacity==8.2.2,tomli==2.0.1,tomlkit==0.11.8,tox==4.6.3,typing==3.7.4.3,typing-extensions==4.7.1,urllib3==2.0.3,virtualenv==20.23.1,wrapt==1.15.0,yarl==1.9.2
py38 run-test-pre: PYTHONHASHSEED='2268615220'
py38 run-test: commands[0] | coverage run --branch --source doozerlib -m unittest discover -t . -s tests/
................................................................................................................................................s.s....................................s.s...s.s...s..s.s.s.................................................................................
----------------------------------------------------------------------
Ran 274 tests in 2.256s

OK (skipped=10)
py38 run-test: commands[1] | flake8
py38 run-test: commands[2] | coverage report
Name                                          Stmts   Miss Branch BrPart  Cover
-------------------------------------------------------------------------------
doozerlib/__init__.py                            12      7      2      1    43%
doozerlib/_version.py                             2      2      0      0     0%
doozerlib/assembly.py                           158     20     87     11    84%
doozerlib/assembly_inspector.py                 220    158    128      8    24%
doozerlib/assertion.py                           13      0      6      0   100%
doozerlib/brew.py                               366    206    152      4    40%
doozerlib/build_status_detector.py               85     10     54      3    86%
doozerlib/cli/__init__.py                       122     64     28      0    39%
doozerlib/cli/__main__.py                      1202   1202    474      0     0%
doozerlib/cli/cli_opts.py                        20      3      8      0    89%
doozerlib/cli/config_plashet.py                 536    536    246      0     0%
doozerlib/cli/config_tag_rpms.py                141     20     77     15    82%
doozerlib/cli/detect_embargo.py                 167     35     70      8    75%
doozerlib/cli/get_nightlies.py                  230     59    127      3    71%
doozerlib/cli/images_health.py                   82     30     26      2    59%
doozerlib/cli/images_streams.py                 687    687    296      0     0%
doozerlib/cli/inspect_stream.py                  66     66     28      0     0%
doozerlib/cli/release_calc_upgrade_tests.py      24     24      6      0     0%
doozerlib/cli/release_gen_assembly.py           277    146    118      3    42%
doozerlib/cli/release_gen_payload.py            703    260    300     20    58%
doozerlib/cli/rpms_build.py                     165     59     58      8    57%
doozerlib/cli/scan_sources.py                   182    141    100      2    17%
doozerlib/comment_on_pr.py                       46      0      8      0   100%
doozerlib/config.py                              97     97     44      0     0%
doozerlib/constants.py                           12      0      0      0   100%
doozerlib/coverity.py                           255    225     82      0     9%
doozerlib/dblib.py                              263    160     68      4    35%
doozerlib/distgit.py                           1456    903    692     40    36%
doozerlib/dotconfig.py                           54     43     31      0    13%
doozerlib/exceptions.py                          14      5      0      0    64%
doozerlib/exectools.py                          226    110     80     10    48%
doozerlib/gitdata.py                            171    137     76      0    14%
doozerlib/image.py                              532    329    232     10    33%
doozerlib/logutil.py                              9      0      2      1    91%
doozerlib/metadata.py                           432    151    184     30    61%
doozerlib/model.py                              113     21     36      2    82%
doozerlib/olm/__init__.py                         0      0      0      0   100%
doozerlib/olm/bundle.py                         315    230     72      0    22%
doozerlib/osbs2_builder.py                      119     30     44     19    67%
doozerlib/plashet.py                            134      9     90     15    89%
doozerlib/pushd.py                               22      0      2      0   100%
doozerlib/release_schedule.py                    28     17      8      0    31%
doozerlib/repos.py                              255     95    123     18    56%
doozerlib/rhcos.py                              248     26     96     15    88%
doozerlib/rpm_builder.py                        232     32    123     32    80%
doozerlib/rpm_delivery.py                        16      1      0      0    94%
doozerlib/rpm_utils.py                          134     23     90     14    81%
doozerlib/rpmcfg.py                             151     61     64      8    55%
doozerlib/runtime.py                            902    686    372      7    18%
doozerlib/source_modifications.py               116     33     26      4    68%
doozerlib/state.py                               23     12      8      0    35%
doozerlib/util.py                               452    217    180     13    48%
-------------------------------------------------------------------------------
TOTAL                                         12287   7388   5224    330    37%
___________________________________ summary ____________________________________
  py38: commands succeeded
  congratulations :)

@locriandev locriandev requested a review from thegreyd July 4, 2023 14:02
@locriandev locriandev merged commit ba41a55 into openshift-eng:master Jul 6, 2023
2 checks passed
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants