Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci_find*: include sub and parent dirs #1130

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

bernt-matthias
Copy link
Contributor

@bernt-matthias bernt-matthias commented Jan 17, 2021

fixes #1129 and more:

include also tools/repos with:

  • change of test data
  • change of supplementary scripts (this already was the case for ci_find_repos)
  • change of macros where the tools are in sub-directories (this failed for ci_find_repos if each tool has an individual shed file)
  • change of a single tool in a collection where the shed file is in the parent

currently this will "fail" if the working dir of the planemo run is not the root of the repository: then changing for instance the README of the repository would lead to the inclusion of the all tools/repos in the repository. We could check if its the root by checking if there is a .git directory?

@bernt-matthias bernt-matthias changed the title ci_find*: include some sub/parent dirs ci_find*: include sub and parent dirs Jan 17, 2021
@bernt-matthias bernt-matthias force-pushed the topic/sub-parent branch 3 times, most recently from 44d0c6a to 2d7ab69 Compare January 17, 2021 13:55
include also tools/repos with:

- change of test data
- change of supplementary scripts
  (this already was the case for ci_find_repos)
- change of macros where the tools are in sub-directories
  (this failed for ci_find_repos if each tool has an
   individual shed file)
- change of a single tool in a collection where the shed
  file is in the parent
@mvdbeek
Copy link
Member

mvdbeek commented Jan 18, 2021

currently this will "fail" if the working dir of the planemo run is not the root of the repository: then changing for instance the README of the repository would lead to the inclusion of the all tools/repos in the repository. We could check if its the root by checking if there is a .git directory?

To clarify, the problem here is if planemo's working dir is e.g tools-iuc/tools/minimap2, and not tools-iuc/ ?
I think that's fine, planemo ci_find* is really meant for running within a CI system, and I think it's reasonable to expect that the working directory is the repository root.

On the other hand I don't think the presence of .git is a good proxy to check for this.

@bernt-matthias
Copy link
Contributor Author

the problem here is if planemo's working dir is e.g tools-iuc/tools/minimap2, and not tools-iuc/ ?

This might also be a problem. But I thought more of the case when the working dir is the parent of tools-iuc/.

I think that's fine, planemo ci_find* is really meant for running within a CI system, and I think it's reasonable to expect that the working directory is the repository root.

Then everything should be fine.

diff_paths = set()
for diff_dir in diff_dirs:
new_diff_paths = set()
while diff_dir != "" and len(new_diff_paths) == 0:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe a bit more explanation on the potential problem:

With diff_dir != "" I ensure that glob.glob(diff_dir + "/**/" is not called for the working dir. This is needed because otherwise a change of a file in the repository root would lead to the inclusion of all tools / repos.

Hence calling from outside of the repo this would not work .. BUT this is irrelevant, because git diff will fail before that.

Calling from subdirs would lead to ignoring changes in this dir.

Maybe we can assert the assumption by comparing the output of git rev-parse --show-toplevel with the working dir.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we could avoid going into these details maybe if you added a new flag for the recursive behavior (maybe --recursive) where in the help text you can list the limitations and how this is supposed to be used ?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That said if this lets you make progress on the planemo action I'm happy to merge and iterate on it if there are issues.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's fine. I can easily use test cases that include changes in the tool xml.

Just need to think of something different than --recursive, because this does not capture the essence. Maybe --extended_git_diff.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

ci_find_tools misses changed tools if tool XML is unchanged
2 participants