Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add: Add gitlab and github workflows. Also remove GDS question and GDS files #85

Open
wants to merge 28 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
ff55181
add: add basic github workflow file to package
joshlynchONS Mar 13, 2023
e12a7f4
fix: add workflow directories to env variables
joshlynchONS Mar 13, 2023
d5f4ae8
fix: escape jinja templating in the package_build workflow
joshlynchONS Mar 13, 2023
05fcffb
fix: fix syntax of jinja escape
joshlynchONS Mar 13, 2023
7dda5e5
fix: jinja escape syntax
joshlynchONS Mar 13, 2023
1356d0a
fix: jinja escape syntax
joshlynchONS Mar 13, 2023
89c897f
fix: fix the syntax for jinja escape on all matrix defs
joshlynchONS Mar 13, 2023
67359d5
add: add gitlab ci file
joshlynchONS Mar 14, 2023
eef7c44
add: add condition on python specific files
joshlynchONS Mar 14, 2023
017726a
fix: add conditions on the Github and gitlab files
joshlynchONS Mar 14, 2023
c2ae1ec
fix: fix injected variable count
joshlynchONS Mar 14, 2023
cab188b
fix: fix docker image for ons gitlab ci
joshlynchONS Mar 24, 2023
166f44f
fix: add extra cookiecutter question for in ons or not
joshlynchONS Mar 24, 2023
01b3063
chore: update cookiecutter var name for ons git
joshlynchONS Mar 24, 2023
4792760
chore: update workflow file name
joshlynchONS Mar 24, 2023
15a0481
fix: replace GDS question for ONS question for gitlab ci
joshlynchONS Mar 30, 2023
841a78b
remove: remove tests for aqa framework
joshlynchONS Mar 30, 2023
c2fd7a7
fix: move workflow templates for gitlab and github
joshlynchONS Mar 30, 2023
a2179ed
fix: add gitlab dir in envs and remove no longer required tests
joshlynchONS Mar 30, 2023
4cf5b1d
remove: remove organisational framework docs
joshlynchONS Mar 31, 2023
210067b
docs: remove organisational framework references from docs
joshlynchONS Mar 31, 2023
cbb2a3f
fix: add .gitlab to manifest.json to remove if chose github
joshlynchONS Mar 31, 2023
da34c6b
fix: rename pull request template for github
joshlynchONS Mar 31, 2023
447f092
fix: pre-commit error in gitlab-ci file
joshlynchONS Mar 31, 2023
d61c553
fix: pre-commit format error
joshlynchONS Mar 31, 2023
607f411
fix: revert gitlab ci format
joshlynchONS Mar 31, 2023
fc5d02b
fix: resolve merge conflicts with main
joshlynchONS Mar 31, 2023
a84f969
fix: update init.py style to match isort
joshlynchONS Mar 31, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
8 changes: 0 additions & 8 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -106,16 +106,8 @@ documentation][docs-write-accessible-documentation], as well as our [guidance on
writing Sphinx documentation][docs-write-sphinx-documentation]. This allows you to
build the documentation into an accessible, searchable website.

## Organisational frameworks

Organisational frameworks are stored in the
`.govcookiecutter/organisational_frameworks` folder. [If you would like to add your own
organisation's framework, follow the instructions][docs-govcookiecutter-frameworks] in
the `README.md` file in that folder.

[code-of-conduct]: https://github.com/best-practice-and-impact/govcookiecutter/blob/main/CODE_OF_CONDUCT.md
[coverage]: https://coverage.readthedocs.io/
[docs-govcookiecutter-frameworks]: https://github.com/best-practice-and-impact/govcookiecutter/blob/main/%7B%7B%20cookiecutter.repo_name%20%7D%7D/.govcookiecutter/organisational_frameworks/README.md
[docs-pre-commit-hooks]: https://github.com/best-practice-and-impact/govcookiecutter/blob/main/%7B%7B%20cookiecutter.repo_name%20%7D%7D/docs/contributor_guide/pre_commit_hooks.md
[docs-pre-commit-hooks-secrets-definition]: https://github.com/best-practice-and-impact/govcookiecutter/blob/main/%7B%7B%20cookiecutter.repo_name%20%7D%7D/docs/contributor_guide/pre_commit_hooks.md#definition-of-a-secret-according-to-detect-secrets
[docs-updating-gitignore]: https://github.com/best-practice-and-impact/govcookiecutter/blob/main/%7B%7B%20cookiecutter.repo_name%20%7D%7D/docs/contributor_guide/updating_gitignore.md
Expand Down
2 changes: 1 addition & 1 deletion cookiecutter.json
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
"repository_hosting_platform": ["GitHub", "GitLab"],
"organisation_handle": "Your GitHub/GitLab organisation name, for example ukgovdatascience",
"contact_email": "organisation@email.address",
"organisational_framework": ["GDS", "N/A"],
"organisational_framework": ["ONS", "N/A"],

"project_name": "Your new project name",
"repo_name": "{{ cookiecutter.project_name.lower().replace(' ', '_').replace('-', '_') }}",
Expand Down
1 change: 0 additions & 1 deletion docs/contributing_guide/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@ This is the contributor guide for the `govcookiecutter` project.
../CODE_OF_CONDUCT.md
../CONTRIBUTING.md
./modify_govcookiecutter.md
../{{ cookiecutter.repo_name }}/.govcookiecutter/organisational_frameworks/README.md
../{{ cookiecutter.repo_name }}/docs/contributor_guide/pre_commit_hooks.md
../{{ cookiecutter.repo_name }}/docs/contributor_guide/updating_gitignore.md
../{{ cookiecutter.repo_name }}/docs/contributor_guide/writing_accessible_documentation.md
Expand Down
7 changes: 0 additions & 7 deletions docs/contributing_guide/modify_govcookiecutter.md
Original file line number Diff line number Diff line change
Expand Up @@ -137,12 +137,6 @@ Notice the hyphen before the trailing `%` in each Jinja statement; this hyphen c
blank space after the statement. A hyphen after the leading `%` in a Jinja statement
controls blank space before the element.

## Replacing folders and files

[Replacing folders and files a more involved change, and is currently supported for
AQA frameworks and pull/merge request templates only][docs-organisational-frameworks].
These are performed in the `hooks/post_gen_project.py`file.

## Tests, coverage, and continuous integration

All pre- and post-generation hooks should be fully tested, alongside any generic
Expand Down Expand Up @@ -207,7 +201,6 @@ then use semantic versioning to number our releases][semver]. This helps our use
select a different version of `govcookiecutter` to use based on their individual needs.

[cookiecutter]: https://cookiecutter.readthedocs.io
[docs-organisational-frameworks]: https://github.com/best-practice-and-impact/govcookiecutter/blob/main/docs/%7B%7B%20cookiecutter.repo_name%20%7D%7D/.govcookiecutter/organisational_frameworks/README.md
[github-issues]: https://github.com/best-practice-and-impact/govcookiecutter/issues
[html5-email-format]: https://html.spec.whatwg.org/multipage/input.html#valid-e-mail-address
[jinja]: https://jinja.palletsprojects.com
Expand Down
10 changes: 0 additions & 10 deletions docs/reference/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,16 +28,6 @@ package. These include moving the selected organisational frameworks to the corr
location, as well as deleting unnecessary files and folders. If any post-generation
hooks fail, the generated project will be rolled-back, and deleted.

### Public sector organisational framework functions

```{eval-rst}
.. autosummary::
:toctree: api/

set_aqa_framework
set_request_template

```

### Post-generation clean up

Expand Down

This file was deleted.

9 changes: 1 addition & 8 deletions hooks/__init__.py
Original file line number Diff line number Diff line change
@@ -1,15 +1,8 @@
from hooks.post_gen_project import (
delete_files_and_folders,
parse_features_json,
set_aqa_framework,
set_request_template,
)
from hooks.post_gen_project import delete_files_and_folders, parse_features_json
from hooks.pre_gen_project import check_valid_email_address_format

__all__ = (
"delete_files_and_folders",
"check_valid_email_address_format",
"parse_features_json",
"set_aqa_framework",
"set_request_template",
)
93 changes: 0 additions & 93 deletions hooks/post_gen_project.py
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You removed these scripts. Do we need to adjust them rather than remove them completely?

Original file line number Diff line number Diff line change
Expand Up @@ -24,88 +24,6 @@ def delete_files_and_folders(paths: Union[Path, str, List[Path], List[str]]) ->
_ = [f.unlink() for f in paths if f.is_file() and f.exists()]


def set_aqa_framework(
dir_organisational_framework_aqa: Union[Path, str],
dir_cookiecutter_docs_aqa: Union[Path, str],
) -> None:
"""Set a specific organisational analytical quality assurance (AQA) framework.

Args:
dir_organisational_framework_aqa: A folder path that contains a specific
organisational AQA framework.
dir_cookiecutter_docs_aqa: A folder path within the outputted project
structure, where the contents of `dir_organisational_framework_aqa` will
reside.

Returns:
The organisation-specific AQA framework in the outputted project structure's
`dir_cookiecutter_docs_aqa` folder.

"""

# Remove the default `docs/aqa` folder, and its contents
delete_files_and_folders(dir_cookiecutter_docs_aqa)

# Copy the relevant organisational AQA framework to the `docs/aqa` folder
_ = Path(dir_organisational_framework_aqa).rename(dir_cookiecutter_docs_aqa)


def set_request_template(
path_organisational_framework_request_template: Union[Path, str],
dir_govcookiecutter: Union[Path, str],
repository_hosting_platform: str,
) -> None:
"""Set a pull or merge request template in the outputted project structure for a
specific organisation.

A pull request template is created if the user chooses GitHub as their repository
hosting platform. A merge request template is created if they choose GitLab instead.

Args:
path_organisational_framework_request_template: A file path to the specific
organisation pull or merge request template.
dir_govcookiecutter: A folder path to the outputted `govcookiecutter` template.
repository_hosting_platform: The repository hosting platform. Must be one of
"GitHub" or "GitLab" (case insensitive).

Returns:
A organisation-specific pull or merge request template in the correct location,
depending on their choice of GitHub or GitLab as the repository hosting
platform. If neither GitHub or GitLab are chosen, a `ValueError` is raised.

"""

# Define the file path where the request template will be moved, which is dependent
# on the repository hosting platform. If the `dir_request_template` is not
# initially one of `.github` or `.gitlab`, raise a `ValueError`
if repository_hosting_platform.lower() == "github":
path_request_template = Path(dir_govcookiecutter).joinpath(
f".{repository_hosting_platform.lower()}", "pull_request_template.md"
)
elif repository_hosting_platform.lower() == "gitlab":
path_request_template = Path(dir_govcookiecutter).joinpath(
f".{repository_hosting_platform.lower()}",
"merge_request_templates",
"{{ cookiecutter.project_name }}.md",
)
else:
raise ValueError(
"`repository_hosting_platform` must be one of `GitHub` or `GitLab`: "
f"{repository_hosting_platform}"
)

# Recursively create all the directories to the parent directory of
# `path_request_template` if they do not already exist
if not path_request_template.parent.is_dir():
path_request_template.parent.mkdir(parents=True, exist_ok=True)

# Move the `path_organisational_framework_request_template` to
# `path_request_template`
_ = Path(path_organisational_framework_request_template).rename(
path_request_template
)


def parse_features_json(file: Union[Path, str]) -> List[Path]:
"""Parse a JSON file containing filepaths.

Expand Down Expand Up @@ -148,17 +66,6 @@ def parse_features_json(file: Union[Path, str]) -> List[Path]:
"organisational_frameworks", "{{ cookiecutter.organisational_framework }}"
)

# Transfer the `aqa` folder, and the pull/merge request templates to the
# correct folder paths
set_aqa_framework(
DIR_ORGANISATIONAL_FRAMEWORKS.joinpath("aqa"), Path("docs").joinpath("aqa")
)
set_request_template(
DIR_ORGANISATIONAL_FRAMEWORKS.joinpath("request_template.md"),
Path.cwd(),
"{{ cookiecutter.repository_hosting_platform }}",
)

# Delete files defined in the `manifest.json` file
delete_files_and_folders(
parse_features_json(Path(".govcookiecutter", "manifest.json"))
Expand Down
41 changes: 1 addition & 40 deletions tests/test_govcookiecutter_creation.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,45 +5,6 @@
from sphinx.cmd.build import main


@pytest.mark.parametrize("test_input_repository_hosting_platform", ["GitHub", "GitLab"])
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why has this been removed?

@pytest.mark.parametrize("test_input_project_name", ["A project", "Another project"])
@pytest.mark.parametrize("test_input_using_r", ["Yes", "No"])
def test_request_template_generated_correctly(
cookies,
test_input_repository_hosting_platform: str,
test_input_project_name: str,
test_input_using_r: str,
) -> None:
"""Test the pull or merge request templates are created correctly."""

# Create a new project adding extra context; return it's `project_path` attribute
test_output_project = cookies.bake(
extra_context={
"repository_hosting_platform": test_input_repository_hosting_platform,
"project_name": test_input_project_name,
"using_R": test_input_using_r,
}
)

# Check that the build passes
assert test_output_project.exit_code == 0
assert test_output_project.exception is None

# Define the path to the pull or merge request template
test_output = test_output_project.project_path
if test_input_repository_hosting_platform == "GitHub":
assert test_output.joinpath(".github", "pull_request_template.md").is_file()
elif test_input_repository_hosting_platform == "GitLab":
assert test_output.joinpath(
".gitlab", "merge_request_templates", f"{test_input_project_name}.md"
).is_file()
else:
pytest.fail(
"Unknown `repository_hosting_platform` value: "
f"{test_input_repository_hosting_platform}"
)


@pytest.mark.skip(
reason="Unclear how to test this, unless there is a title in each " "framework"
)
Expand Down Expand Up @@ -101,7 +62,7 @@ def test_repo_name_directory_correct(

@pytest.mark.parametrize("test_input_context", args_builds_correctly)
@pytest.mark.parametrize("test_input_repository_hosting_platform", ["GitHub", "GitLab"])
@pytest.mark.parametrize("test_input_organisational_framework", ["GDS", "N/A"])
@pytest.mark.parametrize("test_input_organisational_framework", ["ONS", "N/A"])
@pytest.mark.parametrize("test_input_using_r", ["No", "Yes"])
def test_builds_correctly(
cookies,
Expand Down
2 changes: 1 addition & 1 deletion tests/test_govcookiecutter_injected_variables.py
Original file line number Diff line number Diff line change
Expand Up @@ -165,7 +165,7 @@ def recursive_open_and_count_search_terms(
(
"repo_name",
"repo_2",
{**REPO_NAME_COUNT, "{{ cookiecutter.repo_name }}": 5},
{**REPO_NAME_COUNT, "{{ cookiecutter.repo_name }}": 3},
{"using_R": "Yes"},
),
("overview", "overview_1", OVERVIEW_COUNT, {"using_R": "No"}),
Expand Down
91 changes: 0 additions & 91 deletions tests/test_post_gen_project/test_set_aqa_framework.py

This file was deleted.