Manage CI/CD job token scope allowlist

[nickgriffiths]

It would be great to be able to use GitLabForm to configure the allowlist of projects that are can use a project's CI_JOB_TOKEN.

https://docs.gitlab.com/ee/ci/jobs/ci_job_token.html#configure-cicd-job-token-access

The API for this is https://docs.gitlab.com/ee/api/project_job_token_scopes.html

[amimas]

Thanks for the suggestion @nickgriffiths. What should the gitlabform config syntax look like? Can you post some proposals?

Will you or any of your colleague be able to contribute this feature?

[nickgriffiths]

Thanks for the suggestion @nickgriffiths. What should the gitlabform config syntax look like? Can you post some proposals?

Will you or any of your colleague be able to contribute this feature?

I'm afraid I haven't given any thought to syntax, and I can't work on this myself at the moment, but I have privately flagged this to some colleagues to see if anyone might be interested in taking it on.

[amimas]

Sounds good. Thanks @nickgriffiths

[TimKnight-DWP]

Hi, I plan to take on this work after getting my head into using python-gitlab

[TimKnight-DWP]

Related Gitlab Issues:

• https://gitlab.com/gitlab-org/gitlab/-/merge_requests/143132 - MR to allow configuration of job token and adding "groups" via GraphQL API
• https://gitlab.com/gitlab-org/gitlab/-/issues/435903 - overarching issue for configuring groups

[TimKnight-DWP]

@amimas
My proposal is we should add something to project_settings, as per API: https://docs.gitlab.com/ee/api/project_job_token_scopes.html#add-a-project-to-a-cicd-job-token-inbound-allowlist

project_settings:
  - ci_job_token_allowlist: [array of strings - project or group name to add]

Right now we'd be able to take Project nam, map to Project ID and invoke the REST endpoint (via python-gitlab) to add that Project

In future, pending Gitlab Development, we can take Group name and invoke the endpoint (we might end up needing to map group name to group id)

[TimKnight-DWP]

Questions I have, is how does Gitlabform handle fields where we coudl take either a group name or project name?

[amimas]

Hi @TimKnight-DWP - Thank you for wanting to contribute this feature. Really appreciate the help.

In terms of config syntax, here's what I'm thinking, although it's not too different from your proposal I believe.

group-foo/project-abc:
  project_settings:
    # existing project settings related configs as-is
  branches:
    # existing branch protection related configs as-is
  job_token_scope:
    # new config proposal
    enabled: ... # accepted values are `true` or `false`
    allowlist:
      enforce: .... # accepted values are `true` or `false`
      projects:
        - 123
        - group-bar/project-xyz

In the above sample, I think this new feature should be under a new key instead of project_settings. It's because project_settings key is used for... well editing project settings supported by this API endpoint. Since gitlabform follows "raw parameter passing" pattern, any new attributes under project_settings will simply be passed to that API, unless we add exception logic. I think let's keep it simple and add the new feature under a new key. That's why suggested job_token_scope but feel free to suggest an alternate name.

Under job_token_scope, I added an attribute named enabled. This doesn't have to be part of this issue's implementation, but I'm including it for the sake of completeness because it's a related configuration and has already been requested in #607

Under job_token_scope:allowlist, I introduced the enforce config. I think this should be included as part of this issue's implementation, but it's not a hard requirement. Take a look at other configs that supports enforce key. If you have to remove a project from the allowlist later, it should be as simple as removing the relevant reference from the config and gitlabform should take care of the rest.

Under job_token_scope:allowlist:projects, I'm using 2 different references. One uses the project ID and the other uses the project path with namespace. I feel the later would be more helpful or human-friendly. It's also a pattern that is specifically supported when referring to users or groups under various other configs; you can refer to the name in addition to the ID. So, doing it this way just follows the existing pattern.

I hope the above make sense. Please let us know if you have questions or suggestions.

[amimas]

Under job_token_scope, I added an attribute named enabled. This doesn't have to be part of this issue's implementation, but I'm including it for the sake of completeness because it's a related configuration and has already been requested in #607

Actually... on second thought, this might have to be implemented as part of this. If job token scope is disabled, the following config would be invalid because you can't also add projects to allowlist, right?

group-foo/project-abc:
  job_token_scope:
    enabled: false
    allowlist:
      projects:
        - 123
        - group-bar/project-xyz

In the above config, I believe you can't add projects to the allowlist. Or is it the other way around (i.e. when enabled: true)? They labelled this setting in the UI as "Limit access to this project". So, it always throws me off for some reason :)

Anyways, my point is, I think this has to be part of the implementation?

What do you think?

[TimKnight-DWP]

@amimas in GL 17 I don't think there will be an enable true/false option, simply there will be an allowlist, so we shouldn't have to worry about accounting for this, unless we want backwards compatibility to GL 16.

At which point if enabled or "Limit Access" is true, hen projects would be added and allow list used.

If enabled is false, it should still add projects to allowlist (replicates UI functionality), but limit access is toggled off

[TimKnight-DWP]

GL have implemented Group addition to allowlist via REST: https://gitlab.com/gitlab-org/gitlab/-/merge_requests/145069

So suspect similar API would be

group-foo/project-abc:
  job_token_scope:
    enabled: false
    allowlist:
      groups:
        - group-bar

or

group-foo/project-abc:
  job_token_scope:
    enabled: false
    allowlist:
      projects_and_groups:
        - group-bar
        - group-abc/project-123

I think the former, with separation of fields is better, matches the udnerlying API and I assumed will be a little easier to handle in the code logic

[TimKnight-DWP]

FYI I'll probably be contributing into python-gitlab python-gitlab/python-gitlab#2767 to help add this functionality into there first

[amimas]

Thanks @TimKnight-DWP. Forgot that the boolean option is deprecated and expected to be removed in v17.

As for a group being added to the allowlist, your first suggestion makes sense to me as well.

[TimKnight-DWP]

@amimas - the groups_allowlist at least is only being released in 16.10 (March 21st), I'm working on the python-gl implementation on my fork: python-gitlab/python-gitlab#2816

Is there a way I can point a branch/fork of Gitlab Form to this fork of python-gl so I can develop in parallel, ready for a release with 16.10?

[amimas]

Hi @TimKnight-DWP . I'm not well versed in python packaging yet. I think you need to update the version reference in setup.py here:

gitlabform/setup.py

Line 55 in dd58a7a

"python-gitlab==4.4.0",

According to this article you can also point it to a github url but maybe that will still expect a package from there...

https://python-packaging.readthedocs.io/en/latest/dependencies.html#packages-not-on-pypi

[amimas]

I was looking at that setting in a project for work today. There're 2 settings. One is called "limit access to this project". The 2nd one is called "limit access from this project". To me it seems the deprecation is for the 2nd option. So the firat boolean setting will still stay I think? Could you confirm @TimKnight-DWP .

[TimKnight-DWP]

@amimas yes you're right, that's my reading of the deprecation notices too.

Let's call the first boolean limit_access_to_this_project ? bit wordy but does map to Gitlab?

[TimKnight-DWP]

Nevermind, they swap between terminology throughout the API and UI for this setting 🙈 .....
Maybe GLF should be consistent and make it do what it says, or at least be consistent with what "enabled" means

In PATCH https://docs.gitlab.com/ee/api/project_job_token_scopes.html#patch-a-projects-cicd-job-token-access-settings, it's "enabled" true/false:

Indicates CI/CD job tokens generated in other projects have restricted access to this project.

(my reading is that, true = Limit Access to project is on)

In GET https://docs.gitlab.com/ee/api/project_job_token_scopes.html#get-a-projects-cicd-job-token-access-settings, it's inbound_enabled true/false

Indicates if the CI/CD job token generated in other projects has access to this project.

(my reading would be true = Limit Access to project is off)

[TimKnight-DWP]

I've tested it, spoken to our contact at Gitlab (the benefits of working somewhere with Ultimate license), and raised an MR to clarify Gitlab's REST api documentation based on what I found out: https://gitlab.com/gitlab-org/gitlab/-/merge_requests/146763

[amimas]

Thanks for digging into that boolean setting. Hopefully things get clarified.

Let's call the first boolean limit_access_to_this_project ? bit wordy but does map to Gitlab?
Maybe GLF should be consistent and make it do what it says, or at least be consistent with what "enabled" means

Too bad GitLab has 2 different terms for it in their API (i.e. enabled vs inbound_enabled). I think your suggestion to use limit_access_to_this_project as the key name in gitlabform config makes sense to me. To me, it's clear and concise. I'm not sure, but I think the 2 comments above are talking about the same thing?

[TimKnight-DWP]

Yes, that was just me stream of thoughting into the comments :D

[TimKnight-DWP]

@amimas @nickgriffiths changes to support this are here: TimKnight-DWP#1

Currently on my fork only because I need to point at my fork of python-gitlab to pick up that part of the chain. Everything is capable of being merged when gitlab 16.10 gets released, so I'll update the MR to point back to mainline python-gitlab when this MR goers in python-gitlab/python-gitlab#2816.

However a review of the proposed changes would be great if possible :)