Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Pipeline - Adding acr img clean up (#2222)
* Pipeline - Adding acr img clean up TODO Fixed #2210 * Adding step to login into Azure * Updating job name and order of azure login step * Updating cli command * Updating cli to get the list of acr images * Adding ACR name and correct image name to get the list * removing show-tags from the cmd * Adding image tag for the list * removing extra tags for the list * updating cmd to get the tags * Filtering out the tags * Adding order by command for the repo * Updating the variable name * Removing limit on the query * Adding comparison for the active PR and images * Adding pipeline to run the matrix * Adding output for the comparison step * Adding prop as JSON param * Deleting test image * Adding image name with tag to delete it * Updating schedule and adding login steps for Azure * cleaning weekly check and adding updated checks * removing the tags limit * Adding limiting to 250 * Adding deletion step to clean up the image * Adding comment and reducing noise * Adding non-pr prefixed tags * Tags - Wrapping it with quote * Updating the variable name * echoing non-pr tags * merging non-pr tags * Fixing the variable name * updating variable name * Adding prefix condition for non-pr tags * wrapping text around input values * Adding additional checks to not delete production and staging acr * Adding staging and production sha * removing PRs for temporary * deleting based on digest value * Getting digest data for specific ACR * replacing repository with registery * manifest for the specific tag * getting digest value in a different way * removing test docker image * Adding github sha to track it * adding period with the output * Removing sha tag from the docker img * Adding prs tags only to delete * prefixing with dollar sign * Adding non-prs tags * Changing the limit of tag list * Cleaning up the steps for the delete slots * Removing redundant steps * wrapping the tags within a quote * Removing acr login * Adding condition to stop the second flow * Removing PR trigger * Passing JSON to call delete pr flow * Adding step to remove untagged digests * Adding PR trigger * Updating the query to retrieve the manifest * Adding acr tag * Updating the props for the list * adding delete command to delete untagged shas * Adding yes prop to confirm delete * removing PR trigger * Removing testing scripts
- Loading branch information
1 parent
05ee207
commit 438aaf8
Showing
6 changed files
with
198 additions
and
13 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,72 @@ | ||
| name: Delete - Docker image from ACR | ||
|
|
||
| on: | ||
| workflow_call: | ||
| inputs: | ||
| imageTags: | ||
| type: string | ||
| description: "Image Tags or PR numbers" | ||
| required: true | ||
| workflow_dispatch: | ||
| inputs: | ||
| imageTag: | ||
| description: "Image Tag or PR number" | ||
| required: true | ||
|
|
||
| defaults: | ||
| run: | ||
| shell: pwsh | ||
|
|
||
| env: | ||
| IMAGE_TAGS: ${{ inputs.imageTags || inputs.imageTag }} | ||
| PREFIX: pr- | ||
| permissions: | ||
| id-token: write | ||
| contents: read | ||
|
|
||
| jobs: | ||
| setting-up-img-tags: | ||
| runs-on: ubuntu-latest | ||
| outputs: | ||
| matrix: ${{ steps.set-matrix.outputs.matrix }} | ||
| steps: | ||
| - name: Get image tags from input | ||
| id: set-matrix | ||
| run: | | ||
| $imageTags = '${{ env.IMAGE_TAGS }}' | ||
| echo "matrix=$imageTags" >> $env:GITHUB_OUTPUT | ||
| delete-acr-image: | ||
| runs-on: ubuntu-latest | ||
| needs: setting-up-img-tags | ||
| strategy: | ||
| matrix: | ||
| IMAGE_TAG: ${{ fromJson(needs.setting-up-img-tags.outputs.matrix) }} | ||
|
|
||
| steps: | ||
| - uses: actions/checkout@v4 | ||
|
|
||
| - name: Load .env file | ||
| uses: xom9ikk/dotenv@v2 | ||
| with: | ||
| path: ./.github | ||
|
|
||
| - name: Azure CLI - Login | ||
| uses: azure/login@v1 | ||
| with: | ||
| client-id: ${{ secrets.AZURE_CLIENT_ID }} | ||
| tenant-id: ${{ secrets.AZURE_TENANT_ID }} | ||
| subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }} | ||
|
|
||
| - name: ACR - Delete image | ||
| run: | | ||
| if('${{ matrix.IMAGE_TAG }}'.Length -le 4) # PR tags consist of 4 digits (i.e pr-xxxx) | ||
| { | ||
| $imageTagWithPrefix = '${{ env.PREFIX }}${{ matrix.IMAGE_TAG}}' | ||
| }else{ | ||
| $imageTagWithPrefix = '${{ matrix.IMAGE_TAG }}' | ||
| } | ||
| az acr repository delete --name ${{ env.ACR_NAME }} ` | ||
| --image ${{ env.IMAGE_NAME }}:$imageTagWithPrefix --yes | ||
| Write-Output "✅ ACR - ${{ env.IMAGE_NAME }}:$imageTagWithPrefix image deleted successfully." |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,103 @@ | ||
| name: Weekly ACR images cleanup | ||
|
|
||
| on: | ||
| schedule: | ||
| # Monday at 2 PM UTC - https://cron.help/#0_14_*_*_MON | ||
| - cron: "0 14 * * MON" | ||
| workflow_dispatch: | ||
|
|
||
| env: | ||
| GH_TOKEN: ${{ github.token }} | ||
|
|
||
| defaults: | ||
| run: | ||
| shell: pwsh | ||
|
|
||
| permissions: | ||
| id-token: write | ||
| contents: read | ||
|
|
||
| jobs: | ||
| check-acr-images: | ||
| runs-on: ubuntu-latest | ||
| outputs: | ||
| imageTagList: ${{ steps.comparison.outputs.imageTagList }} | ||
| steps: | ||
| - name: Checking out | ||
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Load .env file | ||
| uses: xom9ikk/dotenv@v2 | ||
| with: | ||
| path: ./.github | ||
|
|
||
| - name: Azure CLI - Login | ||
| uses: azure/login@v1 | ||
| with: | ||
| client-id: ${{ secrets.AZURE_CLIENT_ID }} | ||
| tenant-id: ${{ secrets.AZURE_TENANT_ID }} | ||
| subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }} | ||
|
|
||
| - name: Delete all untagged images | ||
| run: | | ||
| $untaggedDigests = az acr manifest list-metadata -r ${{ env.ACR_NAME }} -n ` | ||
| ${{ env.IMAGE_NAME }} --query "[?tags==null].digest" -o tsv | ` | ||
| %{ az acr repository delete -n ${{ env.ACR_NAME }} -t ${{ env.IMAGE_NAME }}@$_ --yes} | ||
| Write-Host "✅ All untagged images have been deleted" | ||
| - name: Get list of active PRs | ||
| id: prList | ||
| run: | | ||
| $active_prs=$(gh pr list --state open --json number | jq -r '.[].number') | ||
| echo "active_prs=$active_prs" >> $env:GITHUB_OUTPUT | ||
| - name: Get list of ACR image tags | ||
| id: imageTags | ||
| run: | | ||
| $images = az acr repository show-tags ` | ||
| --name ${{ env.ACR_NAME }} --repository ${{ env.IMAGE_NAME }} ` | ||
| --top 250 --orderby time_asc --output tsv # Limiting to 250 tags because of the GitHub action matrix limit | ||
| # Filter tags that start with "pr-" | ||
| $filteredTags = $images | Where-Object { $_ -like "pr-*" } | ||
| # Remove the "pr-" prefix from filtered tags | ||
| $filteredTagsWithoutPrefix = $filteredTags -replace "^pr-", "" | ||
| echo "filteredTags=$filteredTagsWithoutPrefix" >> $env:GITHUB_OUTPUT | ||
| - name: Compare active PRs with existing ACR images | ||
| id: comparison | ||
| run: | | ||
| # Comparing the number of images and PRs | ||
| $prList = "${{ steps.PRList.outputs.active_prs }}" -split ' ' | ||
| $imageTags = "${{ steps.imageTags.outputs.filteredTags }}" -split ' ' | ||
| $imagesExistThatRequireDeletion = $imageTags | Where-Object { $_ -notin $prList } | ||
| $imagesNeedDeletion = $imagesExistThatRequireDeletion.Length -gt 0 | ||
| if ( ! $imagesNeedDeletion ) { | ||
| echo "✅ - Number of docker images are equal to number of active PRs - 🏃 Skipping next step" | ||
| } | ||
| else { | ||
| echo "❌ - Number of docker images are not equal to number of active PRs" | ||
| Write-Host "⚡- These images need to be deleted : $imagesExistThatRequireDeletion" | ||
| } | ||
| # Convert string into Array | ||
| $tags = $imagesExistThatRequireDeletion -split ' ' | ||
| $imageTagList = ConvertTo-Json -Compress @($tags) | ||
| echo "imageTagList=$imageTagList" >> $env:GITHUB_OUTPUT | ||
| invokeDeleteImage: | ||
| name: Invoking delete-acr-image | ||
| needs: | ||
| - check-acr-images #Adding second check to avoid running this flow | ||
| if: needs.check-acr-images.outputs.imageTagList != '[]' | ||
| uses: ./.github/workflows/template-delete-acr-image.yml | ||
| with: | ||
| imageTags: ${{ needs.check-acr-images.outputs.imageTagList }} | ||
| permissions: | ||
| id-token: write | ||
| contents: read | ||
| secrets: inherit |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters