Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Repositories only containing subrepos (no repodata folder on repository root) are not supported #3452

Open
CERN-LHCb-Online opened this issue Feb 27, 2024 · 3 comments · May be fixed by #3481
Open

Repositories only containing subrepos (no repodata folder on repository root) are not supported #3452

CERN-LHCb-Online opened this issue Feb 27, 2024 · 3 comments · May be fixed by #3481
Assignees
@dralley
Labels
Issue

Comments

@CERN-LHCb-Online
Copy link

Version
pulpcore version: 3.47.0
pulp_rpm version: 3.25.1
Installed with pulp-minimal:3.47.0 docker container

Describe the bug
RHEL9 installation dvds have the following structure :

ls -la
total 53
dr-xr-xr-x 1root root  2048 27 oct.   2022 .
drwxr-xr-x 3 root root  4096  8 juil.  2020 ..
dr-xr-xr-x 1 root root  2048 27 oct.   2022 AppStream
dr-xr-xr-x 1 root root  2048 27 oct.   2022 BaseOS
-r--r--r-- 1 root root    60 27 oct.   2022 .discinfo
dr-xr-xr-x 1 root root  2048 27 oct.   2022 EFI
-r--r--r-- 1 root root  8154 27 oct.   2022 EULA
-r--r--r-- 1 root root  1455 27 oct.   2022 extra_files.json
-r--r--r-- 1 root root 18092 27 oct.   2022 GPL
dr-xr-xr-x 1 root root  2048 27 oct.   2022 images
dr-xr-xr-x 1 root root  2048 27 oct.   2022 isolinux
-r--r--r-- 1 root root   103 27 oct.   2022 media.repo
-r--r--r-- 1 root root  1669 27 oct.   2022 RPM-GPG-KEY-redhat-beta
-r--r--r-- 1 root root  3682 27 oct.   2022 RPM-GPG-KEY-redhat-release
-r--r--r-- 1 root root  1560 27 oct.   2022 .treeinfo

If we define this as a remote, we cannot currently do a sync as the sync immediatly stops trying to access a repodata/repomd.xml file as the root of the repository.

pulp --config /admin/.private/pulp.ini -v rpm repository sync --name RHEL9EB_BaseOS --sync-policy mirror_complete
repositories_rpm_rpm_list : get http://pulp:8080/pulp/api/v3/repositories/rpm/rpm/?name=RHEL9EB_BaseOS&offset=0&limit=1
Response: 200
repositories_rpm_rpm_sync : post http://pulp:8080/pulp/api/v3/repositories/rpm/rpm/018de9c3-229f-793c-9bb3-cc2861832657/sync/
Response: 202
tasks_read : get http://pulp:8080/pulp/api/v3/tasks/018de9c3-73f1-7eaa-bb64-e112852939b3/
Response: 200
Started background task /pulp/api/v3/tasks/018de9c3-73f1-7eaa-bb64-e112852939b3/
Error: Task /pulp/api/v3/tasks/018de9c3-73f1-7eaa-bb64-e112852939b3/ failed: 'An invalid remote URL was provided: http://10.128.124.12:8000/rhel-baseos-9.1-x86_64/'

On the http server, we get the following error :

10.128.124.142 - - [27/Feb/2024 09:51:57] code 404, message File not found
10.128.124.142 - - [27/Feb/2024 09:51:57] "GET /rhel-baseos-9.1-x86_64/repodata/repomd.xml HTTP/1.1" 404 -
10.128.124.142 - - [27/Feb/2024 09:51:57] "GET /rhel-baseos-9.1-x86_64 HTTP/1.1" 301 -
10.128.124.142 - - [27/Feb/2024 09:51:57] "GET /rhel-baseos-9.1-x86_64/ HTTP/1.1" 200 -

To Reproduce

  • Download rhel-baseos-9.1-x86_64-dvd.iso from RedHat
  • Extract the repository from the ISO
  • Run a web server to make the repository available
  • Create the remote and repository
pulp --config /admin/.private/pulp.ini rpm remote create --name RHEL9EB_BaseOS --url http://10.128.124.12:8000/rhel-baseos-9.1-x86_64/ --policy immediate
pulp --config /admin/.private/pulp.ini rpm repository create --name RHEL9EB_BaseOS --no-autopublish --remote RHEL9EB_BaseOS
  • Start the sync :
pulp --config /admin/.private/pulp.ini -v rpm repository sync --name RHEL9EB_BaseOS --sync-policy mirror_complete

Expected behavior
Complete sync of the the installation repository, following the definition in the .treeinfo file.

Additional context
I may have time in few months to make a test setup and try to fix this issue, but not in March
@CERN-LHCb-Online
Copy link
Author

The AlmaLinux-9-latest-x86_64-minimal.iso image also have no repodata folder at the root. It only have a 'Minimal' variant.

@CERN-LHCb-Online
Copy link
Author

In the mean time, I moved the package & repodata folders out of the BaseOS folder and modified the .treeinfo file accordingly and was then able to do a sync.

@dralley
Copy link
Contributor

dralley commented Feb 27, 2024 •
edited

Thank you for the report!

Closing #2306 as a duplicate of this one (which has more info available).

We haven't considered the possibility of a repository having a .treeinfo but no top-level repodata. We need to investigate whether this breaks anything (for example: autopublish), and potentially add a new test fixture to exercise it.

I don't think we had a concrete example of such a repository (or how the scenario would arise) previously.

dralley added a commit to pulp/pulp-fixtures that referenced this issue Feb 27, 2024
@dralley
Add a repository with an empty root, but treeinfo metadata
5b60baf 
re: pulp/pulp_rpm#3452
[noissue]
@dralley dralley mentioned this issue Feb 27, 2024
Merged
ggainey pushed a commit to pulp/pulp-fixtures that referenced this issue Feb 27, 2024
@dralley @ggainey
Add a repository with an empty root, but treeinfo metadata
87944a0 
re: pulp/pulp_rpm#3452
[noissue]
@dralley dralley self-assigned this Mar 25, 2024
dralley added a commit to dralley/pulp_rpm that referenced this issue Mar 25, 2024
@dralley
Fix syncing kickstart repositories with no "root" repo
7ccd083 
closes pulp#3452
dralley added a commit to dralley/pulp_rpm that referenced this issue Mar 25, 2024
@dralley
Fix syncing kickstart repositories with no "root" repo
fc04949 
closes pulp#3452
@dralley dralley linked a pull request Mar 25, 2024 that will close this issue
Draft
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@dralley dralley

Labels
Issue
Projects
RH Pulp Kanban board
Status: Needs review
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix syncing kickstart repositories with no "root" repo dralley/pulp_rpm
3 participants
@dralley @ggainey @CERN-LHCb-Online