This repository has been archived by the owner on Oct 13, 2023. It is now read-only.
[ART-4558] Use repoquery to detect non-latest rpms #764
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Mostly tested locally, but didn't test it in production. Haven't got time to add unit tests. Needs the following PRs before merging:
|
Build #1 |
Build #2 |
Build #3 |
Build #4 |
vfreex
force-pushed
the
scanning-content-based
branch
from
2f240c4
to
0e58c99
Compare
|
Added unit tests. |
vfreex
force-pushed
the
scanning-content-based
branch
from
0e58c99
to
e1b8a8b
Compare
Build #5 |
Build #6 |
joepvd
approved these changes
May 9, 2023
There was a problem hiding this comment.
/lgtm
Left two comments that might need addressing.
Think it is a welcome change to get informed about non-current entries in rhcos builds that are from other repositories than our own. Even if this is an area that will see some movement 🔜
OUTDATED_RPMS_IN_STREAM_BUILD will only create an complaint in the log, and an annotation on the imageStream. The extended complaints will not interfere with release work.
doozerlib/cli/release_gen_payload.py
Outdated
| """ | ||
| Associate relevant assembly issues with an RHCOS PayloadEntry. | ||
| Use assembly inspector to detect assembly issues like RPMs installed in | ||
| the PayloadEntry that are not what the assembly specifies. | ||
| """ | ||
|
|
||
| # record issues found in assembly list and per payload entry | ||
| self.assembly_issues.extend(assembly_inspector.check_rhcos_issues(payload_entry.rhcos_build)) | ||
| rhcos_build = cast(RHCOSBuildInspector, payload_entry.rhcos_build) | ||
| self.assembly_issues.extend(assembly_inspector.check_rhcos_issues(rhcos_build)) | ||
| payload_entry.issues.extend(ai for ai in self.assembly_issues if ai.component == "rhcos") | ||
|
|
||
| if self.runtime.assembly_type is AssemblyTypes.STREAM: | ||
| # For stream alone, we want to enforce that the very latest RPMs are installed. |
There was a problem hiding this comment.
Not your change, but this comment is misleading. We're not enforcing this, we're emitting a warning. In before times, doozer would allow OUTDATED_RPMS_IN_STREAM_BUILD in code. Now we have a permits in stream to say we want this to not be a blocker.
| if cached_result is not None: # cache hit | ||
| return cached_result | ||
|
|
||
| # Cache miss. Will actually query the repo. |
There was a problem hiding this comment.
Think the cache miss also should be inside the lock. Otherwise, there could be multiple queries running at the same time.
There was a problem hiding this comment.
Putting repoquery inside of the lock will prevent from multiple queries over different repos. There is a possibility to run multiple queries over the same repo, I think it is acceptable for now. Though there could still be a room to optimize this by using, such as, more fine grained locks, but I'd like to see how much time this will take in production.
There was a problem hiding this comment.
I added some comments here.
Build #7 |
vfreex
force-pushed
the
scanning-content-based
branch
from
e1b8a8b
to
5222a72
Compare
Build #8 |
Build #9 |
Currently we detect outdated rpms in images and RHCOS by comparing the NVRs of the installed rpms to the NVRs of the rpms in our `rhaos-x.y-rhel-z-candidate` Brew tag. The current approach has 2 issues. See ART-4558 and ART-3965. With this new approach, `repoquery` command will be used to query available RPMs in the configured build repos instead of the candidate Brew tag. This will provide more accurate detection and also be able to detect rpms that are not in rhaos repos (e.g. RHEL repos). Note this PR only covers the `gen-payload` change. A change for `scan-sources` will be another PR.
vfreex
force-pushed
the
scanning-content-based
branch
from
5222a72
to
55a9652
Compare
Build #10 |
vfreex
added a commit
to vfreex/doozer
that referenced
this pull request
Jun 20, 2023
In openshift-eng#764, the idea of using repoquery to detect outdated rpms has been proved. This PR will adopt the same approach in `scan_sources`.
vfreex
added a commit
to vfreex/doozer
that referenced
this pull request
Jun 20, 2023
In openshift-eng#764, the idea of using repoquery to detect outdated rpms has been proved in `gen-payload`. This PR will adopt the same approach in `scan_sources`.
vfreex
added a commit
to vfreex/doozer
that referenced
this pull request
Jun 20, 2023
In openshift-eng#764, the idea of using repoquery to detect outdated rpms has been proved in `gen-payload`. This PR will adopt the same approach in `scan_sources`.
vfreex
added a commit
to vfreex/doozer
that referenced
this pull request
Jul 11, 2023
In openshift-eng#764, the idea of using repoquery to detect outdated rpms has been proved in `gen-payload`. This PR will adopt a similar approach in `scan_sources`, get rid of `repoquery` command and also add rpm modularity awareness.
vfreex
added a commit
to vfreex/doozer
that referenced
this pull request
Jul 12, 2023
Currently we use a tag based approach to detect rpm changes in stream builds. It considers a package X is outdated if a newer version Y is tagged into any of X's tags. This approach has some limitations. Especially, it can't detect rpm changes after RHEL version moves or the image moves to a different repo (see https://issues.redhat.com/browse/ART-3965 for a real problem). I will give you an example to show how it fails to detect a change: - Image A enables a RHEL 8.4 repo. - Package `foo-1.0.0-1.el8_3.x86_64.rpm` is installed in image A. It has `rhel-8.3.0-z` tag in Brew. - A newer version `foo-1.1.0-1.el8_4.x86_64.rpm` is released and has `rhel-8.4.0-z` tag. - Doozer doesn't think `foo-1.0.0-1.el8_3.x86_64.rpm` is outdated because the newer version `foo-1.1.0-1.el8_4.x86_64.rpm` doesn't have `rhel-8.3.0-z` tag. To address this limitation, a new approach is proposed to actually query the content of all enabled repos and compare the installed versions with the latest versions in repos. I will call it "content based" approach. This content based approach has been proved in `gen-payload` command (invoked in `build-sync` job) with the merge of openshift-eng#764. If you look at [the logs of build-sync build](https://saml.buildvm.hosts.prod.psi.bos.redhat.com:8888/job/aos-cd-builds/job/build%252Fbuild-sync/34458/consoleFull), you will see it shows you which images contain outdated rpms. It contains entries like: ```yaml - code: OUTDATED_RPMS_IN_STREAM_BUILD msg: Found outdated RPM (perl-Pod-Perldoc-3.28-396.el8) installed in openshift-enterprise-builder-container-v4.14.0-202307111728.p0.gfe78ee6.assembly.stream (x86_64) when perl-Pod-Perldoc-3.28.01-443.module+el8.6.0+13324+628a2397 was available in repo rhel-8-appstream-rpms ``` However, the above example is actually a false positive. It is because package `perl-Pod-Perldoc-3.28-396.el8` comes from a modular repo but our code is not module-aware. A modular rpm repo provides multiple upgrade paths. We should only consider a newer version is upgradable if it belongs to the same stream. For more information about modules, see https://docs.fedoraproject.org/en-US/modularity/. That is a major reason why this new approach is adopted to `scan-sources`, otherwise it would constantly trigger unnecessary rebuilds! This PR not only updates `scan-sources` to use the new approach, but only adds module-awareness to eliminate false positives. The new algorithm (probably oversimplified): 1. For an image build, get all modular rpms in enabled repos. 2. For each installed rpm, check if the rpm is contained by a module. 3. If yes, we will consider that module stream is "enabled" for this image. 4. For each enabled module stream, find the module with the largest version. 5. Collect all modular rpms from those modules as collection M. 6. Collect all non-modular rpms with the latest version as collection N. If the package name of a non-modular rpm is already included in the modular rpms collection, exclude it. 7. For each installed rpm, check if it is a modular rpm or not. If it is modular, check if it is older than the one in collection M. Otherwise, check if it is older than the one in collection N. 8. If it is older, we consider is it an outdated rpm. Note: 1. There is no Brew API or any other cheap way to precisely determine which module streams are enabled during an image build. The method used in the above algorithm is actually *guess*. It is not perfect, but my tests conclude it is good enough for our use cases (probably because we don't use a lot of modular rpms in our images). 2. Some images use pins in their Dockerfiles. We should exempt them from scan-sources. The exemption can be defined in ocp-build-data image metadata. e.g. ```yaml scan_sources: exempted_packages: - openvswitch3.1 - python3-openvswitch3.1 ```
vfreex
added a commit
to vfreex/doozer
that referenced
this pull request
Jul 12, 2023
Currently we use a tag based approach to detect rpm changes in stream builds. It considers a package X is outdated if a newer version Y is tagged into any of X's tags. This approach has some limitations. Especially, it can't detect rpm changes after RHEL version moves or the image moves to a different repo (see https://issues.redhat.com/browse/ART-3965 for a real problem). I will give you an example to show how it fails to detect a change: - Image A enables a RHEL 8.4 repo. - Package `foo-1.0.0-1.el8_3.x86_64.rpm` is installed in image A. It has `rhel-8.3.0-z` tag in Brew. - A newer version `foo-1.1.0-1.el8_4.x86_64.rpm` is released and has `rhel-8.4.0-z` tag. - Doozer doesn't think `foo-1.0.0-1.el8_3.x86_64.rpm` is outdated because the newer version `foo-1.1.0-1.el8_4.x86_64.rpm` doesn't have `rhel-8.3.0-z` tag. To address this limitation, a new approach is proposed to actually query the content of all enabled repos and compare the installed versions with the latest versions in repos. I will call it "content based" approach. This content based approach has been proved in `gen-payload` command (invoked in `build-sync` job) with the merge of openshift-eng#764. If you look at [the logs of build-sync build](https://saml.buildvm.hosts.prod.psi.bos.redhat.com:8888/job/aos-cd-builds/job/build%252Fbuild-sync/34458/consoleFull), you will see it shows you which images contain outdated rpms. It contains entries like: ```yaml - code: OUTDATED_RPMS_IN_STREAM_BUILD msg: Found outdated RPM (perl-Pod-Perldoc-3.28-396.el8) installed in openshift-enterprise-builder-container-v4.14.0-202307111728.p0.gfe78ee6.assembly.stream (x86_64) when perl-Pod-Perldoc-3.28.01-443.module+el8.6.0+13324+628a2397 was available in repo rhel-8-appstream-rpms ``` However, the above example is actually a false positive. It is because package `perl-Pod-Perldoc-3.28-396.el8` comes from a modular repo but our code is not module-aware. A modular rpm repo provides multiple upgrade paths. We should only consider a newer version is upgradable if it belongs to the same stream. For more information about modules, see https://docs.fedoraproject.org/en-US/modularity/. That is a major reason why this new approach is adopted to `scan-sources`, otherwise it would constantly trigger unnecessary rebuilds! This PR not only updates `scan-sources` to use the new approach, but only adds module-awareness to eliminate false positives. The new algorithm (probably oversimplified): 1. For an image build, get all modular rpms in enabled repos. 2. For each installed rpm, check if the rpm is contained by a module. 3. If yes, we will consider that module stream is "enabled" for this image. 4. For each enabled module stream, find the module with the largest version. 5. Collect all modular rpms from those modules as collection M. 6. Collect all non-modular rpms with the latest version as collection N. If the package name of a non-modular rpm is already included in the modular rpms collection, exclude it. 7. For each installed rpm, check if it is a modular rpm or not. If it is modular, check if it is older than the one in collection M. Otherwise, check if it is older than the one in collection N. 8. If it is older, we consider is it an outdated rpm. Note: 1. There is no Brew API or any other cheap way to precisely determine which module streams are enabled during an image build. The method used in the above algorithm is actually *guess*. It is not perfect, but my tests conclude it is good enough for our use cases (probably because we don't use a lot of modular rpms in our images). 2. Some images use pins in their Dockerfiles. We should exempt them from scan-sources. The exemption can be defined in ocp-build-data image metadata. e.g. ```yaml scan_sources: exempted_packages: - openvswitch3.1 - python3-openvswitch3.1 ```
vfreex
added a commit
to vfreex/doozer
that referenced
this pull request
Jul 12, 2023
Currently we use a tag based approach to detect rpm changes in stream builds. It considers a package X is outdated if a newer version Y is tagged into any of X's tags. This approach has some limitations. Especially, it can't detect rpm changes after RHEL version moves or the image moves to a different repo (see https://issues.redhat.com/browse/ART-3965 for a real problem). I will give you an example to show how it fails to detect a change: - Image A enables a RHEL 8.4 repo. - Package `foo-1.0.0-1.el8_3.x86_64.rpm` is installed in image A. It has `rhel-8.3.0-z` tag in Brew. - A newer version `foo-1.1.0-1.el8_4.x86_64.rpm` is released and has `rhel-8.4.0-z` tag. - Doozer doesn't think `foo-1.0.0-1.el8_3.x86_64.rpm` is outdated because the newer version `foo-1.1.0-1.el8_4.x86_64.rpm` doesn't have `rhel-8.3.0-z` tag. To address this limitation, a new approach is proposed to actually query the content of all enabled repos and compare the installed versions with the latest versions in repos. I will call it "content based" approach. This content based approach has been proved in `gen-payload` command (invoked in `build-sync` job) with the merge of openshift-eng#764. If you look at [the logs of build-sync build](https://saml.buildvm.hosts.prod.psi.bos.redhat.com:8888/job/aos-cd-builds/job/build%252Fbuild-sync/34458/consoleFull), you will see it shows you which images contain outdated rpms. It contains entries like: ```yaml - code: OUTDATED_RPMS_IN_STREAM_BUILD msg: Found outdated RPM (perl-Pod-Perldoc-3.28-396.el8) installed in openshift-enterprise-builder-container-v4.14.0-202307111728.p0.gfe78ee6.assembly.stream (x86_64) when perl-Pod-Perldoc-3.28.01-443.module+el8.6.0+13324+628a2397 was available in repo rhel-8-appstream-rpms ``` However, the above example is actually a false positive. It is because package `perl-Pod-Perldoc-3.28-396.el8` comes from a modular repo but our code is not module-aware. A modular rpm repo provides multiple upgrade paths. We should only consider a newer version is upgradable if it belongs to the same stream. For more information about modules, see https://docs.fedoraproject.org/en-US/modularity/. That is a major reason why this new approach is adopted to `scan-sources`, otherwise it would constantly trigger unnecessary rebuilds! This PR not only updates `scan-sources` to use the new approach, but only adds module-awareness to eliminate false positives. The new algorithm (probably oversimplified): 1. For an image build, get all modular rpms in enabled repos. 2. For each installed rpm, check if the rpm is contained by a module. 3. If yes, we will consider that module stream is "enabled" for this image. 4. For each enabled module stream, find the module with the largest version. 5. Collect all modular rpms from those modules as collection M. 6. Collect all non-modular rpms with the latest version as collection N. If the package name of a non-modular rpm is already included in the modular rpms collection, exclude it. 7. For each installed rpm, check if it is a modular rpm or not. If it is modular, check if it is older than the one in collection M. Otherwise, check if it is older than the one in collection N. 8. If it is older, we consider is it an outdated rpm. Note: 1. There is no Brew API or any other cheap way to precisely determine which module streams are enabled during an image build. The method used in the above algorithm is actually *guess*. It is not perfect, but my tests conclude it is good enough for our use cases (probably because we don't use a lot of modular rpms in our images). 2. Some images use pins in their Dockerfiles. We should exempt them from scan-sources. The exemption can be defined in ocp-build-data image metadata. e.g. ```yaml scan_sources: exempted_packages: - openvswitch3.1 - python3-openvswitch3.1 ```
vfreex
added a commit
to vfreex/doozer
that referenced
this pull request
Jul 12, 2023
Currently we use a tag based approach to detect rpm changes in stream builds. It considers a package X is outdated if a newer version Y is tagged into any of X's tags. This approach has some limitations. Especially, it can't detect rpm changes after RHEL version moves or the image moves to a different repo (see https://issues.redhat.com/browse/ART-3965 for a real problem). I will give you an example to show how it fails to detect a change: - Image A enables a RHEL 8.4 repo. - Package `foo-1.0.0-1.el8_3.x86_64.rpm` is installed in image A. It has `rhel-8.3.0-z` tag in Brew. - A newer version `foo-1.1.0-1.el8_4.x86_64.rpm` is released and has `rhel-8.4.0-z` tag. - Doozer doesn't think `foo-1.0.0-1.el8_3.x86_64.rpm` is outdated because the newer version `foo-1.1.0-1.el8_4.x86_64.rpm` doesn't have `rhel-8.3.0-z` tag. To address this limitation, a new approach is proposed to actually query the content of all enabled repos and compare the installed versions with the latest versions in repos. I will call it "content based" approach. This content based approach has been proved in `gen-payload` command (invoked in `build-sync` job) with the merge of openshift-eng#764. If you look at [the logs of build-sync build](https://saml.buildvm.hosts.prod.psi.bos.redhat.com:8888/job/aos-cd-builds/job/build%252Fbuild-sync/34458/consoleFull), you will see it shows you which images contain outdated rpms. It contains entries like: ```yaml - code: OUTDATED_RPMS_IN_STREAM_BUILD msg: Found outdated RPM (perl-Pod-Perldoc-3.28-396.el8) installed in openshift-enterprise-builder-container-v4.14.0-202307111728.p0.gfe78ee6.assembly.stream (x86_64) when perl-Pod-Perldoc-3.28.01-443.module+el8.6.0+13324+628a2397 was available in repo rhel-8-appstream-rpms ``` However, the above example is actually a false positive. It is because package `perl-Pod-Perldoc-3.28-396.el8` comes from a modular repo but our code is not module-aware. A modular rpm repo provides multiple upgrade paths. We should only consider a newer version is upgradable if it belongs to the same stream. For more information about modules, see https://docs.fedoraproject.org/en-US/modularity/. That is a major reason why this new approach is adopted to `scan-sources`, otherwise it would constantly trigger unnecessary rebuilds! This PR not only updates `scan-sources` to use the new approach, but only adds module-awareness to eliminate false positives. The new algorithm (probably oversimplified): 1. For an image build, get all modular rpms in enabled repos. 2. For each installed rpm, check if the rpm is contained by a module. 3. If yes, we will consider that module stream is "enabled" for this image. 4. For each enabled module stream, find the module with the largest version. 5. Collect all modular rpms from those modules as collection M. 6. Collect all non-modular rpms with the latest version as collection N. If the package name of a non-modular rpm is already included in the modular rpms collection, exclude it. 7. For each installed rpm, check if it is a modular rpm or not. If it is modular, check if it is older than the one in collection M. Otherwise, check if it is older than the one in collection N. 8. If it is older, we consider is it an outdated rpm. Note: 1. There is no Brew API or any other cheap way to precisely determine which module streams are enabled during an image build. The method used in the above algorithm is actually *guess*. It is not perfect, but my tests conclude it is good enough for our use cases (probably because we don't use a lot of modular rpms in our images). 2. Some images use pins in their Dockerfiles. We should exempt them from scan-sources. The exemption can be defined in ocp-build-data image metadata. e.g. ```yaml scan_sources: exempted_packages: - openvswitch3.1 - python3-openvswitch3.1 ```
vfreex
added a commit
to vfreex/doozer
that referenced
this pull request
Jul 14, 2023
Currently we use a tag based approach to detect rpm changes in stream builds. It considers a package X is outdated if a newer version Y is tagged into any of X's tags. This approach has some limitations. Especially, it can't detect rpm changes after RHEL version moves or the image moves to a different repo (see https://issues.redhat.com/browse/ART-3965 for a real problem). I will give you an example to show how it fails to detect a change: - Image A enables a RHEL 8.4 repo. - Package `foo-1.0.0-1.el8_3.x86_64.rpm` is installed in image A. It has `rhel-8.3.0-z` tag in Brew. - A newer version `foo-1.1.0-1.el8_4.x86_64.rpm` is released and has `rhel-8.4.0-z` tag. - Doozer doesn't think `foo-1.0.0-1.el8_3.x86_64.rpm` is outdated because the newer version `foo-1.1.0-1.el8_4.x86_64.rpm` doesn't have `rhel-8.3.0-z` tag. To address this limitation, a new approach is proposed to actually query the content of all enabled repos and compare the installed versions with the latest versions in repos. I will call it "content based" approach. This content based approach has been proved in `gen-payload` command (invoked in `build-sync` job) with the merge of openshift-eng#764. If you look at [the logs of build-sync build](https://saml.buildvm.hosts.prod.psi.bos.redhat.com:8888/job/aos-cd-builds/job/build%252Fbuild-sync/34458/consoleFull), you will see it shows you which images contain outdated rpms. It contains entries like: ```yaml - code: OUTDATED_RPMS_IN_STREAM_BUILD msg: Found outdated RPM (perl-Pod-Perldoc-3.28-396.el8) installed in openshift-enterprise-builder-container-v4.14.0-202307111728.p0.gfe78ee6.assembly.stream (x86_64) when perl-Pod-Perldoc-3.28.01-443.module+el8.6.0+13324+628a2397 was available in repo rhel-8-appstream-rpms ``` However, the above example is actually a false positive. It is because package `perl-Pod-Perldoc-3.28-396.el8` comes from a modular repo but our code is not module-aware. A modular rpm repo provides multiple upgrade paths. We should only consider a newer version is upgradable if it belongs to the same stream. For more information about modules, see https://docs.fedoraproject.org/en-US/modularity/. That is a major reason why this new approach is adopted to `scan-sources`, otherwise it would constantly trigger unnecessary rebuilds! This PR not only updates `scan-sources` to use the new approach, but only adds module-awareness to eliminate false positives. The new algorithm (probably oversimplified): 1. For an image build, get all modular rpms in enabled repos. 2. For each installed rpm, check if the rpm is contained by a module. 3. If yes, we will consider that module stream is "enabled" for this image. 4. For each enabled module stream, find the module with the largest version. 5. Collect all modular rpms from those modules as collection M. 6. Collect all non-modular rpms with the latest version as collection N. If the package name of a non-modular rpm is already included in the modular rpms collection, exclude it. 7. For each installed rpm, check if it is a modular rpm or not. If it is modular, check if it is older than the one in collection M. Otherwise, check if it is older than the one in collection N. 8. If it is older, we consider is it an outdated rpm. Note: 1. There is no Brew API or any other cheap way to precisely determine which module streams are enabled during an image build. The method used in the above algorithm is actually *guess*. It is not perfect, but my tests conclude it is good enough for our use cases (probably because we don't use a lot of modular rpms in our images). 2. Some images use pins in their Dockerfiles. We should exempt them from scan-sources. The exemption can be defined in ocp-build-data image metadata. e.g. ```yaml scan_sources: exempted_packages: - openvswitch3.1 - python3-openvswitch3.1 ```
github-actions bot
pushed a commit
to ashwindasr/art-tools
that referenced
this pull request
Jul 21, 2023
Currently we use a tag based approach to detect rpm changes in stream builds. It considers a package X is outdated if a newer version Y is tagged into any of X's tags. This approach has some limitations. Especially, it can't detect rpm changes after RHEL version moves or the image moves to a different repo (see https://issues.redhat.com/browse/ART-3965 for a real problem). I will give you an example to show how it fails to detect a change: - Image A enables a RHEL 8.4 repo. - Package `foo-1.0.0-1.el8_3.x86_64.rpm` is installed in image A. It has `rhel-8.3.0-z` tag in Brew. - A newer version `foo-1.1.0-1.el8_4.x86_64.rpm` is released and has `rhel-8.4.0-z` tag. - Doozer doesn't think `foo-1.0.0-1.el8_3.x86_64.rpm` is outdated because the newer version `foo-1.1.0-1.el8_4.x86_64.rpm` doesn't have `rhel-8.3.0-z` tag. To address this limitation, a new approach is proposed to actually query the content of all enabled repos and compare the installed versions with the latest versions in repos. I will call it "content based" approach. This content based approach has been proved in `gen-payload` command (invoked in `build-sync` job) with the merge of openshift-eng/doozer#764. If you look at [the logs of build-sync build](https://saml.buildvm.hosts.prod.psi.bos.redhat.com:8888/job/aos-cd-builds/job/build%252Fbuild-sync/34458/consoleFull), you will see it shows you which images contain outdated rpms. It contains entries like: ```yaml - code: OUTDATED_RPMS_IN_STREAM_BUILD msg: Found outdated RPM (perl-Pod-Perldoc-3.28-396.el8) installed in openshift-enterprise-builder-container-v4.14.0-202307111728.p0.gfe78ee6.assembly.stream (x86_64) when perl-Pod-Perldoc-3.28.01-443.module+el8.6.0+13324+628a2397 was available in repo rhel-8-appstream-rpms ``` However, the above example is actually a false positive. It is because package `perl-Pod-Perldoc-3.28-396.el8` comes from a modular repo but our code is not module-aware. A modular rpm repo provides multiple upgrade paths. We should only consider a newer version is upgradable if it belongs to the same stream. For more information about modules, see https://docs.fedoraproject.org/en-US/modularity/. That is a major reason why this new approach is adopted to `scan-sources`, otherwise it would constantly trigger unnecessary rebuilds! This PR not only updates `scan-sources` to use the new approach, but only adds module-awareness to eliminate false positives. The new algorithm (probably oversimplified): 1. For an image build, get all modular rpms in enabled repos. 2. For each installed rpm, check if the rpm is contained by a module. 3. If yes, we will consider that module stream is "enabled" for this image. 4. For each enabled module stream, find the module with the largest version. 5. Collect all modular rpms from those modules as collection M. 6. Collect all non-modular rpms with the latest version as collection N. If the package name of a non-modular rpm is already included in the modular rpms collection, exclude it. 7. For each installed rpm, check if it is a modular rpm or not. If it is modular, check if it is older than the one in collection M. Otherwise, check if it is older than the one in collection N. 8. If it is older, we consider is it an outdated rpm. Note: 1. There is no Brew API or any other cheap way to precisely determine which module streams are enabled during an image build. The method used in the above algorithm is actually *guess*. It is not perfect, but my tests conclude it is good enough for our use cases (probably because we don't use a lot of modular rpms in our images). 2. Some images use pins in their Dockerfiles. We should exempt them from scan-sources. The exemption can be defined in ocp-build-data image metadata. e.g. ```yaml scan_sources: exempted_packages: - openvswitch3.1 - python3-openvswitch3.1 ```
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Currently we detect outdated rpms in images and RHCOS by comparing the NVRs of the installed rpms to the NVRs of the rpms in our
rhaos-x.y-rhel-z-candidateBrew tag.The current approach has 2 issues. See ART-4558 and ART-3965.
With this new approach,
repoquerycommand will be used to query available RPMs in the configured build repos instead of the candidate Brew tag. This will provide more accurate detection and also be able to detect rpms that are not in rhaos repos (e.g. RHEL repos).Note this PR only covers the
gen-payloadchange. A change forscan-sourceswill be another PR.