pkg.installed should support kwargs for setting --setopt

[djsly]

Description of Issue/Question

We need to enable the --setopt=obsoletes=0 option for yum to allow installing Docker using Salt

moby/moby#33930

Setup

docker-ce_pkgs:
  pkg.installed:
      - pkgs:
          - docker-ce-selinux: 17.03.2.ce-1.el7.centos
          - docker-ce: 17.03.2.ce-1.el7.centos
      - hold: true
      - update_holds: true
      - kwargs:
          setopt: obsoletes=0

Steps to Reproduce Issue

[INFO    ] Executing command ['systemd-run', '--scope', 'yum', '-y', 'install', 'docker-ce-17.03.2.ce-1.el7.centos', 'docker-ce-selinux-17.03.2.ce-1.el7.centos'] in directory '/root'
[ERROR   ] Command '['systemd-run', '--scope', 'yum', '-y', 'install', 'docker-ce-17.03.2.ce-1.el7.centos', 'docker-ce-selinux-17.03.2.ce-1.el7.centos']' failed with return code: 1
[ERROR   ] stdout: Running scope as unit run-1126.scope.
Loaded plugins: fastestmirror, versionlock
Loading mirror speeds from cached hostfile
 * epel: mirror.csclub.uwaterloo.ca
Package docker-ce-selinux is obsoleted by docker-ce, trying to install docker-ce-17.12.0.ce-1.el7.centos.x86_64 instead
Resolving Dependencies
--> Running transaction check
---> Package docker-ce.x86_64 0:17.03.2.ce-1.el7.centos will be installed
--> Processing Dependency: docker-ce-selinux >= 17.03.2.ce-1.el7.centos for package: docker-ce-17.03.2.ce-1.el7.centos.x86_64
Package docker-ce-selinux is obsoleted by docker-ce, but obsoleting package does not provide for requirements
---> Package docker-ce.x86_64 0:17.12.0.ce-1.el7.centos will be installed
--> Processing Dependency: container-selinux >= 2.9 for package: docker-ce-17.12.0.ce-1.el7.centos.x86_64
--> Running transaction check
---> Package container-selinux.noarch 2:2.36-1.gitff95335.el7 will be installed
---> Package docker-ce.x86_64 0:17.03.2.ce-1.el7.centos will be installed
--> Processing Dependency: docker-ce-selinux >= 17.03.2.ce-1.el7.centos for package: docker-ce-17.03.2.ce-1.el7.centos.x86_64
Package docker-ce-selinux is obsoleted by docker-ce, but obsoleting package does not provide for requirements
--> Finished Dependency Resolution
Error: Package: docker-ce-17.03.2.ce-1.el7.centos.x86_64 (official_docker_io_engine_repo)
           Requires: docker-ce-selinux >= 17.03.2.ce-1.el7.centos
           Available: docker-ce-selinux-17.03.0.ce-1.el7.centos.noarch (official_docker_io_engine_repo)
               docker-ce-selinux = 17.03.0.ce-1.el7.centos
           Available: docker-ce-selinux-17.03.1.ce-1.el7.centos.noarch (official_docker_io_engine_repo)
               docker-ce-selinux = 17.03.1.ce-1.el7.centos
           Available: docker-ce-selinux-17.03.2.ce-1.el7.centos.noarch (official_docker_io_engine_repo)
               docker-ce-selinux = 17.03.2.ce-1.el7.centos
 You could try using --skip-broken to work around the problem
 You could try running: rpm -Va --nofiles --nodigest

Versions Report

Salt Version:
           Salt: 2017.7.3
 
Dependency Versions:
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: Not Installed
      docker-py: 1.10.6
          gitdb: 0.6.4
      gitpython: 1.0.1
          ioflo: Not Installed
         Jinja2: 2.7.2
        libgit2: Not Installed
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.5.1
   mysql-python: Not Installed
      pycparser: Not Installed
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: Not Installed
         Python: 2.7.5 (default, Aug  4 2017, 00:39:18)
   python-gnupg: Not Installed
         PyYAML: 3.10
          PyZMQ: 15.3.0
           RAET: Not Installed
          smmap: 0.9.0
        timelib: Not Installed
        Tornado: 4.2.1
            ZMQ: 4.1.4
 
System Versions:
           dist: centos 7.4.1708 Core
         locale: UTF-8
        machine: x86_64
        release: 3.10.0-693.11.6.el7.x86_64
         system: Linux
        version: CentOS Linux 7.4.1708 Core

[djsly]

The yumpkg module seems to have a function for parsing the extra args

https://github.com/saltstack/salt/blob/develop/salt/modules/yumpkg.py#L277-L300

Unfortunately, this helper function isn't used for pkg.installed only pkg.update which is linked to the pkg.uptodate state.

Maybe it could be added here ?
https://github.com/saltstack/salt/blob/develop/salt/modules/yumpkg.py#L1348

[garethgreenaway]

@terminalmage Thoughts on this being added to installed as well as update?

[terminalmage]

It looks like setopt is something that can be set more than once, right?

[terminalmage]

Yes, I confirmed that's the case. I've opened #46263

[terminalmage]

The usage here would be:

docker-ce_pkgs:
  pkg.installed:
      - pkgs:
          - docker-ce-selinux: 17.03.2.ce-1.el7.centos
          - docker-ce: 17.03.2.ce-1.el7.centos
      - hold: true
      - update_holds: true
      - setopt:
        - obsoletes=0

[thedebugger]

running into the same issue. Is there a way i can stil pass obsoletes=0 in old salt versions? if no, Is using yum directly is my only option in old salt version?

[djsly]

that's what we ended up doing

disable_obsolete_yum_conf:
    file.line:
        - name: /etc/yum.conf
        - mode: replace
        - match: obsoletes
        - content: obsoletes=0
        - prereq:
            - pkg: docker-ce_pkgs

docker-ce_pkgs:
  pkg.installed:
      - pkgs:
          - docker-ce-selinux: {{ docker.version }}
          - docker-ce: {{ docker.version }}
      - hold: true
      - update_holds: true

enable_obsolete_yum_conf:
    file.line:
        - name: /etc/yum.conf
        - mode: replace
        - match: obsoletes
        - content: obsoletes=1
        - onchanges:
            - file: disable_obsolete_yum_conf

[rallytime]

The docs on dynamic module distribution are helpful here. :)

[djsly]

maybe I'm missing the link, could you elaborate @rallytime ? thanks!

[terminalmage]

By copying a patched salt/modules/yumpkg.py to salt://_modules/ in your Salt fileserver, you can then use saltutil.sync_modules to distribute it to minions. It will supersede the yumpkg.py that Salt distributes.

You just need to remember to remove this file from salt://_modules/ and re-run the sync once the Fluorine release of Salt is released (likely this fall).

[djsly]

Awesome, thanks for the explaination!

[terminalmage]

No problem! Keep in mind though that the patch may not apply cleanly to earlier Salt releases, and users are on their own when it comes to using this method to distribute patched salt modules to minions. This method of distirbuting modules is intended primarily for distributing custom salt states/modules/etc. that users write for their own internal usage. It just has the added benefit of superseding existing modules, which makes it handy for these sort of use cases.

[djsly]

understood, a override switch that can break :)

…

On Fri, Mar 16, 2018 at 10:19 AM, Erik Johnson ***@***.***> wrote: No problem! Keep in mind though that the patch may not apply cleanly to earlier Salt releases, and users are on their own when it comes to using this method to distribute patched salt modules to minions. This method of distirbuting modules is intended primarily for distributing custom salt states/modules/etc. that users write for their own internal usage. It just has the added benefit of superseding existing modules, which makes it handy for these sort of use cases. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#46212 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AEwEKt6k5w5KrT7TN2lo_BSMAZn1oARZks5te8oDgaJpZM4SVBIr> .

[terminalmage]

Indeed 😄

[ysagon]

Hi, is this feature implemented in version 3004? Nothing in the documentation and doesn't seems to work, or at least no effect if the option is present.

pcp_pmda_infiniband_pkg:
  pkg.installed:
    - pkgs:
      - pcp-pmda-infiniband
    - setopt:
      - obsoletes=0

[terminalmage]

Yes, this pull request is in 3004.

[ysagon]

Thanks for the confirmation. But still, it doesn't seems to work. I tried to use -l debug while calling salt-call but I'm not able to see the exact yum command executed. Is there something wrong in my example?

edit: my bad, it seems the issue is with the package mlnx-ofed-all trying to replace pcp-pmda-infiniband. Sorry for the noise.