pip_package not working on OSX

[kplimack]

looks like the -mpip is not being inserted

           ================================================================================
           Error executing action `install` on resource 'python_package[assumeAWSRole]'
           ================================================================================

           Mixlib::ShellOut::ShellCommandFailed
           ------------------------------------
           Expected process to exit with [0], but received '1'
           ---- Begin output of /usr/bin/python -  mpip install https://pypi.myorg.net/root/myorg/+simple/ --trusted-host pypi.myorg.net  assumeawsrole ----
           STDOUT:
           STDERR: No handlers could be found for logger "pip.index"
           Traceback (most recent call last):
             File "<stdin>", line 38, in <module>
             File "/Library/Python/2.7/site-packages/pip/index.py", line 514, in find_requirement
        'No matching distribution found for %s' % req
           pip.exceptions.DistributionNotFound: No matching distribution found for mpip
           ---- End output of /usr/bin/python -  mpip install https://pypi.myorg.net/root/myorg/+simple/ --trusted-host pypi.myorg.net  assumeawsrole ----
           Ran /usr/bin/python - https://pypi.myorg.net/root/myorg/+simple/ --trusted-host pypi.myorg.net  assumeawsrole returned 1

           Cookbook Trace:
           ---------------
           /tmp/kitchen/cache/cookbooks/poise-languages/files/halite_gem/poise_languages/command/mixin.rb:207:in `tap'
           /tmp/kitchen/cache/cookbooks/poise-languages/files/halite_gem/poise_languages/command/mixin.rb:207:in `language_command_shell_out!'
           /tmp/kitchen/cache/cookbooks/poise-languages/files/halite_gem/poise_languages/command/mixin.rb:222:in `block in language_command_mixin'
           /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:317:in `pip_command'
           /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:341:in `pip_outdated'
           /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:209:in `check_package_versions'
           /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:189:in `load_current_resource'

           Resource Declaration:
           ---------------------
           # In /tmp/kitchen/cache/cookbooks/myorg_aws/recipes/assume_role.rb

            15: python_package 'assumeAWSRole' do
            16:   options 'https://pypi.myorg.net/root/derps/+simple/ --trusted-host pypi.myorg.net'
            17: end

[coderanger]

What version of the cookbook are you using? That should be a pretty different command in the current rev.

[coderanger]

Also whats with the URL in the options string?

[kplimack]

@coderanger using version 1.6.0. is there a better way to supply a source ?

python_package 'assumeAWSRole' do
  options 'https://pypi.myorg.net/root/myorg/+simple/ --trusted-host pypi.myorg.net'
end

[coderanger]

The runner entry point in 1.6 is -m pip.__main__, so you should see that in the command string (even if broken). How sure are you that you are on 1.6.0?

[kplimack]

@coderanger

default-macosx-1011:~ vagrant$ sudo cat /tmp/kitchen/cache/cookbooks/poise-python/metadata.json | jq .version
"1.6.0"

[coderanger]

Also the stack trace shows it is failing inside pip_outdated but the command being run is an install. That's very odd. Any chance of some old monkey-patches or something?

[kplimack]

nope. this is a fresh cookbook on a vanilla osx VM. this is the entire recipe:

case node['platform_family']
when 'mac_os_x'
  include_recipe 'homebrew'
  package 'openconnect'
end

include_recipe 'poise-python'

python_package 'awsassumerole' do
  options '-i https://pypi.myorg.net/root/myorg/+simple/'
end

[coderanger]

That is more certainly not the recipe code as shown in the error. Please give the recipe content and debug error message for a single point in time. When you are changing both throughout the debugging process, it makes it yet harder to know whats going on.

[kplimack]

@coderanger sorry, i had tried supplying the -i option for pypi index url, but got the same result. heres the original recipe and output.

recipe

case node['platform_family']
when 'mac_os_x'
  include_recipe 'homebrew'
  package 'openconnect'
end

include_recipe 'poise-python'

python_package 'assumeAWSRole' do
  options 'https://pypi.myorg.net/root/myorg/+simple/ --trusted-host pypi.myorg.net'
end

output

       Recipe: myorg_aws::assume_role
         * python_package[assumeAWSRole] action install

           ================================================================================
           Error executing action `install` on resource 'python_package[assumeAWSRole]'
           ================================================================================

           Mixlib::ShellOut::ShellCommandFailed
           ------------------------------------
           Expected process to exit with [0], but received '1'
           ---- Begin output of /usr/bin/python -  https://pypi.myorg.net/root/myorg/+simple/ --trusted-host pypi.myorg.net  assumeawsrole ----
           STDOUT:
           STDERR: Traceback (most recent call last):
             File "<stdin>", line 38, in <module>
             File "/Library/Python/2.7/site-packages/pip/index.py", line 465, in find_requirement
        all_candidates = self.find_all_candidates(req.name)
             File "/Library/Python/2.7/site-packages/pip/index.py", line 383, in find_all_candidates
        index_locations = self._get_index_urls_locations(project_name)
             File "/Library/Python/2.7/site-packages/pip/index.py", line 373, in _get_index_urls_locations
        return [mkurl_pypi_url(url) for url in self.index_urls]
             File "/Library/Python/2.7/site-packages/pip/index.py", line 363, in mkurl_pypi_url
        urllib_parse.quote(canonicalize_name(project_name)))
             File "/Library/Python/2.7/site-packages/pip/_vendor/packaging/utils.py", line 14, in canonicalize_name
        return _canonicalize_regex.sub("-", name).lower()
           TypeError: expected string or buffer
           ---- End output of /usr/bin/python -  https://pypi.myorg.net/root/myorg/+simple/ --trusted-host pypi.myorg.net  assumeawsrole ----
           Ran /usr/bin/python -  https://pypi.myorg.net/root/myorg/+simple/ --trusted-host pypi.myorg.net  assumeawsrole returned 1

           Cookbook Trace:
           ---------------
           /tmp/kitchen/cache/cookbooks/poise-languages/files/halite_gem/poise_languages/command/mixin.rb:207:in `tap'
           /tmp/kitchen/cache/cookbooks/poise-languages/files/halite_gem/poise_languages/command/mixin.rb:207:in `language_command_shell_out!'
           /tmp/kitchen/cache/cookbooks/poise-languages/files/halite_gem/poise_languages/command/mixin.rb:222:in `block in language_command_mixin'
           /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:317:in `pip_command'
           /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:341:in `pip_outdated'
           /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:209:in `check_package_versions'
           /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:189:in `load_current_resource'

           Resource Declaration:
           ---------------------
           # In /tmp/kitchen/cache/cookbooks/myorg_aws/recipes/assume_role.rb

            15: python_package 'assumeAWSRole' do
            16:   options 'https://pypi.myorg.net/root/myorg/+simple/ --trusted-host pypi.myorg.net'
            17: end

           Compiled Resource:
           ------------------
           # Declared in /tmp/kitchen/cache/cookbooks/myorg_aws/recipes/assume_role.rb:15:in `from_file'

           python_package("assumeAWSRole") do
             package_name "assumeAWSRole"
             action [:install]
             retries 0
             retry_delay 2
             default_guard_interpreter :default
             declared_type :python_package
             cookbook_name "myorg_aws"
             recipe_name "assume_role"
             options "https://pypi.myorg.net/root/myorg/+simple/ --trusted-host pypi.myorg.net"
             parent_python python_runtime[2]
             timeout 900
           end

           Platform:
           ---------
           x86_64-darwin13

[coderanger]

Does it work for a public package without the options? Also what version of Chef?

[kplimack]

trying to install a public pacakge w/ chef-client 12.17.44

         * python_package[Django] action install

           ================================================================================
           Error executing action `install` on resource 'python_package[Django]'
           ================================================================================

           Mixlib::ShellOut::ShellCommandFailed
           ------------------------------------
           Expected process to exit with [0], but received '1'
           ---- Begin output of ["/usr/bin/python", "-m", "pip.__main__", "install", "Django==2.0"] ----
           STDOUT: Collecting Django==2.0
             Downloading Django-2.0.tar.gz (8.0MB)
        Complete output from command python setup.py egg_info:
        Traceback (most recent call last):
          File "<string>", line 1, in <module>
          File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/pip-build-GcSZfc/Django/setup.py", line 32, in <module>
            version = __import__('django').get_version()
          File "django/__init__.py", line 1, in <module>
            from django.utils.version import get_version
          File "django/utils/version.py", line 61, in <module>
            @functools.lru_cache()
        AttributeError: 'module' object has no attribute 'lru_cache'

        ----------------------------------------
           STDERR: Command "python setup.py egg_info" failed with error code 1 in /private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/pip-build-GcSZfc/Django/
           ---- End output of ["/usr/bin/python", "-m", "pip.__main__", "install", "Django==2.0"] ----
           Ran ["/usr/bin/python", "-m", "pip.__main__", "install", "Django==2.0"] returned 1

           Cookbook Trace:
           ---------------
           /tmp/kitchen/cache/cookbooks/poise-languages/files/halite_gem/poise_languages/command/mixin.rb:207:in `tap'
           /tmp/kitchen/cache/cookbooks/poise-languages/files/halite_gem/poise_languages/command/mixin.rb:207:in `language_command_shell_out!'
           /tmp/kitchen/cache/cookbooks/poise-languages/files/halite_gem/poise_languages/command/mixin.rb:222:in `block in language_command_mixin'
           /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:317:in `pip_command'
           /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:331:in `pip_install'
           /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:238:in `install_package'

           Resource Declaration:
           ---------------------
           # In /tmp/kitchen/cache/cookbooks/myorg_aws/recipes/assume_role.rb

            15: python_package 'Django'
            16:

           Compiled Resource:
           ------------------
           # Declared in /tmp/kitchen/cache/cookbooks/myorg_aws/recipes/assume_role.rb:15:in `from_file'

           python_package("Django") do
             package_name "Django"
             action [:install]
             retries 0
             retry_delay 2
             default_guard_interpreter :default
             declared_type :python_package
             cookbook_name "myorg_aws"
             recipe_name "assume_role"
             parent_python python_runtime[2]
             timeout 900
           end

           Platform:
           ---------
           x86_64-darwin13

[coderanger]

Okay, so at least the right command there. It's using the system python on macOS, might want to check what version that is. I don't specifically support macOS so you may need to override some paths if you want to use Python from Homebrew.