Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failing to install scenario on MacOS #422

Open
4 tasks done
lrapetti opened this issue Nov 13, 2021 · 12 comments
Open
4 tasks done

Failing to install scenario on MacOS #422

lrapetti opened this issue Nov 13, 2021 · 12 comments

Comments

@lrapetti
Copy link
Member

lrapetti commented Nov 13, 2021

Description:

I noticed that macOS is not officially supported. I thought it was good to report anyway.

Steps to reproduce

I tried to install it with

pip install scenario

Output can be found below

Additional context

Output
$pip install scenario
Collecting scenario
  Downloading scenario-1.3.1.tar.gz (151 kB)
     |████████████████████████████████| 151 kB 352 kB/s 
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Collecting packaging
  Using cached packaging-21.2-py3-none-any.whl (40 kB)
Collecting pyparsing<3,>=2.0.2
  Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Building wheels for collected packages: scenario
  Building wheel for scenario (PEP 517) ... error
  ERROR: Command errored out with exit status 1:
   command: /Users/lorenzorapetti/Software/dic-iit/ADAM/adamEnv/bin/python3.9 /Users/lorenzorapetti/Software/dic-iit/ADAM/adamEnv/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py build_wheel /var/folders/v4/8y04pr2912dd1n1pryz743r80000gp/T/tmpb33hqy26
       cwd: /private/var/folders/v4/8y04pr2912dd1n1pryz743r80000gp/T/pip-install-d6fi17h8/scenario_93d92748aa99469aba71337237b49d10
  Complete output (80 lines):
  running bdist_wheel
  running build
  running build_ext
  -- The C compiler identification is AppleClang 12.0.5.12050022
  -- The CXX compiler identification is AppleClang 12.0.5.12050022
  -- Detecting C compiler ABI info
  -- Detecting C compiler ABI info - done
  -- Check for working C compiler: /usr/local/opt/ccache/libexec/cc - skipped
  -- Detecting C compile features
  -- Detecting C compile features - done
  -- Detecting CXX compiler ABI info
  -- Detecting CXX compiler ABI info - done
  -- Check for working CXX compiler: /usr/local/opt/ccache/libexec/c++ - skipped
  -- Detecting CXX compile features
  -- Detecting CXX compile features - done
  -- Looking for pthread.h
  -- Looking for pthread.h - found
  -- Performing Test CMAKE_HAVE_LIBC_PTHREAD
  -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
  -- Found Threads: TRUE
  -- Found Python3: /Users/lorenzorapetti/Software/dic-iit/ADAM/adamEnv/bin/python3.9 (found version "3.9.6") found components: Interpreter Development Development.Module Development.Embed
  -- Using Python: /Users/lorenzorapetti/Software/dic-iit/ADAM/adamEnv/bin/python3.9
  CMake Error at CMakeLists.txt:205 (message):
    Found no Ignition distribution for PyPI package

-- Configuring incomplete, errors occurred!
See also "/private/var/folders/v4/8y04pr2912dd1n1pryz743r80000gp/T/pip-install-d6fi17h8/scenario_93d92748aa99469aba71337237b49d10/build/temp.macosx-11-x86_64-3.9_ScenarioCMakeProject/CMakeFiles/CMakeOutput.log".

==> Configuring:
$ cmake -S /private/var/folders/v4/8y04pr2912dd1n1pryz743r80000gp/T/pip-install-d6fi17h8/scenario_93d92748aa99469aba71337237b49d10 -B /private/var/folders/v4/8y04pr2912dd1n1pryz743r80000gp/T/pip-install-d6fi17h8/scenario_93d92748aa99469aba71337237b49d10/build/temp.macosx-11-x86_64-3.9_ScenarioCMakeProject -GNinja -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX:PATH=/private/var/folders/v4/8y04pr2912dd1n1pryz743r80000gp/T/pip-install-d6fi17h8/scenario_93d92748aa99469aba71337237b49d10/build/lib.macosx-11-x86_64-3.9/scenario -DSCENARIO_CALL_FROM_SETUP_PY:BOOL=ON -DSCENARIO_BUILD_SHARED_LIBRARY:BOOL=OFF -DPython3_EXECUTABLE:PATH=/Users/lorenzorapetti/Software/dic-iit/ADAM/adamEnv/bin/python3.9

==> Building:
$ cmake --build /private/var/folders/v4/8y04pr2912dd1n1pryz743r80000gp/T/pip-install-d6fi17h8/scenario_93d92748aa99469aba71337237b49d10/build/temp.macosx-11-x86_64-3.9_ScenarioCMakeProject --config Release

==> Installing:
$ cmake --install /private/var/folders/v4/8y04pr2912dd1n1pryz743r80000gp/T/pip-install-d6fi17h8/scenario_93d92748aa99469aba71337237b49d10/build/temp.macosx-11-x86_64-3.9_ScenarioCMakeProject

Traceback (most recent call last):
File "/Users/lorenzorapetti/Software/dic-iit/ADAM/adamEnv/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 280, in
main()
File "/Users/lorenzorapetti/Software/dic-iit/ADAM/adamEnv/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 263, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
File "/Users/lorenzorapetti/Software/dic-iit/ADAM/adamEnv/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 204, in build_wheel
return _build_backend().build_wheel(wheel_directory, config_settings,
File "/private/var/folders/v4/8y04pr2912dd1n1pryz743r80000gp/T/pip-build-env-r6jc4hrg/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 230, in build_wheel
return self._build_with_temp_dir(['bdist_wheel'], '.whl',
File "/private/var/folders/v4/8y04pr2912dd1n1pryz743r80000gp/T/pip-build-env-r6jc4hrg/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 215, in _build_with_temp_dir
self.run_setup()
File "/private/var/folders/v4/8y04pr2912dd1n1pryz743r80000gp/T/pip-build-env-r6jc4hrg/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 158, in run_setup
exec(compile(code, file, 'exec'), locals())
File "setup.py", line 10, in
setuptools.setup(
File "/private/var/folders/v4/8y04pr2912dd1n1pryz743r80000gp/T/pip-build-env-r6jc4hrg/overlay/lib/python3.9/site-packages/setuptools/init.py", line 159, in setup
return distutils.core.setup(**attrs)
File "/usr/local/Cellar/python@3.9/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/core.py", line 148, in setup
dist.run_commands()
File "/usr/local/Cellar/python@3.9/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/dist.py", line 966, in run_commands
self.run_command(cmd)
File "/usr/local/Cellar/python@3.9/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/private/var/folders/v4/8y04pr2912dd1n1pryz743r80000gp/T/pip-build-env-r6jc4hrg/overlay/lib/python3.9/site-packages/wheel/bdist_wheel.py", line 299, in run
self.run_command('build')
File "/usr/local/Cellar/python@3.9/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/usr/local/Cellar/python@3.9/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/usr/local/Cellar/python@3.9/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/command/build.py", line 135, in run
self.run_command(cmd_name)
File "/usr/local/Cellar/python@3.9/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/usr/local/Cellar/python@3.9/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/private/var/folders/v4/8y04pr2912dd1n1pryz743r80000gp/T/pip-build-env-r6jc4hrg/overlay/lib/python3.9/site-packages/cmake_build_extension/build_extension.py", line 110, in run
self.build_extension(ext)
File "/private/var/folders/v4/8y04pr2912dd1n1pryz743r80000gp/T/pip-build-env-r6jc4hrg/overlay/lib/python3.9/site-packages/cmake_build_extension/build_extension.py", line 224, in build_extension
subprocess.check_call(configure_command)
File "/usr/local/Cellar/python@3.9/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/subprocess.py", line 373, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '-S', '/private/var/folders/v4/8y04pr2912dd1n1pryz743r80000gp/T/pip-install-d6fi17h8/scenario_93d92748aa99469aba71337237b49d10', '-B', '/private/var/folders/v4/8y04pr2912dd1n1pryz743r80000gp/T/pip-install-d6fi17h8/scenario_93d92748aa99469aba71337237b49d10/build/temp.macosx-11-x86_64-3.9_ScenarioCMakeProject', '-GNinja', '-DCMAKE_BUILD_TYPE=Release', '-DCMAKE_INSTALL_PREFIX:PATH=/private/var/folders/v4/8y04pr2912dd1n1pryz743r80000gp/T/pip-install-d6fi17h8/scenario_93d92748aa99469aba71337237b49d10/build/lib.macosx-11-x86_64-3.9/scenario', '-DSCENARIO_CALL_FROM_SETUP_PY:BOOL=ON', '-DSCENARIO_BUILD_SHARED_LIBRARY:BOOL=OFF', '-DPython3_EXECUTABLE:PATH=/Users/lorenzorapetti/Software/dic-iit/ADAM/adamEnv/bin/python3.9']' returned non-zero exit status 1.

ERROR: Failed building wheel for scenario
Failed to build scenario
ERROR: Could not build wheels for scenario which use PEP 517 and cannot be installed directly

Environment

  • OS: MacOS 11.3.1
  • Python: Python3.9.6
@diegoferigo
Copy link
Member

Hi @lrapetti, yes as you read from the support policy, macOS is not currently supported. If you inspect the CMake output, you'll notice that it fails with the following error:

  CMake Error at CMakeLists.txt:205 (message):
    Found no Ignition distribution for PyPI package

You should install Ignition Gazebo manually before attempting to install scenario. Open Robotics currently supports macOS even though I cannot guarantee that scenario will work ootb. If you are willing do iterate on possible errors that might emerge, I can provide support.

I actually already did some work on macOS in #406 while attempting to finalize conda-forge/staged-recipes#16582, but the last update was conda-forge/staged-recipes#16582 (comment) in which everything was building correctly on macOS (a nice start), but tests were segfaulting.

@diegoferigo diegoferigo added complexity::medium help wanted Extra attention is needed labels Nov 14, 2021
@diegoferigo
Copy link
Member

diegoferigo commented Nov 14, 2021

I just remembered that we had a previous discussion regarding macOS in #100, not sure if things have changes since.

@lrapetti
Copy link
Member Author

Open Robotics currently supports macOS even though I cannot guarantee that scenario will work ootb.

Following the ignition gazebo installation, I was able to install both scenario and gym-ignition via pip without errors.

@lrapetti
Copy link
Member Author

lrapetti commented Nov 15, 2021

Still, I am experiencing the command line tool mentioned in #100 (comment). It looks like the problem is still present in the ignition version I have installed.

I was able to run some gym-ignition code such as gym_ignition_models, but not those using gazebo such as the examples:

$python3 launch_cartpole.py
Traceback (most recent call last):
  File "/Users/lorenzorapetti/Software/dic-iit/ADAM/adamEnv/lib/python3.9/site-packages/scenario/__init__.py", line 141, in check_gazebo_installation
    result = subprocess.run(command, capture_output=True, text=True, check=True)
  File "/usr/local/Cellar/python@3.9/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/subprocess.py", line 528, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['ign', 'gazebo', '--versions']' returned non-zero exit status 255.

@diegoferigo
Copy link
Member

What fails here is just a check for a compatible Ignition Gazebo version, it's not strictly needed during runtime.

Can you please report if the following command executed from a normal terminal succeeds, and if it does, its output?

ign gazebo --versions

If it does not succeed, you can try commenting out the following line in /Users/lorenzorapetti/Software/dic-iit/ADAM/adamEnv/lib/python3.9/site-packages/scenario/__init__.py:

check_gazebo_installation()

@lrapetti
Copy link
Member Author

lrapetti commented Nov 15, 2021

Can you please report if the following command executed from a normal terminal succeeds, and if it does, its output?

ign gazebo --versions

Since there is the problem with the command line tool, I am getting the following

ign gazebo --versions
I cannot find any available 'ign' command:
	* Did you install any ignition library?
	* Did you set the IGN_CONFIG_PATH environment variable?
	    E.g.: export IGN_CONFIG_PATH=$HOME/local/share/ignition

So I commted out the lines as you where suggesting, but I am getting the following

 $python3 launch_cartpole.py
Traceback (most recent call last):
  File "/Users/lorenzorapetti/Software/dic-iit/gym-ignition/examples/python/launch_cartpole.py", line 9, in <module>
    from gym_ignition.utils import logger
  File "/Users/lorenzorapetti/Software/dic-iit/ADAM/adamEnv/lib/python3.9/site-packages/gym_ignition/__init__.py", line 14, in <module>
    from gym_ignition.utils import resource_finder
  File "/Users/lorenzorapetti/Software/dic-iit/ADAM/adamEnv/lib/python3.9/site-packages/gym_ignition/utils/__init__.py", line 5, in <module>
    from . import logger, math, misc, resource_finder, scenario
  File "/Users/lorenzorapetti/Software/dic-iit/ADAM/adamEnv/lib/python3.9/site-packages/gym_ignition/utils/math.py", line 10, in <module>
    from scenario import gazebo as scenario
  File "/Users/lorenzorapetti/Software/dic-iit/ADAM/adamEnv/lib/python3.9/site-packages/scenario/__init__.py", line 228, in <module>
    import_gazebo()
  File "/Users/lorenzorapetti/Software/dic-iit/ADAM/adamEnv/lib/python3.9/site-packages/scenario/__init__.py", line 203, in import_gazebo
    import scenario.bindings.gazebo
  File "/Users/lorenzorapetti/Software/dic-iit/ADAM/adamEnv/lib/python3.9/site-packages/scenario/bindings/gazebo.py", line 13, in <module>
    from . import _gazebo
ImportError: dlopen(/Users/lorenzorapetti/Software/dic-iit/ADAM/adamEnv/lib/python3.9/site-packages/scenario/bindings/_gazebo.so, 10): Library not loaded: /usr/local/opt/urdfdom/lib/liburdfdom_sensor.3.0.dylib
  Referenced from: /usr/local/opt/sdformat12/lib/libsdformat12.12.dylib
  Reason: image not found

I guess that is due to some conflict or wrong configuration on my laptop. Indeed I see the following

$ ls /usr/local/opt/urdfdom/lib
liburdfdom_model.1.0.dylib		liburdfdom_sensor.dylib
liburdfdom_model.dylib			liburdfdom_world.1.0.dylib
liburdfdom_model_state.1.0.dylib	liburdfdom_world.dylib
liburdfdom_model_state.dylib		pkgconfig
liburdfdom_sensor.1.0.dylib		urdfdom

@diegoferigo
Copy link
Member

I see that you have liburdfdom_sensor.1.0.dylib while sdformat12 requires liburdfdom_sensor.3.0.dylib. Could it be a problem of the packages you already had installed in your system before installing Ignition Gazebo? Can you try to update the library?

Another option is to try using a conda environment instead of relying on the official macOS packages. Though, even with conda, something could be missing.

@srmainwaring
Copy link

Not sure if this is the right place for reporting experimental environments, but I have had success in getting ScenarIO running on macOS (with GUI) using development branches that add Metal support to Ignition (which will be slowly making there way into released versions - hopefully in time for Ignition Garden). I'd be happy to outline the approach here or somewhere suitable if there is an interest. This looks like a great tool btw! I expect to be using it a good deal.

@diegoferigo
Copy link
Member

Hi @srmainwaring, this is great news, thanks! I'm not sure if @lrapetti has any news after his last attempts, in any case it would be very helpful if you can describe your working setup and whether there's anything missing on our side to run on macOS.

@lrapetti
Copy link
Member Author

I'm not sure if @lrapetti has any news after his last attempts

Sorry, I didn't proceed further at this stage. In case I'll let you know when I start again this investigation.

@srmainwaring
Copy link

srmainwaring commented Nov 30, 2021

Hi @diegoferigo - here's the description, it may take a number of edits to get right as it is fairly involved at this stage. I hope it will simplify as more of the development work is merged into the relevant repos.

Install Ignition on macOS

The first step is to get the Ignition graphics API running properly on macOS. I've documented how to install Ignition with Metal rendering on macOS here: https://github.com/srmainwaring/ign-rendering/wiki. Over time this page should become redundant as more features are merged into the https://github.com/ignitionrobotics repos.

Install iDynTree

The installation on macOS largely follows the iDynTree install instructions with an extra step to install to /usr/local/Cellar/idyntree/1.0.0 and use brew to link into /usr/local.

Clone and install:

# clone to ~/robotology
$ cd ~/robotology
$ git clone https://github.com/robotology/idyntree.git
$ cd idyntree
$ mkdir build && cd build
$ cmake .. \
  -DCMAKE_INSTALL_PREFIX=/usr/local/Cellar/idyntree/1.0.0 \
  -DCMAKE_BUILD_TYPE=RelWithDebInfo \
  -DIDYNTREE_USES_PYTHON=ON
$ make -j16
$ make install
$ brew link idyntree

Check the installation:

>>> import idyntree
>>> import idyntree.bindings as iDynTree
>>> p = iDynTree.Position()
>>> print(p)
1.46135e+224 1.0947e+248 1.02265e-259

Run the example:

$ cd ~/robotology/idyntree/examples/python
$ python KinDynComputationsTutorial.py 

Install ScenarIO

We use a patched version of ScenarIO which includes dependency definitions for Ignition Garden. The libraries are installed to /usr/local/Cellar/scenario/1.0.0 and then brew linked into /usr/local:

# clone to ~/robotology
$ cd ~/robotology
$ git clone https://github.com/robotology/gym-ignition.git
$ git remote add fork https://github.com/srmainwaring/gym-ignition.git
$ git checkout fork/feature/ign-garden-macos
$ cd gym-ignition/scenario
$ mkdir build && cd build
$ cmake .. \
  -DCMAKE_INSTALL_PREFIX=/usr/local/Cellar/scenario/1.0.0 \
  -DCMAKE_BUILD_TYPE=RelWithDebInfo \
  -DSCENARIO_USE_IGNITION=ON \
  -DSCENARIO_ENABLE_BINDINGS=ON
$ make -j16
$ make install
$ brew link scenario
$ cd ~/robotology/gym-ignition
$ pip install --user -e scenario
$ pip install --user -e .

I have not used a Python virtual environment for my installation, but that should also work.

Configuration

ScenarIO uses a custom physics plugin that overrides the defaults in World.cpp

bool World::setPhysicsEngine(const PhysicsEngine engine)
{
// Get the name of the physics plugin
const std::string pluginLib = [&engine]() -> std::string {
switch (engine) {
case PhysicsEngine::Dart:
return "ignition-physics"
+ std::to_string(IGNITION_PHYSICS_MAJOR_VERSION)
+ "-dartsim-plugin";
}
return "";
}();
if (pluginLib.empty()) {
sError << "Failed to retrieve the name of physics plugin library";
return false;
}
// This component is read by the Physics system during its configuration
utils::setComponentData<ignition::gazebo::components::PhysicsEnginePlugin>(
m_ecm, m_entity, pluginLib);
// Vendored Physics system
const std::string libName = "PhysicsSystem";
const std::string className = "scenario::plugins::gazebo::Physics";
// Load the Physics system
if (!this->insertWorldPlugin(libName, className)) {
sError << "Failed to insert the physics plugin" << std::endl;
return false;
}
return true;
}

It expects to find the system plugin libPhysicsSystem.dylib so we need to add the install location to IGN_GAZEBO_PLUGIN_INSTALL_DIR when running C++ scenarios:

export IGN_GAZEBO_PLUGIN_INSTALL_DIR=${IGN_GAZEBO_PLUGIN_INSTALL_DIR}:/usr/local/lib/scenario/plugins

Testing

Run the pendulum Python example from VSCode. Set up a launch configuration entry in ~/robotology/.vscode/launch.json:

{
  "version": "0.2.0",
  "configurations": [
    {
      "name": "Python: ScenarIO",
      "type": "python",
      "request": "launch",
      "program": "${file}",
      "console": "integratedTerminal",
      "env": {
        "IGN_VERBOSE": "0",
        "IGN_CONFIG_PATH": "/usr/local/share/ignition",
        "IGN_RENDERING_RESOURCE_PATH": "/usr/local/share/ignition/ignition-rendering7",
      },
    },
  ]
}

Add the following lines to the pendulum example to ensure that ignition uses the brew installed version of ruby:

# ensure we are using /usr/local/opt/ruby/bin/ruby instead of the system version
import os
PATH = os.getenv("PATH")
PATH = "/usr/local/opt/ruby/bin:" + PATH 
os.environ["PATH"] = PATH

Then run the example:

scenario_pendulum_vscode

scenario_pendulum_gazebo

@diegoferigo
Copy link
Member

Wow @srmainwaring great work, nicely done! Thanks for this big effort to smooth the macOS experience. I don't have a macOS machine to test everything myself, but I will eagerly wait for your contributions to land upstream and then giving another shot in the conda-forge packaging I drafted in conda-forge/staged-recipes#16582 (unfortunately I'm a bit busy lately and I have to postpone it a little longer). As you noted, from the ScenarIO point of view, changes are minimal, only the introduction of Garden support was missing. I can surely add it to the nightly channel so that your setup could get rid of a fork, and those that are interested in building the project from source can benefit from it! And, actually, note to myself, it would a good idea in the future adding a similar CMake support of the development Ignition distribution right after the previous upstream stable version is released.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants