Test conda installation #279
Assignees
Comments
|
I am giving conda a try. I did the following in a plain
After these steps, I hoped that the tensorflow problems related to protobuf would be solved. Sadly, they are not. If tensorflow is imported after scenario, tensorflow will segfault. segfaultThe problem is the At this point, I tried to avoid altering the dlopen flags when loading I think at this point the best workaround that fixes both conda and non-conda environments is detecting if tensorflow is installed and, if found, open the problematic shared library, similarly to apache/arrow#2210. This should be done by cc @traversaro |
Strange, conda-forge should contain ogre version that works fine with ignition-rendering, see conda-forge/staged-recipes#13677 .
How did you install |
That is what I thought. Maybe there was something strange in my setup (even though it is a plain docker image), but I got compilation errors. I didn't save the log, my fault, but I remember some failed checks to
I installed the Now that you make me think about it, I originally thought that this |
Interesting. I think that the tensorflow from the defaults channel is build from the recipes in https://github.com/AnacondaRecipes/tensorflow_recipes . I think it is probably building against the defaults channel protobuf, built from https://github.com/AnacondaRecipes/protobuf-feedstock . However, to double check it would be interesting to understand which packages are actually installed in your environment with |
|
I didn't reach yet any conclusion, let's see when I find some more time to continue this experiment. Here below the current list of packages. I have some cos6 and cos7 duplicates, but I guess I'm using the cos7. mamba listIn any case, so far this was a kind of successful experiment. Despite the few hacks, I managed to compile:
I also need ray and rllib, however I don't think it would be a problem. In earlier experiments I tried to install it from conda-forge together with tensorflow from conda's default channel and they seemed working fine. |
|
I'm trying to prepare a docker image containing what I've got so far, but now I'm stuck with the following error. Comparing the conda packages inside the docker image and what posted in the comment above did not show anything relevant. ignition-rendering4 compilation errorThis is happening with ogre from conda-forge. I tried compiling ogre from source and install into the conda environment as I did last time but I get the same error. |
|
Weird, it seems that commenting out the include of #ifndef GLX_GLXEXT_LEGACY
#include <GL/glxext.h> /* This is line 333 */
#endif /* GLX_GLXEXT_LEGACY */What I don't yet understand is what could have defined |
|
Related discussion in the ignition-rendering recipe PR : conda-forge/staged-recipes#13677 (comment) . |
|
Thanks for the link, the problem is the same. I didn't get if a solution was found. In the meantime, I paste here the Dockerfile I got. I'll commit it somewhere, not sure yet where. DockerfileARG from=nvidia/cuda:11.1.1-cudnn8-runtime-ubuntu20.04
FROM ${from}
# Change default shell to bash. This is effective only in the Dockerfile.
SHELL ["/bin/bash", "-i", "-c"]
# Setup locales and timezone
ARG TZ=Europe/Rome
ARG DEBIAN_FRONTEND=noninteractive
RUN rm -f /etc/localtime &&\
ln -s /usr/share/zoneinfo/"${TZ}" /etc/localtime &&\
apt-get update &&\
apt-get install -y --no-install-recommends locales locales-all tzdata &&\
rm -rf /var/lib/apt/lists/*
# Utilities
RUN apt-get update &&\
apt-get install -y --no-install-recommends \
software-properties-common \
apt-transport-https \
apt-utils \
wget \
nano \
bash-completion \
gnupg2 \
colordiff \
curl \
zip \
unzip \
lsof \
net-tools \
iputils-ping \
strace \
less \
&&\
rm -rf /var/lib/apt/lists/*
ENV CONDA_PREFIX=/conda
ARG CONDA_PYTHON_VERSION=3.8
ENV MAMBA_ROOT_PREFIX=$CONDA_PREFIX/.mamba
# Install micromamba and create conda environment
RUN cd /usr/local &&\
wget -qO- https://micromamba.snakepit.net/api/micromamba/linux-64/latest \
| tar -xvj bin/micromamba &&\
eval "$(micromamba shell hook -s bash)" &&\
micromamba create -y -p $CONDA_PREFIX python==$CONDA_PYTHON_VERSION mamba -c conda-forge &&\
micromamba activate $CONDA_PREFIX &&\
conda config --system --add channels conda-forge &&\
conda config --system --set channel_priority strict
# Enable by default the conda environment for non-root users
RUN echo 'function activate_conda() {' >> /etc/bash.bashrc &&\
echo ' eval "$(micromamba shell hook -s bash)"' >> /etc/bash.bashrc &&\
echo ' micromamba activate $CONDA_PREFIX' >> /etc/bash.bashrc &&\
#echo ' export CMAKE_PREFIX_PATH=$CONDA_PREFIX:$CMAKE_PREFIX_PATH' >> /etc/bash.bashrc &&\
echo '}' >> /etc/bash.bashrc &&\
echo '[[ $(id -u) -gt 0 ]] && activate_conda' >> /etc/bash.bashrc
# Install buildchain
RUN activate_conda &&\
mamba install -y \
compilers cmake make pkg-config ninja pybind11 git
# Default directory with sources
ARG SRC_DIR=/usr/local/src
# Install dartsim
RUN activate_conda &&\
mamba install -y \
eigen assimp libccd boost openscenegraph nlopt ipopt bullet libode octomap flann tinyxml2 \
urdfdom xorg-libxi xorg-libxmu freeglut fcl &&\
cd $SRC_DIR &&\
git clone https://github.com/diegoferigo/dart &&\
mkdir -p dart/build && cd dart/build &&\
cmake .. \
-GNinja \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_INSTALL_PREFIX=$CONDA_PREFIX \
-DHAVE_BULLET:BOOL=ON \
-DHAVE_DOXYGEN:BOOL=OFF \
-DHAVE_ODE:BOOL=ON \
&&\
ninja install
# Ignition Robotics
RUN activate_conda &&\
mamba install -y \
vcstool colcon-common-extensions eigen freeimage gts glib ffmpeg \
ruby tinyxml2 tinyxml protobuf urdfdom zeromq cppzmq \
ogre \
jsoncpp libzip qt \
mesa-libgl-devel-cos7-x86_64 libglu
#RUN activate_conda &&\
# mamba install -y libxrandr-devel-cos7-x86_64 xorg-libxrandr &&\
# cd $SRC_DIR &&\
# git clone --depth=1 https://github.com/OGRECave/ogre &&\
# mkdir -p ogre/build && cd ogre/build &&\
# cmake .. \
# -GNinja \
# -DCMAKE_BUILD_TYPE=Release \
# -DCMAKE_INSTALL_PREFIX=$CONDA_PREFIX \
# -DOGRE_BUILD_COMPONENT_CSHARP=OFF \
# -DOGRE_BUILD_COMPONENT_PYTHON=OFF \
# &&\
# ninja install
RUN activate_conda &&\
cd $SRC_DIR &&\
mkdir -p workspace/src && cd workspace/src &&\
wget -q https://raw.githubusercontent.com/ignition-tooling/gazebodistro/master/collection-dome.yaml &&\
vcs import < collection-dome.yaml &&\
colcon graph
RUN activate_conda &&\
sed -i "s|#include <GL/glxext.h>|//#include <GL/glxext.h>|g" /conda/x86_64-conda-linux-gnu/sysroot/usr/include/GL/glx.h &&\
cd $SRC_DIR/workspace &&\
colcon build \
--merge-install \
--cmake-args \
-GNinja \
-DBUILD_TESTING:BOOL=OFF \
-DCMAKE_BUILD_TYPE=Release \
&&\
echo "[[ -f $SRC_DIR/workspace/install/setup.sh ]] && source $SRC_DIR/workspace/install/setup.sh" >> /etc/bash.bashrc
ENV IGN_GAZEBO_PHYSICS_ENGINE_PATH=$SRC_DIR/workspace/install/lib/ign-physics-3/engine-plugins
# Superbuild
RUN activate_conda &&\
mamba install -y \
ace \
asio \
boost \
eigen \
glew \
glfw \
gsl \
ipopt \
libjpeg-turbo \
libmatio \
libode \
libxml2 \
opencv \
pkg-config \
portaudio \
qt \
sdl \
sdl2 \
sqlite \
tinyxml \
jsoncpp \
&&\
mamba install -y \
expat-cos7-x86_64 \
freeglut \
libselinux-cos7-x86_64 \
libxau-cos7-x86_64 \
libxcb-cos7-x86_64 \
libxdamage-cos7-x86_64 \
libxext-cos7-x86_64 \
libxfixes-cos7-x86_64 \
libxxf86vm-cos7-x86_64 \
mesalib \
mesa-libgl-cos7-x86_64
RUN activate_conda &&\
cd $SRC_DIR &&\
git clone https://github.com/robotology/robotology-superbuild &&\
mkdir -p robotology-superbuild/build && cd robotology-superbuild/build &&\
cmake \
-GNinja \
-DCMAKE_BUILD_TYPE=Release \
-DROBOTOLOGY_ENABLE_CORE:BOOL=ON \
-DROBOTOLOGY_ENABLE_DYNAMICS:BOOL=ON \
-DROBOTOLOGY_ENABLE_DYNAMICS_FULL_DEPS:BOOL=ON \
-DROBOTOLOGY_ENABLE_ROBOT_TESTING:BOOL=OFF \
-DROBOTOLOGY_ENABLE_HUMAN_DYNAMICS:BOOL=OFF \
-DROBOTOLOGY_USES_GAZEBO:BOOL=OFF \
-DROBOTOLOGY_USES_PYTHON:BOOL=ON \
-DROBOTOLOGY_USES_OCTAVE:BOOL=OFF \
-DROBOTOLOGY_USES_MATLAB:BOOL=OFF \
-DNON_INTERACTIVE_BUILD:BOOL=ON \
-DBUILD_TESTING:BOOL=OFF \
-DYCM_DISABLE_SYSTEM_PACKAGES:BOOL=ON \
-DROBOTOLOGY_PROJECT_TAGS=Stable \
-DYCM_EP_ADDITIONAL_CMAKE_ARGS="-DWITH_LAPACK=ON -DWITH_MUMPS=ON -DWITH_OSQP=ON -DWITH_PYTHON=ON -DWITH_PYTHON3=ON -DPYTHON_PREFIX=$SRC_DIR/robotology-superbuild/build/install/lib/python3/site-packages -DWITH_QPOASES=ON" \
.. &&\
ninja &&\
sed -i 's/_object/object/g' $SRC_DIR/robotology-superbuild/build/install/lib/python3/site-packages/casadi/casadi.py &&\
echo 'export PYTHONPATH=$PYTHONPATH:/usr/local/src/robotology-superbuild/build/install/lib/python3.8/site-packages/' >> /etc/bash.bashrc &&\
echo "source $SRC_DIR/robotology-superbuild/build/install/share/robotology-superbuild/setup.sh" >> /etc/bash.bashrc
RUN activate_conda &&\
mamba install -y \
ipython \
tensorflow-gpu \
scipy \
numpy \
gym \
lxml \
matplotlib \
setuptools_scm \
ray-rllib ray-tune ray-dashboard
RUN activate_conda &&\
cd $SRC_DIR &&\
git clone https://github.com/robotology/gym-ignition &&\
mkdir gym-ignition/build && cd gym-ignition/build &&\
cmake .. \
-GNinja \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_INSTALL_PREFIX=$CONDA_PREFIX \
&&\
ninja install |
No, it was just passed the |
|
I managed to reproduce the ogre compilation error I reported above in #279 (comment): This error occurs when The only alternative I didn't yet check is compiling and installing ogre 1.9.1 from sources. This should match the same configuration with ogre1 of the ignition packages provided in the osrf's ppa. ogre compilation error |
Ok, this make sensei as that ogre is the one used in the ignition-rendering4 official conda build.
Did you tried to add |
I checked a few of the compilation errors, and they seems all to be at points where there is an ifdef for compatibility between 1.9 and 1.12 , such as https://github.com/ignitionrobotics/ign-rendering/blob/ign-rendering4/ogre/src/OgreMaterialSwitcher.cc#L82 . Probably the ifdef logic needs to be change to correctly account for Ogre 1.10.12 . |
|
I guess this discussion could be relevant/interesting for @JShep1 @wolfv @Tobias-Fischer . |
Yep. However as reported above, this ogre version has runtime errors when opening Ignition Gazebo's GUI.
Thanks for the hint, I just tried to compile again ign-rendering4 from colcon against conda-forge's The first line is Ignition Rendering 4 that complains about the
I might try to alter the compiler definition by hand, let's see if I manage to get something out of this. |
The related CMake code is https://github.com/ignitionrobotics/ign-rendering/blob/ign-rendering4/ogre/src/CMakeLists.txt#L18 . |
|
Hi @diegoferigo, could you please report on the precise version of ogre that is installed in your conda environment? We fixed the bug about the missing |
No, actually I think I misunderstood the role of |
For this test I was using
Interestingly, using sed -i "s|if(OGRE_VERSION VERSION_LESS 1.10.3)|if(OGRE_VERSION VERSION_LESS 1.11.0)|g" src/ign-rendering/ogre/src/CMakeLists.txt
sed -i "s|if(OGRE_VERSION VERSION_LESS 1.10.1)|if(OGRE_VERSION VERSION_LESS 1.11.0)|g" src/ign-rendering/ogre/src/CMakeLists.txtignition gui
|
diegoferigo
added
issue::status::in-progress
complexity::hard
and removed
complexity::medium
labels
|
It seems that the patch was applied also to the conda-forge ign-rendering recipe conda-forge/libignition-rendering4-feedstock#8. |
1 task
6 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Using conda and particularly the
conda-forgechannel could introduce many benefits to our project:About 1), for the sake of testing, manually compiling ign-gazebo with colcon and using the conda environment for the dependencies should suffice.
On conda-forge there are already many of the ign-gazebo dependencies, but few of them are still missing. As soon as the entire ignition distribution will be included in the channel, we'll be able to bypass the manual installation of ign-gazebo from Open Robotics' PPA.
Edit: some preliminary conda instructions have been recently added upstream: https://github.com/ignitionrobotics/docs/blob/master/citadel/install_windows_src.md. The process is for windows, but it can used also on other OSs.
The text was updated successfully, but these errors were encountered: