Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Contact forces are still zero in Ant-v2 #2593

Closed
JamesKCS opened this issue Feb 4, 2022 · 13 comments
Closed

Contact forces are still zero in Ant-v2 #2593

JamesKCS opened this issue Feb 4, 2022 · 13 comments
Labels
mujoco

Comments

@JamesKCS
Copy link

JamesKCS commented Feb 4, 2022 •
edited

This is the same as #1636 and #1541.

For Ant-v2, contact forces (all observation dimensions after the first 27) seem to be 0. To check this, I ran 10 million time steps with random action selection, and never got a non-zero value for those 84 observation dimensions in that run.

The (closed) issues above seem to suggest downgrading to MuJoCo<2.0 as a solution. However, as I understand it, MuJoCo<2.0 was before the open-sourcing of MuJoCo, and so requires purchasing a license, and, even then, doesn't play well with cluster computing (please correct me if I am wrong).

Even if MuJoCo<2.0 does not require a license, leaving the issues above as closed because downgrading to <2.0 fixes them does not seem correct, since https://github.com/openai/mujoco-py instructs users to use mujoco-py 2.1 and mujoco 2.1.0. I think that perhaps those issues should be reopened.

My versions:
gym: v0.17.1
MuJoCo: v2.1.0
Python: 3.7.2
mujoco-py: 2.1.2.14

Apologies if I am making a trivial error; I am new to MuJoCo. I also apologize if this issue should be posted on mujoco-py instead; I decided to post it here since the above issues were posted here. Thank you in advance for the assistance!
@ikostrikov
Copy link

@JamesKCS could you check whether they are still zero in this fork?

https://github.com/rodrigodelazcano/gym

@JamesKCS
Copy link
Author

JamesKCS commented Feb 4, 2022

@ikostrikov Thank you; I'm away from my primary computer and won't be able to check until Tuesday or Wednesday next week, but will check and report back then. Thank you again.

@jkterry1 jkterry1 added the mujoco label Feb 5, 2022
@JamesKCS
Copy link
Author

JamesKCS commented Feb 8, 2022 •
edited

@ikostrikov Thank you again for your help. I tried duplicating my conda environment, and downloading the fork, and overwriting the gym version with the manually downloaded fork:

~/anaconda3/envs/myenv_temp_version/bin/pip install ~/Downloads/gym-master

I also tried making a new conda environment from scratch, installing gym with the command

~/anaconda3/envs/temp_change_check_env/bin/pip install ~/Downloads/gym-master

In both cases, on the line

env = gym.make('Ant-v2')

I get an error about ModuleNotFoundError: No module named 'dm_control'

  File "main.py", line 2, in <module>
    env = gym.make('Ant-v2')
  File "/home/james/anaconda3/envs/temp_change_check_env/lib/python3.7/site-packages/gym/envs/registration.py", line 592, in make
    return registry.make(id, **kwargs)
  File "/home/james/anaconda3/envs/temp_change_check_env/lib/python3.7/site-packages/gym/envs/registration.py", line 495, in make
    return spec.make(**kwargs)
  File "/home/james/anaconda3/envs/temp_change_check_env/lib/python3.7/site-packages/gym/envs/registration.py", line 114, in make
    cls = load(self.entry_point)
  File "/home/james/anaconda3/envs/temp_change_check_env/lib/python3.7/site-packages/gym/envs/registration.py", line 30, in load
    mod = importlib.import_module(mod_name)
  File "/home/james/anaconda3/envs/temp_change_check_env/lib/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/james/anaconda3/envs/temp_change_check_env/lib/python3.7/site-packages/gym/envs/mujoco/__init__.py", line 1, in <module>
    from gym.envs.mujoco.mujoco_env import MujocoEnv
  File "/home/james/anaconda3/envs/temp_change_check_env/lib/python3.7/site-packages/gym/envs/mujoco/mujoco_env.py", line 7, in <module>
    from gym.envs.mujoco.viewer import WindowViewer
  File "/home/james/anaconda3/envs/temp_change_check_env/lib/python3.7/site-packages/gym/envs/mujoco/viewer.py", line 6, in <module>
    import dm_control.mujoco as dm_mujoco
ModuleNotFoundError: No module named 'dm_control'

Please let me know how I can correctly install your fork to check your fix. Thank you!

@ikostrikov
Copy link

The fork depends on dm_control instead of mujoco_py.

Could you try to install it with

pip install git+git://github.com/rodrigodelazcano/gym#egg=gym[mujoco]

?

@JamesKCS
Copy link
Author

JamesKCS commented Feb 8, 2022

Thank you. I tried that, and got the error

/home/james/anaconda3/envs/myclone/lib/python3.7/site-packages/gym/envs/registration.py:481: UserWarning: WARN: The environment Ant-v2 is out of date. You should consider upgrading to version `v3` with the environment ID `Ant-v3`.
  f"The environment {path} is out of date. You should consider "
WARNING:absl:mjbindings failed to import mjlib and other functions. libmujoco.so may not be accessible.
Traceback (most recent call last):
  File "main.py", line 2, in <module>
    env = gym.make('Ant-v2')
  File "/home/james/anaconda3/envs/myclone/lib/python3.7/site-packages/gym/envs/registration.py", line 592, in make
    return registry.make(id, **kwargs)
  File "/home/james/anaconda3/envs/myclone/lib/python3.7/site-packages/gym/envs/registration.py", line 495, in make
    return spec.make(**kwargs)
  File "/home/james/anaconda3/envs/myclone/lib/python3.7/site-packages/gym/envs/registration.py", line 114, in make
    cls = load(self.entry_point)
  File "/home/james/anaconda3/envs/myclone/lib/python3.7/site-packages/gym/envs/registration.py", line 30, in load
    mod = importlib.import_module(mod_name)
  File "/home/james/anaconda3/envs/myclone/lib/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/james/anaconda3/envs/myclone/lib/python3.7/site-packages/gym/envs/mujoco/__init__.py", line 1, in <module>
    from gym.envs.mujoco.mujoco_env import MujocoEnv
  File "/home/james/anaconda3/envs/myclone/lib/python3.7/site-packages/gym/envs/mujoco/mujoco_env.py", line 7, in <module>
    from gym.envs.mujoco.viewer import WindowViewer
  File "/home/james/anaconda3/envs/myclone/lib/python3.7/site-packages/gym/envs/mujoco/viewer.py", line 6, in <module>
    import dm_control.mujoco as dm_mujoco
  File "/home/james/anaconda3/envs/myclone/lib/python3.7/site-packages/dm_control/mujoco/__init__.py", line 18, in <module>
    from dm_control.mujoco.engine import action_spec
  File "/home/james/anaconda3/envs/myclone/lib/python3.7/site-packages/dm_control/mujoco/engine.py", line 42, in <module>
    from dm_control.mujoco import index
  File "/home/james/anaconda3/envs/myclone/lib/python3.7/site-packages/dm_control/mujoco/index.py", line 88, in <module>
    from dm_control.mujoco.wrapper import util
  File "/home/james/anaconda3/envs/myclone/lib/python3.7/site-packages/dm_control/mujoco/wrapper/__init__.py", line 20, in <module>
    from dm_control.mujoco.wrapper.core import callback_context
  File "/home/james/anaconda3/envs/myclone/lib/python3.7/site-packages/dm_control/mujoco/wrapper/core.py", line 29, in <module>
    from dm_control.mujoco.wrapper.mjbindings import functions
  File "/home/james/anaconda3/envs/myclone/lib/python3.7/site-packages/dm_control/mujoco/wrapper/mjbindings/functions.py", line 17, in <module>
    mjlib = util.get_mjlib()
  File "/home/james/anaconda3/envs/myclone/lib/python3.7/site-packages/dm_control/mujoco/wrapper/util.py", line 120, in get_mjlib
    raise OSError("Cannot find MuJoCo library at {}.".format(library_path))
OSError: Cannot find MuJoCo library at /home/james/.mujoco/lib/libmujoco.so.2.1.1.

My MuJoCo directory does not have this structure (no lib directory near the top level):

ls ~/.mujoco/mujoco210/
bin/                 include/             model/               sample/              THIRD_PARTY_NOTICES

Are there alternative instructions for installing MuJoCo I should follow to make this work? Thank you again very much for your help.

@ikostrikov
Copy link

You need to install mujoco 2.1.1:

https://mujoco.org/download

@JamesKCS
Copy link
Author

JamesKCS commented Feb 8, 2022 •
edited

Thank you, the install seems to work now, but this fork does not seem to fix the original problem. I ran 100k random actions this time; the code and results are below (just in case I'm making a trivial error). I also tried Ant-v3 (for 10k actions) as suggested by the warning below, and got a similar result. Thank you again for your help and time.

import gym
env = gym.make('Ant-v2')
from copy import copy
observation = env.reset()
initial_vals_set = False
import numpy as np

for _ in range(100000):
    action = env.action_space.sample() # your agent here (this takes random actions)
    observation, reward, done, info = env.step(action)
    if not initial_vals_set:
        the_min = copy(observation)
        the_max = copy(observation)
        initial_vals_set = True
    else:
        the_min = np.minimum(the_min, observation)
        the_max = np.maximum(the_max, observation)

    if done:
        observation = env.reset()

print("min:")
print(the_min)

print("\n\nmax:")
print(the_max)
env.close()

/home/james/anaconda3/envs/myclone/lib/python3.7/site-packages/gym/envs/registration.py:481: UserWarning: WARN: The environment Ant-v2 is out of date. You should consider upgrading to version `v3` with the environment ID `Ant-v3`.
  f"The environment {path} is out of date. You should consider "
min:
[  0.23676215  -0.99998241  -0.99997345  -0.99991652  -0.999996
  -0.67696312   0.35651666  -0.67609774  -1.36959719  -0.67511285
  -1.36332004  -0.67291766   0.35584749  -3.46077718  -3.96622676
  -3.66692864  -8.82351436  -8.4316648   -8.63963981 -16.59423101
 -14.12172282 -16.41878187 -18.06531397 -16.56702041 -17.9456844
 -16.5034511  -14.30085571   0.           0.           0.
   0.           0.           0.           0.           0.
   0.           0.           0.           0.           0.
   0.           0.           0.           0.           0.
   0.           0.           0.           0.           0.
   0.           0.           0.           0.           0.
   0.           0.           0.           0.           0.
   0.           0.           0.           0.           0.
   0.           0.           0.           0.           0.
   0.           0.           0.           0.           0.
   0.           0.           0.           0.           0.
   0.           0.           0.           0.           0.
   0.           0.           0.           0.           0.
   0.           0.           0.           0.           0.
   0.           0.           0.           0.           0.
   0.           0.           0.           0.           0.
   0.           0.           0.           0.           0.
   0.        ]


max:
[ 1.18575309  0.99999604  0.99969254  0.99974872  0.99991619  0.68067865
  1.36156789  0.68048161 -0.35548828  0.68097272 -0.35868979  0.67882319
  1.34712235  4.1930571   3.47957626  6.19362933 11.93152865  9.16725658
  9.01330906 16.80663473 18.6547767  16.54984733 13.9673883  16.45289604
 13.93382616 15.98880362 17.89721724  0.          0.          0.
  0.          0.          0.          0.          0.          0.
  0.          0.          0.          0.          0.          0.
  0.          0.          0.          0.          0.          0.
  0.          0.          0.          0.          0.          0.
  0.          0.          0.          0.          0.          0.
  0.          0.          0.          0.          0.          0.
  0.          0.          0.          0.          0.          0.
  0.          0.          0.          0.          0.          0.
  0.          0.          0.          0.          0.          0.
  0.          0.          0.          0.          0.          0.
  0.          0.          0.          0.          0.          0.
  0.          0.          0.          0.          0.          0.
  0.          0.          0.          0.          0.          0.
  0.          0.          0.        ]

Edit: accidentally pasted the output from Ant-v3, fixed it by replacing it with the output from v2.

@ikostrikov
Copy link

I can reproduce it on my machine. I will have time to take a closer look tomorrow.

@ikostrikov
Copy link

@JamesKCS it seems that contact forces are not displayed properly in the main branch while the latest pypi release is correct:

pip install git+git://github.com/openai/gym#egg=gym[mujoco]

has all zeros even though it uses an old version of mujoco_py

pip install gym[mujoco]

works correctly. Could you verify it?

@rodrigodelazcano could you take a look? We need to resolve this issue before mujoco_py is replaced with dm_control.

@JamesKCS
Copy link
Author

JamesKCS commented Feb 9, 2022

@ikostrikov Thank you for looking into this.

pip install git+git://github.com/openai/gym#egg=gym[mujoco]

fails because it wants an old version of MuJoCo:

You appear to be missing MuJoCo. We expected to find the file here: /home/james/.mujoco/mjpro150

I need a license to use pre-2.0 MuJoCo, correct? So I don't think I can verify; please correct me if I'm wrong or am making an error. Thank you again for your help.

@ikostrikov
Copy link

@JamesKCS they provide a free license now for the older versions of mujoco:
https://roboti.us/license.html

@PBarde PBarde mentioned this issue Feb 11, 2022
Open
@JamesKCS
Copy link
Author

@ikostrikov That seems to work! The outputs of the program above are below (1 mil timesteps). Would you recommend using MuJoCo 1.5 for the time being, until this is fixed? Thank you.

min:
[  0.23113901  -0.99997165  -0.99997876  -0.99998188  -0.99995824
  -0.68331792   0.35274529  -0.68552905  -1.36111614  -0.68188489
  -1.37895513  -0.68424382   0.35249764  -4.12479723  -4.09526031
  -3.7662062  -13.20352795 -13.19956766  -9.3412916  -16.9794841
 -14.419281   -16.72894085 -18.68670642 -16.92535395 -18.65970163
 -16.84481169 -14.39286872   0.           0.           0.
   0.           0.           0.          -1.          -1.
  -1.          -1.          -1.           0.           0.
   0.           0.           0.           0.           0.
  -1.          -1.          -1.          -1.          -1.
   0.          -1.          -1.          -1.          -1.
  -1.           0.           0.           0.           0.
   0.           0.           0.          -1.          -1.
  -1.          -1.          -1.           0.          -1.
  -1.          -1.          -1.          -1.           0.
   0.           0.           0.           0.           0.
   0.          -1.          -1.          -1.          -1.
  -1.           0.          -1.          -1.          -1.
  -1.          -1.           0.           0.           0.
   0.           0.           0.           0.          -1.
  -1.          -1.          -1.          -1.           0.
  -1.          -1.          -1.          -1.          -1.
   0.        ]


max:
[ 1.22715073  0.99999702  0.99994359  0.99998798  0.99998988  0.68037107
  1.37800402  0.68391051 -0.35200692  0.68483242 -0.35273681  0.68102172
  1.3723244   4.37358389  5.24040536  6.32937827 10.15779831 10.67948283
  9.84644744 17.02361686 19.11335975 16.95297147 14.65629154 16.68166261
 14.44694742 17.1225593  18.66107739  0.          0.          0.
  0.          0.          0.          1.          1.          1.
  1.          1.          1.          0.          0.          0.
  0.          0.          0.          1.          1.          1.
  1.          1.          1.          1.          1.          1.
  1.          1.          1.          0.          0.          0.
  0.          0.          0.          1.          1.          1.
  1.          1.          1.          1.          1.          1.
  1.          1.          1.          0.          0.          0.
  0.          0.          0.          1.          1.          1.
  1.          1.          1.          1.          1.          1.
  1.          1.          1.          0.          0.          0.
  0.          0.          0.          1.          1.          1.
  1.          1.          1.          1.          1.          1.
  1.          1.          1.        ]

@jkterry1
Copy link
Collaborator

Closed by #2595

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
mujoco
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@ikostrikov @jkterry1 @JamesKCS