Feature Request: Gymnasium Compatibility

[elliottower]

Hi, would it be possible to allow this repo to be used with Gymnasium environments? Certain libraries like MineRL are not compatible with Gymnasium yet but it would be great to allow environments to be of either type, so it could be used the wide range of new libraries that are based on Gymnasium.

For background, Gymnasium a maintained fork of openai gym and is designed as a drop-in replacement (import gym -> import gymnasium as gym). For context, many popular RL training libraries have switched (rllib, tianshou, CleanRL, stable-baselines3), along with many environment repositories (see third party environments). We are also considering compiling a list of repositories for popular training libraries and models implementations which can be used with Gymnasium, so this could potentially be listed on the website.

For information about upgrading and compatibility, see migration guide and gym compatibility. The main difference is the API has switched to returning truncated and terminated, rather than done, in order to give more information and mitigate edge case issues (for example, many popular tutorials/implementations of Q learning using gym were actually incorrect because of done, there will be an upcoming blog post explaining more details about this on the Farama site (https://farama.org/blog)).

[qxcv]

Wrapper: https://gist.github.com/qxcv/e8641342c102c2aa714c9caeca724101
It's just from_gym.py with a few minor changes to support gymnasium.
Tested to work with Minigrid.

(edit: actually to support minigrid you also need to fix a bug in nets.py: stages = int(np.log2(self._shape[-2]) - np.log2(self._minres)) should be stages = int(np.ceil(np.log2(self._shape[-2]) - np.log2(self._minres))) or else it will error out on image resolutions that aren't a power of 2)

(edit^2: actually there's also a bug with the way that padding is computed below that; x = x[:, int(np.ceil(padh)):int(padh), :]; x = x[:, :, int(np.ceil(padw)):int(padw)] should be x = x[:, int(np.ceil(padh)):x.shape[1]-int(padh), :]; x = x[:, :, int(np.ceil(padw)):x.shape[2]-int(padw)] to deal with the case when padh or padw round to 0)

[jareducherek]

@qxcv Could you provide your example.py changes as well? I am having some trouble when trying out other gymnasium environments.

[qxcv]

I didn't touch example.py. Instead I used train.py in dreamerv3/, which is configured via configs.yaml (I'm using dreamer as a baseline and didn't want to rewrite all this config stuff myself).

Maybe try my fork? https://github.com/qxcv/dreamerv3

Specifically:

• Check out at 33ea375 (see changes w/ git diff HEAD^^^)
• Install the deps (GPU Jax first, then requirements.txt)
• Run python -m dreamerv3 --configs minigrid --logdir ./logs/minigrid --batch_size 2

If it turns out more changes are needed for other Gymnasium envs then one of us should probably file a PR 🙂