Feature request: upgrade from gym to gymnasium

[elliottower]

Hi, would it be possible to upgrade this repository to use gymnasium rather than gym? We would like to list this project in this list of third party environments, but are only adding environments which use gymnasium.

Gymnasium a maintained fork of openai gym and is designed as a drop-in replacement (import gym -> import gymnasium as gym). Beyond just bugfixes, many RL training libraries have also switched (rllib, tianshou, CleanRL), or are planning to switch (stable-baselines3). It would be great if users could train agents using the latest models and features from these libraries (e.g., scalable distributed training/model serving using Ray/RLlib).

For information about upgrading and compatibility, see migration guide and gym compatibility. The main difference is the API has switched to returning truncated and terminated, rather than done, in order to give more information and mitigate edge case issues (for example, many popular tutorials/implementations of Q learning using gym were actually incorrect because of done, there will be an upcoming blog post explaining more details about this on the Farama site (https://farama.org/blog).