Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

check_env warning for FrameStacked observation in stable_baselines3.common.env_checker #1500

Open
4 tasks done
corentinlger opened this issue May 12, 2023 · 4 comments 路 May be fixed by #1574
Open
4 tasks done

check_env warning for FrameStacked observation in stable_baselines3.common.env_checker #1500

corentinlger opened this issue May 12, 2023 · 4 comments 路 May be fixed by #1574
Labels
bug Something isn't working openai gym related to OpenAI Gym interface

Comments

@corentinlger
Copy link
Contributor

馃悰 Bug

When using the "check_env "function of "stable_baselines3.common.env_checker" with an environment wrapped in a "FrameStack" wrapper from "gymnasium.wrappers", I get an error on the type of observation returned.

Indeed, the observation has a <class 'gymnasium.wrappers.frame_stack.LazyFrames'> instead of a np.array type (which is normal because it is indicated in the gymnasium documentation).

But then I can still use this environment with FrameStacked observation to train an agent. I know it is not critical but is this normal that the check_env warns about a problem while the environment can still be used to train agents with stable_baselines3 algorithms?

To Reproduce

import gymnasium as gym
from gymnasium.wrappers import FrameStack
from stable_baselines3.common.env_checker import check_env

env = gym.make("CartPole-v1")
env = FrameStack(env, n_stack)

# check the obs type
obs, info = env.reset()
print(type(obs))

check_env(env, warn = True)

Relevant log output / Error message

<class 'gymnasium.wrappers.frame_stack.LazyFrames'>

---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
Cell In[8], line 7
      4 obs, info = env.reset()
      5 print(type(obs))
----> 7 check_env(env, warn = True)

File ~/.local/lib/python3.10/site-packages/stable_baselines3/common/env_checker.py:433, in check_env(env, warn, skip_render_check)
    428         warnings.warn(
    429             f"Your action space has dtype {action_space.dtype}, we recommend using np.float32 to avoid cast errors."
    430         )
    432 # ============ Check the returned values ===============
--> 433 _check_returned_values(env, observation_space, action_space)
    435 # ==== Check the render method and the declared render modes ====
    436 if not skip_render_check:

File ~/.local/lib/python3.10/site-packages/stable_baselines3/common/env_checker.py:274, in _check_returned_values(env, observation_space, action_space)
    272             raise AssertionError(f"Error while checking key={key}: " + str(e)) from e
    273 else:
--> 274     _check_obs(obs, observation_space, "reset")
    276 # Sample a random action
    277 action = action_space.sample()

File ~/.local/lib/python3.10/site-packages/stable_baselines3/common/env_checker.py:189, in _check_obs(obs, observation_space, method_name)
    187     assert np.issubdtype(type(obs), np.integer), f"The observation returned by `{method_name}()` method must be an int"
    188 elif _is_numpy_array_space(observation_space):
--> 189     assert isinstance(obs, np.ndarray), f"The observation returned by `{method_name}()` method must be a numpy array"
    191 # Additional checks for numpy arrays, so the error message is clearer (see GH#1399)
    192 if isinstance(obs, np.ndarray):
    193     # check obs dimensions, dtype and bounds

AssertionError: The observation returned by `reset()` method must be a numpy array

System Info

I am using these versions :

  • Python: 3.10.6
  • stable_baselines : 2.0.0a5
  • gymnasium : 0.28.1

Checklist

  • I have checked that there is no similar issue in the repo
  • I have read the documentation
  • I have provided a minimal working example to reproduce the bug
  • I've used the markdown code blocks for both code and stack traces.
@corentinlger corentinlger added the bug Something isn't working label May 12, 2023
@qgallouedec
Copy link
Collaborator

Thank you for reporting. I agree that it shouldn't trigger a warning.

I think we should replace isinstance by np.can_cast, just like here: https://github.com/openai/gym/blob/dcd185843a62953e27c2d54dc8c2d647d604b635/gym/spaces/box.py#L234

However we should first check that it doesn't break anything.
This should not be the case if we always use the functional style (np.sum(x)) instead of the method style (x.sum())

@araffin araffin added the openai gym related to OpenAI Gym interface label May 12, 2023
@corentinlger
Copy link
Contributor Author

Do you think I could try to fix it ? (even if I never did open source yet)

@qgallouedec
Copy link
Collaborator

It's time to get started!

First, read the contribution guidelines, fork the repo, make your changes and open a pull request.

As a new, novice contributor, I'm curious to know what difficulties you had in this process. If you don't mind, I'd like you to compile a note of the points that gave you a hard time. (Something quite raw, to understand where are the points that are not directly crystal clear.) You can then share this note in a new issue when we've finished. This could help to improve support for new contributors.

@corentinlger
Copy link
Contributor Author

Before opening a pull request I had a question. Indeed the solution you proposed worked for this problem with Box and Dict observations (the 2 authorized observation spaces for FrameStack : https://gymnasium.farama.org/api/wrappers/observation_wrappers/#gymnasium.wrappers.FrameStack), but I still get 2 warnings and I don't know if it is the result you expected :

/home/flowers/Desktop/python/stable-baselines3/stable_baselines3/common/env_checker.py:238: UserWarning: Your observation has an unconventional shape (neither an image, nor a 1D vector). We recommend you to flatten the observation to have only a 1D vector or use a custom policy to properly process the data.
warnings.warn(
/home/flowers/.local/lib/python3.10/site-packages/gymnasium/spaces/box.py:230: UserWarning: WARN: Casting input x to numpy array.
gym.logger.warn("Casting input x to numpy array.")

The first warning is logical because indeed the observation won't be an image or a 1D vector after FrameStack. The second one doesn't seem critical either and depends of gymnasium but I was wondering if you knew a way not to trigger this warning.

And sure for the note if it can help others ! But I should maybe try to do more contributions before doing it, because you already gave me the solution here !

@corentinlger corentinlger linked a pull request Jun 26, 2023 that will close this issue
Open
7 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working openai gym related to OpenAI Gym interface
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix Framestack obs env_checker corentinlger/stable-baselines3
3 participants
@araffin @qgallouedec @corentinlger