Update the Multiprocessing section of the Examples

[BertrandDecoster]

This page was not fully updated for the passage to gymnasium.

Fixed the step() method that now returns a quintuplet and env.reset() that returns a tuple Made render_mode="rgb_array" for the training, but keep it "human" for visual evaluation Passed from SubprocVecEnv to DummyVecEnv as the former raises an Error

Description

Motivation and Context

It makes the code in the doc actually work

• I have raised an issue to propose this change (required for new features and bug fixes)

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Documentation (update in the documentation)

Checklist

• I've read the CONTRIBUTION guide (required)
• I have updated the changelog accordingly (required).
  No, it's too small
• My change requires a change to the documentation.
• I have updated the tests accordingly (required for a bug fix or a new feature).
• I have updated the documentation accordingly.
• I have opened an associated PR on the SB3-Contrib repository (if necessary)
• I have opened an associated PR on the RL-Zoo3 repository (if necessary)
• I have reformatted the code using make format (required)
• I have checked the codestyle using make check-codestyle and make lint (required)
• I have ensured make pytest and make type both pass. (required)
• I have checked that the documentation builds using make doc (required)

Note: You can run most of the checks using make commit-checks.

Note: we are using a maximum length of 127 characters per line

[BertrandDecoster]

I guess that the goal of this code sample is to show how faster a VecEnv can be (in wall clock time), and as such, having a human rendering for training just negates any benefit.

I agree that having switching from human rendering to rgb_array was not a bugfix technically (as it worked before), but I believe that the change is actually a "fix" in the intention of this section to showcase a speedup.

[araffin]

Hello,

Passed from SubprocVecEnv to DummyVecEnv as the former raises an Error

Can you elaborate on that?

I think the only change that makes sense here is to switch to "rgb_array" mode for training (SB3 VecEnv should be kept at test time, we support both modes using OpenCV).

[BertrandDecoster]

Regarding the error : this is what I get when I run the code

Exception has occurred: EOFError       (note: full exception trace is shown but execution is paused at: _run_module_as_main)
exception: no description
  File "/Users/bertranddecoster/mambaforge/envs/farama/lib/python3.10/multiprocessing/connection.py", line 383, in _recv
    raise EOFError
  File "/Users/bertranddecoster/mambaforge/envs/farama/lib/python3.10/multiprocessing/connection.py", line 414, in _recv_bytes
    buf = self._recv(4)
  File "/Users/bertranddecoster/mambaforge/envs/farama/lib/python3.10/multiprocessing/connection.py", line 250, in recv
    buf = self._recv_bytes()
  File "/Users/bertranddecoster/mambaforge/envs/farama/lib/python3.10/site-packages/stable_baselines3/common/vec_env/subproc_vec_env.py", line 120, in __init__
    observation_space, action_space = self.remotes[0].recv()
  File "/Users/bertranddecoster/Projects/farama/sb3errors.py", line 28, in <module>
    vec_env = SubprocVecEnv([make_env(env_id, i) for i in range(num_cpu)])
  File "/Users/bertranddecoster/mambaforge/envs/farama/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/Users/bertranddecoster/mambaforge/envs/farama/lib/python3.10/runpy.py", line 196, in _run_module_as_main (Current frame)
    return _run_code(code, main_globals, None,
EOFError: 

Config

MacOS 13.4 
M1 Pro
SB3 == 2.0.0

I can revert the changes to vec_env, should I also revert the change from quadruplet to quintuplet?

[araffin]

Regarding the error : this is what I get when I run the code

hmm, might be related to mac os.
I cannot reproduce the issue locally and I guess you cannot reproduce it either in a colab notebook?

I can revert the changes to vec_env, should I also revert the change from quadruplet to quintuplet?

yes please, only change the render_mode.