You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Describe the bug MaskablePPO calculates FPS here, which checks self._num_timesteps_at_start. However MaskablePPO overrides _setup_learn which is where this gets updated (in BaseAlgorithmhere), so it never updates this field. This means it's always 0, so it's incorrect when reset_num_timestamps=False.
Code example
N/A (the issue is present but looking at the code I linked above)
Just passing reset_num_timesteps=False into MaskablePPO.learn() will log incorrect fps after the first call to learn
Additional context
For ex. here's my tensorboard graph when using the following code:
while True:
ppo.learn(total_timesteps=1000000,
tb_log_name='agent',
reset_num_timesteps=False,
callback=checkpoint_callback)
As you can see, FPS shot up right after the first 1000000 timesteps, when learn was called again.
The text was updated successfully, but these errors were encountered:
Naton1
changed the title
MaskablePPO doesn't set _num_timesteps_at_start so fps count is wrong when reset_num_timesteps=False
MaskablePPO doesn't set _num_timesteps_at_start so fps count is wrong when reset_num_timesteps=False in learn()
Mar 4, 2023
Describe the bug
MaskablePPO
calculates FPS here, which checksself._num_timesteps_at_start
. HoweverMaskablePPO
overrides_setup_learn
which is where this gets updated (inBaseAlgorithm
here), so it never updates this field. This means it's always 0, so it's incorrect whenreset_num_timestamps=False
.Code example
N/A (the issue is present but looking at the code I linked above)
Just passing
reset_num_timesteps=False
intoMaskablePPO.learn()
will log incorrect fps after the first call tolearn
System Info
Additional context
For ex. here's my tensorboard graph when using the following code:
As you can see, FPS shot up right after the first 1000000 timesteps, when
learn
was called again.The text was updated successfully, but these errors were encountered: