Bugs in PPO

[moonblue333]

1. counter

2. for index in BatchSampler(SubsetRandomSampler(range(self.buffer_capacity), self.batch_size, True)):

[yuntao-ma]

1. How to solve bug 2?
2. It seems that "done" from the env hasn't been dealt with. Why?

Thanks.

[HuangHaoyu1997]

@yuntao-ma
for index in BatchSampler(SubsetRandomSampler(range(self.buffer_capacity)), self.batch_size, True):

[brezezee]

Why can I train with this code to only get nan actions

[xxx-007]

I get nan actions too

[HzcIrving]

I change the code to :
for index in BatchSampler(SubsetRandomSampler(range(self.buffer_capacity)), self.batch_size, True):
but there still exists a bug:
Traceback (most recent call last): File "E:/AAAFor_PHD/UUV_SCI_Modif/UUV_obs_env/PPO2/Demo/PPO_demo.py", line 195, in <module> main() File "E:/AAAFor_PHD/UUV_SCI_Modif/UUV_obs_env/PPO2/Demo/PPO_demo.py", line 175, in main next_state, reward, done, info = env.step(action) File "F:\Anaconda\envs\Obstacle_Avoid\lib\site-packages\gym\envs\classic_control\pendulum.py", line 49, in step u = np.clip(u, -self.max_torque, self.max_torque)[0] IndexError: invalid index to scalar variable.

[haohaoqian]

Transition = namedtuple('Transition',['state', 'aciton', 'reward', 'a_log_prob', 'next_state'])
'aciton' should be 'action'