You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
There are four processes generating data into four folders simultaneously when i run ant_data_collect.py. But when one of the processes produces 2000 sets of data, the program will be interrupted with the following error:
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "ant_data_collect.py", line 41, in
run_sweep_parallel(main, params_dict, repeat=4)
File "/home/wsco/inverse_rl/inverse_rl/utils/hyper_sweep.py", line 57, in run_sweep_parallel
pool.map(kwargs_wrapper, exp_args)
File "/home/wsco/anaconda3/envs/irl_wjw/lib/python3.5/multiprocessing/pool.py", line 260, in map
return self._map_async(func, iterable, mapstar, chunksize).get()
File "/home/wsco/anaconda3/envs/irl_wjw/lib/python3.5/multiprocessing/pool.py", line 608, in get
raise self._value
AssertionError: Nesting violated for default stack of <class 'tensorflow.python.client.session.Session'> objects
Could you tell me how to fix the problem?Thanks.
The text was updated successfully, but these errors were encountered:
There are four processes generating data into four folders simultaneously when i run ant_data_collect.py. But when one of the processes produces 2000 sets of data, the program will be interrupted with the following error:
2019-10-31 21:22:16.830585 EDT | itr #1999 | Saving snapshot...
0% [########################## ] 100% | ETA: 00:00:012019-10-31 21:22:17.182188 EDT | itr #1999 | Saved
2019-10-31 21:22:17.183297 EDT | ----------------------- --------------
2019-10-31 21:22:17.183448 EDT | PolicyExecTime 0.370542
2019-10-31 21:22:17.183600 EDT | EnvExecTime 13.9964
2019-10-31 21:22:17.183725 EDT | ProcessExecTime 0.203515
2019-10-31 21:22:17.183844 EDT | Iteration 1999
2019-10-31 21:22:17.183962 EDT | AverageDiscountedReturn 172.879
2019-10-31 21:22:17.184080 EDT | AverageReturn 1034.59
2019-10-31 21:22:17.184197 EDT | ExplainedVariance 0.978998
2019-10-31 21:22:17.184314 EDT | NumTrajs 40
2019-10-31 21:22:17.184431 EDT | Entropy -0.779469
2019-10-31 21:22:17.184549 EDT | Perplexity 0.458649
2019-10-31 21:22:17.184672 EDT | StdReturn 26.7545
2019-10-31 21:22:17.184789 EDT | MaxReturn 1081.51
2019-10-31 21:22:17.184906 EDT | MinReturn 949.139
2019-10-31 21:22:17.185023 EDT | AvgRewardFwd 2.17163
2019-10-31 21:22:17.185139 EDT | AvgRewardCtrl -0.101136
2019-10-31 21:22:17.185255 EDT | AvgRewardContact -0.00131123
2019-10-31 21:22:17.185371 EDT | AvgRewardFlipped 0
2019-10-31 21:22:17.185488 EDT | AveragePolicyStd 0.220438
2019-10-31 21:22:17.185605 EDT | LossBefore 0.077947
2019-10-31 21:22:17.185721 EDT | LossAfter 0.0633926
2019-10-31 21:22:17.185839 EDT | MeanKLBefore 0
2019-10-31 21:22:17.185955 EDT | MeanKL 0.00640616
2019-10-31 21:22:17.186072 EDT | dLoss 0.0145543
2019-10-31 21:22:17.186189 EDT | Time 30710.5
2019-10-31 21:22:17.186305 EDT | ItrTime 15.5186
2019-10-31 21:22:17.186433 EDT | ----------------------- --------------
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
File "/home/wsco/anaconda3/envs/irl_wjw/lib/python3.5/multiprocessing/pool.py", line 119, in worker
result = (True, func(*args, **kwds))
File "/home/wsco/anaconda3/envs/irl_wjw/lib/python3.5/multiprocessing/pool.py", line 44, in mapstar
return list(map(*args))
File "/home/wsco/inverse_rl/inverse_rl/utils/hyper_sweep.py", line 47, in kwargs_wrapper
return method(**args)
File "ant_data_collect.py", line 34, in main
algo.train()
File "/home/wsco/anaconda3/envs/irl_wjw/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1592, in exit
exec_tb)
File "/home/wsco/anaconda3/envs/irl_wjw/lib/python3.5/contextlib.py", line 66, in exit
next(self.gen)
File "/home/wsco/anaconda3/envs/irl_wjw/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 5467, in get_controller
type(default))
AssertionError: Nesting violated for default stack of <class 'tensorflow.python.client.session.Session'> objects
"""
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "ant_data_collect.py", line 41, in
run_sweep_parallel(main, params_dict, repeat=4)
File "/home/wsco/inverse_rl/inverse_rl/utils/hyper_sweep.py", line 57, in run_sweep_parallel
pool.map(kwargs_wrapper, exp_args)
File "/home/wsco/anaconda3/envs/irl_wjw/lib/python3.5/multiprocessing/pool.py", line 260, in map
return self._map_async(func, iterable, mapstar, chunksize).get()
File "/home/wsco/anaconda3/envs/irl_wjw/lib/python3.5/multiprocessing/pool.py", line 608, in get
raise self._value
AssertionError: Nesting violated for default stack of <class 'tensorflow.python.client.session.Session'> objects
Could you tell me how to fix the problem?Thanks.
The text was updated successfully, but these errors were encountered: