An issue when running "multi_agent_training_with_rllib" notebook

[aslansd]

Hello all!

I could install AI-Economist with all its dependencies in my laptop as it is was explained in your Github. I could also run successfully most of the notebooks without any issues. However, now I am running "multi_agent_training_with_rllib" notebook and it gives me an error in the following part:

Below, we fetch the dense logs for each rollout worker and environment within

dense_logs = {}

Note: worker 0 is reserved for the trainer actor

for worker in range((trainer_config["num_workers"] > 0), trainer_config["num_workers"] + 1):
for env_id in range(trainer_config["num_envs_per_worker"]):
dense_logs["worker={};env_id={}".format(worker, env_id)] =
trainer.workers.foreach_worker(lambda w: w.async_env)[worker].envs[env_id].env.previous_episode_dense_log

Here is the error:

---

RayTaskError(TypeError) Traceback (most recent call last)
~\AppData\Local\Temp\ipykernel_2452\4135580017.py in
6 for env_id in range(trainer_config["num_envs_per_worker"]):
7 dense_logs["worker={};env_id={}".format(worker, env_id)] =
----> 8 trainer.workers.foreach_worker(lambda w: w.async_env)[worker].envs[env_id].env.previous_episode_dense_log

~\anaconda3\envs\ai-economist\lib\site-packages\ray\rllib\evaluation\worker_set.py in foreach_worker(self, func)
351 if self.local_worker() is not None:
352 local_result = [func(self.local_worker())]
--> 353 remote_results = ray.get([w.apply.remote(func) for w in self.remote_workers()])
354 return local_result + remote_results
355

~\anaconda3\envs\ai-economist\lib\site-packages\ray_private\client_mode_hook.py in wrapper(*args, **kwargs)
103 if func.name != "init" or is_client_mode_enabled_by_default:
104 return getattr(ray, func.name)(*args, **kwargs)
--> 105 return func(*args, **kwargs)
106
107 return wrapper

~\anaconda3\envs\ai-economist\lib\site-packages\ray\worker.py in get(object_refs, timeout)
1829 worker.core_worker.dump_object_store_memory_usage()
1830 if isinstance(value, RayTaskError):
-> 1831 raise value.as_instanceof_cause()
1832 else:
1833 raise value

RayTaskError(TypeError): ray::RolloutWorker.apply() (pid=4748, ip=127.0.0.1, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x00000213920F8848>)
File "python\ray_raylet.pyx", line 728, in ray._raylet.execute_task
File "python\ray_raylet.pyx", line 729, in ray._raylet.execute_task
File "python\ray_raylet.pyx", line 2028, in ray._raylet.CoreWorker.store_task_outputs
File "C:\Users\Aslan\anaconda3\envs\ai-economist\lib\site-packages\ray\serialization.py", line 421, in serialize
return self._serialize_to_msgpack(value)
File "C:\Users\Aslan\anaconda3\envs\ai-economist\lib\site-packages\ray\serialization.py", line 400, in _serialize_to_msgpack
metadata, python_objects
File "C:\Users\Aslan\anaconda3\envs\ai-economist\lib\site-packages\ray\serialization.py", line 361, in _serialize_to_pickle5
raise e
File "C:\Users\Aslan\anaconda3\envs\ai-economist\lib\site-packages\ray\serialization.py", line 357, in _serialize_to_pickle5
value, protocol=5, buffer_callback=writer.buffer_callback
File "C:\Users\Aslan\anaconda3\envs\ai-economist\lib\site-packages\ray\cloudpickle\cloudpickle_fast.py", line 74, in dumps
cp.dump(obj)
File "C:\Users\Aslan\anaconda3\envs\ai-economist\lib\site-packages\ray\cloudpickle\cloudpickle_fast.py", line 621, in dump
return Pickler.dump(self, obj)
TypeError: can't pickle generator objects

I think this error is related to the compatibility issues of different modules. Here is the details of my operating system:

Processor Intel(R) Core(TM) i7-8550U CPU @ 1.80GHz 1.99 GHz
Installed RAM 16.0 GB (15.9 GB usable)
System type 64-bit operating system, x64-based processor

Edition Windows 10 Pro
Version 21H1
Experience Windows Feature Experience Pack 120.2212.4180.0

Here is the installed modules in my ai-economist environment:

Name Version Build Channel

absl-py 1.1.0 pypi_0 pypi
aiosignal 1.2.0 pypi_0 pypi
alabaster 0.7.12 py37_0 anaconda
astroid 2.9.0 py37haa95532_0 anaconda
astunparse 1.6.3 pypi_0 pypi
attrs 21.2.0 pypi_0 pypi
babel 2.9.1 pyhd3eb1b0_0 anaconda
backcall 0.2.0 pyhd3eb1b0_0 anaconda
beautifulsoup4 4.11.1 py37haa95532_0 anaconda
blas 1.0 mkl
bleach 4.1.0 pyhd3eb1b0_0 anaconda
brotlipy 0.7.0 py37h2bbff1b_1003 anaconda
ca-certificates 2022.4.26 haa95532_0 anaconda
cachetools 5.2.0 pypi_0 pypi
certifi 2022.5.18.1 py37haa95532_0
cffi 1.14.5 pypi_0 pypi
chardet 4.0.0 py37haa95532_1003 anaconda
charset-normalizer 2.0.4 pyhd3eb1b0_0 anaconda
cloudpickle 2.0.0 pyhd3eb1b0_0 anaconda
colorama 0.4.4 pyhd3eb1b0_0 anaconda
cryptography 36.0.0 py37h21b164f_0 anaconda
cudatoolkit 11.3.1 h59b6b97_2
debugpy 1.5.1 py37hd77b12b_0 anaconda
decorator 5.0.9 pypi_0 pypi
defusedxml 0.7.1 pyhd3eb1b0_0 anaconda
distlib 0.3.4 pypi_0 pypi
docutils 0.17.1 py37haa95532_1 anaconda
entrypoints 0.4 py37haa95532_0 anaconda
filelock 3.7.1 pypi_0 pypi
flatbuffers 1.12 pypi_0 pypi
freetype 2.10.4 hd328e21_0
frozenlist 1.3.0 pypi_0 pypi
gast 0.4.0 pypi_0 pypi
google-auth 2.8.0 pypi_0 pypi
google-auth-oauthlib 0.4.6 pypi_0 pypi
google-pasta 0.2.0 pypi_0 pypi
grpcio 1.43.0 pypi_0 pypi
gym 0.21.0 pypi_0 pypi
icu 58.2 vc14hc45fdbb_0 [vc14] anaconda
idna 3.3 pyhd3eb1b0_0 anaconda
imageio 2.19.3 pypi_0 pypi
imagesize 1.3.0 pyhd3eb1b0_0 anaconda
importlib-metadata 4.11.3 py37haa95532_0 anaconda
importlib_metadata 4.11.3 hd3eb1b0_0 anaconda
importlib_resources 5.2.0 pyhd3eb1b0_1 anaconda
intel-openmp 2021.4.0 haa95532_3556
ipykernel 6.15.0 pypi_0 pypi
ipython 7.34.0 pypi_0 pypi
ipython_genutils 0.2.0 pyhd3eb1b0_1 anaconda
isort 5.9.3 pyhd3eb1b0_0 anaconda
jedi 0.18.0 pypi_0 pypi
jinja2 3.0.3 pyhd3eb1b0_0 anaconda
jpeg 9b vc14h4d7706e_1 [vc14] anaconda
jsonschema 4.4.0 py37haa95532_0 anaconda
jupyter-client 7.3.4 pypi_0 pypi
jupyter_client 7.2.2 py37haa95532_0
jupyter_core 4.10.0 py37haa95532_0
jupyterlab-pygments 0.2.2 pypi_0 pypi
jupyterlab_pygments 0.1.2 py_0 anaconda
keras 2.9.0 pypi_0 pypi
keras-preprocessing 1.1.2 pypi_0 pypi
keyring 23.4.0 py37haa95532_0 anaconda
lazy-object-proxy 1.6.0 py37h2bbff1b_0 anaconda
libclang 14.0.1 pypi_0 pypi
libpng 1.6.37 h2a8f88b_0 anaconda
libtiff 4.2.0 hd0e1b90_0
libuv 1.40.0 he774522_0
libwebp 1.2.2 h2bbff1b_0
lz4-c 1.9.3 h2bbff1b_1
markdown 3.3.7 pypi_0 pypi
markupsafe 2.0.1 py37h2bbff1b_0 anaconda
matplotlib-inline 0.1.2 pyhd3eb1b0_2 anaconda
mccabe 0.7.0 pyhd3eb1b0_0 anaconda
mistune 0.8.4 py37hfa6e2cd_1001 anaconda
mkl 2021.4.0 haa95532_640
mkl-service 2.4.0 py37h2bbff1b_0
mkl_fft 1.3.1 py37h277e83a_0
mkl_random 1.2.2 py37hf11a4ad_0
msgpack 1.0.4 pypi_0 pypi
nbclient 0.5.13 py37haa95532_0 anaconda
nbconvert 6.4.4 py37haa95532_0 anaconda
nbformat 5.3.0 py37haa95532_0 anaconda
nest-asyncio 1.5.5 py37haa95532_0 anaconda
networkx 2.6.3 pypi_0 pypi
numpy 1.21.5 py37h7a0a035_3
numpy-base 1.21.5 py37hca35cd5_3
numpydoc 1.2 pyhd3eb1b0_0 anaconda
oauthlib 3.2.0 pypi_0 pypi
openssl 1.1.1o h2bbff1b_0
opt-einsum 3.3.0 pypi_0 pypi
packaging 20.9 pypi_0 pypi
pandocfilters 1.5.0 pyhd3eb1b0_0 anaconda
parso 0.8.2 pypi_0 pypi
pickleshare 0.7.5 pyhd3eb1b0_1003 anaconda
pillow 9.0.1 py37hdc2b20a_0
pip 22.1.2 pypi_0 pypi
platformdirs 2.4.0 pyhd3eb1b0_0 anaconda
prompt-toolkit 3.0.18 pypi_0 pypi
psutil 5.8.0 py37h2bbff1b_1 anaconda
pyasn1 0.4.8 pypi_0 pypi
pyasn1-modules 0.2.8 pypi_0 pypi
pycodestyle 2.8.0 pyhd3eb1b0_0 anaconda
pycparser 2.20 pypi_0 pypi
pyflakes 2.4.0 pyhd3eb1b0_0 anaconda
pygments 2.9.0 pypi_0 pypi
pylint 2.12.2 py37haa95532_1 anaconda
pyopenssl 22.0.0 pyhd3eb1b0_0 anaconda
pyparsing 3.0.4 pyhd3eb1b0_0 anaconda
pyqt 5.9.2 py37ha878b3d_0 anaconda
pyrsistent 0.17.3 pypi_0 pypi
pysocks 1.7.1 py37_1 anaconda
python 3.7.13 h6244533_0
python-dateutil 2.8.2 pyhd3eb1b0_0
python-fastjsonschema 2.15.1 pyhd3eb1b0_0 anaconda
pytorch 1.11.0 py3.7_cuda11.3_cudnn8_0 pytorch
pytorch-mutex 1.0 cuda pytorch
pytz 2021.3 pyhd3eb1b0_0 anaconda
pywavelets 1.3.0 pypi_0 pypi
pywin32 302 py37h2bbff1b_2 anaconda
pywin32-ctypes 0.2.0 py37_1001 anaconda
pywinpty 2.0.5 pypi_0 pypi
pyzmq 23.2.0 pypi_0 pypi
qt 5.9.7 vc14h73c81de_0 [vc14] anaconda
qtawesome 1.0.3 pyhd3eb1b0_0 anaconda
qtconsole 5.3.0 pyhd3eb1b0_0 anaconda
qtpy 2.0.1 pyhd3eb1b0_0 anaconda
ray 1.13.0 pypi_0 pypi
requests 2.27.1 pyhd3eb1b0_0 anaconda
requests-oauthlib 1.3.1 pypi_0 pypi
rope 0.22.0 pyhd3eb1b0_0 anaconda
rsa 4.8 pypi_0 pypi
scikit-image 0.19.3 pypi_0 pypi
setuptools 62.6.0 pypi_0 pypi
sip 6.5.1 py37hd77b12b_0 anaconda
six 1.16.0 pyhd3eb1b0_1 anaconda
snowballstemmer 2.2.0 pyhd3eb1b0_0 anaconda
soupsieve 2.3.1 pyhd3eb1b0_0 anaconda
sphinx 4.4.0 pyhd3eb1b0_0 anaconda
sphinxcontrib-applehelp 1.0.2 pyhd3eb1b0_0 anaconda
sphinxcontrib-devhelp 1.0.2 pyhd3eb1b0_0 anaconda
sphinxcontrib-htmlhelp 2.0.0 pyhd3eb1b0_0 anaconda
sphinxcontrib-jsmath 1.0.1 pyhd3eb1b0_0 anaconda
sphinxcontrib-qthelp 1.0.3 pyhd3eb1b0_0 anaconda
sphinxcontrib-serializinghtml 1.1.5 pyhd3eb1b0_0 anaconda
spyder 3.3.6 py37_0 anaconda
spyder-kernels 0.5.2 py37_0 anaconda
sqlite 3.38.3 h2bbff1b_0
tabulate 0.8.9 pypi_0 pypi
tensorboard 2.9.1 pypi_0 pypi
tensorboard-data-server 0.6.1 pypi_0 pypi
tensorboard-plugin-wit 1.8.1 pypi_0 pypi
tensorboardx 2.5.1 pypi_0 pypi
tensorflow 2.9.1 pypi_0 pypi
tensorflow-estimator 2.9.0 pypi_0 pypi
tensorflow-io-gcs-filesystem 0.26.0 pypi_0 pypi
termcolor 1.1.0 pypi_0 pypi
testpath 0.5.0 pyhd3eb1b0_0 anaconda
tifffile 2021.11.2 pypi_0 pypi
tk 8.6.12 h2bbff1b_0
toml 0.10.2 pyhd3eb1b0_0 anaconda
torchaudio 0.11.0 py37_cu113 pytorch
torchvision 0.12.0 py37_cu113 pytorch
tornado 6.1 py37h2bbff1b_0 anaconda
traitlets 5.3.0 pypi_0 pypi
typed-ast 1.4.3 py37h2bbff1b_1 anaconda
typing-extensions 3.10.0.0 pypi_0 pypi
typing_extensions 4.1.1 pyh06a4308_0
urllib3 1.26.9 py37haa95532_0 anaconda
vc 14.2 h21ff451_1
virtualenv 20.14.1 pypi_0 pypi
vs2015_runtime 14.27.29016 h5e58377_2
wcwidth 0.2.5 pyhd3eb1b0_0 anaconda
webencodings 0.5.1 py37_1 anaconda
werkzeug 2.1.2 pypi_0 pypi
wheel 0.37.1 pyhd3eb1b0_0
win_inet_pton 1.1.0 py37haa95532_0 anaconda
wincertstore 0.2 py37haa95532_2
wrapt 1.13.3 py37h2bbff1b_2 anaconda
xz 5.2.5 h8cc25b3_1
zipp 3.7.0 pyhd3eb1b0_0 anaconda
zlib 1.2.11 vc14h1cdd9ab_1 [vc14] anaconda
zstd 1.4.9 h19a0ad4_0

I was wondering if you could hint any possible solution. Many thanks in advance!