Issues using the custom neural network models in AI-Economist

[aslansd]

Hello,

I am developing AI-Economist and I am trying to run my own modified version of AI-Economist with your provided custom neural network models: KerasConvLSTM and RandomAction.

Here is my system properties:

Model Name: MacBook Pro
Model Identifier: MacBookPro17,1
Chip: Apple M1
Total Number of Cores: 8 (4 performance and 4 efficiency)
Memory: 16 GB

Here is the list of modules in my modified ai-economist environment:

Name Version Build Channel

absl-py 1.2.0 pypi_0 pypi
aiosignal 1.2.0 pypi_0 pypi
appdirs 1.4.4 pypi_0 pypi
appnope 0.1.2 py310hca03da5_1001 anaconda
argon2-cffi 20.1.0 pypi_0 pypi
astroid 2.5.6 pypi_0 pypi
asttokens 2.0.5 pyhd3eb1b0_0 anaconda
astunparse 1.6.3 pypi_0 pypi
async-generator 1.10 pypi_0 pypi
attrs 21.2.0 pypi_0 pypi
backcall 0.2.0 pyhd3eb1b0_0 anaconda
beautifulsoup4 4.9.3 pypi_0 pypi
black 22.8.0 pypi_0 pypi
bleach 3.3.0 pypi_0 pypi
bs4 0.0.1 pypi_0 pypi
bzip2 1.0.8 h620ffc9_4 anaconda
c-ares 1.18.1 h3422bc3_0 conda-forge
ca-certificates 2022.07.19 hca03da5_0
cached-property 1.5.2 hd8ed1ab_1 conda-forge
cached_property 1.5.2 pyha770c72_1 conda-forge
cachetools 5.2.0 pypi_0 pypi
certifi 2020.12.5 pypi_0 pypi
cffi 1.14.5 pypi_0 pypi
chardet 4.0.0 pypi_0 pypi
click 8.0.1 pypi_0 pypi
cloudpickle 2.1.0 pypi_0 pypi
cycler 0.10.0 pypi_0 pypi
debugpy 1.5.1 py310hc377ac9_0 anaconda
decorator 5.0.9 pypi_0 pypi
defusedxml 0.7.1 pypi_0 pypi
distlib 0.3.6 pypi_0 pypi
dm-tree 0.1.7 pypi_0 pypi
entrypoints 0.3 pypi_0 pypi
et-xmlfile 1.1.0 pypi_0 pypi
executing 0.8.3 pyhd3eb1b0_0 anaconda
filelock 3.8.0 pypi_0 pypi
flake8 3.9.2 pypi_0 pypi
flatbuffers 1.12 pypi_0 pypi
fonttools 4.37.1 pypi_0 pypi
frozenlist 1.3.1 pypi_0 pypi
gast 0.4.0 pypi_0 pypi
google-auth 2.11.0 pypi_0 pypi
google-auth-oauthlib 0.4.6 pypi_0 pypi
google-pasta 0.2.0 pypi_0 pypi
gputil 1.4.0 pypi_0 pypi
grpcio 1.43.0 py310h00ca444_0 conda-forge
gym 0.21.0 pypi_0 pypi
h5py 3.6.0 nompi_py310hb8bbf05_100 conda-forge
hdf5 1.12.1 nompi_h829dc4f_103 conda-forge
idna 2.10 pypi_0 pypi
imageio 2.21.2 pypi_0 pypi
iniconfig 1.1.1 pypi_0 pypi
ipykernel 5.5.5 pypi_0 pypi
ipython 7.31.1 pypi_0 pypi
ipython-genutils 0.2.0 pypi_0 pypi
ipywidgets 7.6.3 pypi_0 pypi
isort 5.8.0 pypi_0 pypi
jedi 0.18.0 pypi_0 pypi
jinja2 3.0.1 pypi_0 pypi
jsonschema 3.2.0 pypi_0 pypi
jupyter 1.0.0 pypi_0 pypi
jupyter-client 6.1.12 pypi_0 pypi
jupyter-console 6.4.0 pypi_0 pypi
jupyter-core 4.7.1 pypi_0 pypi
jupyter_client 7.2.2 py310hca03da5_0 anaconda
jupyter_core 4.10.0 py310hca03da5_0 anaconda
jupyterlab-pygments 0.1.2 pypi_0 pypi
jupyterlab-widgets 1.0.0 pypi_0 pypi
keras 2.9.0 pypi_0 pypi
keras-preprocessing 1.1.2 pypi_0 pypi
kiwisolver 1.3.1 pypi_0 pypi
krb5 1.19.2 hfff258c_3 conda-forge
lazy-object-proxy 1.6.0 pypi_0 pypi
libblas 3.9.0 16_osxarm64_openblas conda-forge
libcblas 3.9.0 16_osxarm64_openblas conda-forge
libclang 14.0.6 pypi_0 pypi
libcurl 7.81.0 h8d07098_0 conda-forge
libcxx 12.0.0 hf6beb65_1 anaconda
libedit 3.1.20191231 hc8eb9b7_2 conda-forge
libev 4.33 h642e427_1 conda-forge
libffi 3.4.2 hc377ac9_4 anaconda
libgfortran 5.0.0 11_3_0_hd922786_25 conda-forge
libgfortran5 11.3.0 hdaf2cc0_25 conda-forge
liblapack 3.9.0 16_osxarm64_openblas conda-forge
libnghttp2 1.46.0 h62a264d_0 conda-forge
libopenblas 0.3.21 openmp_hc731615_3 conda-forge
libsodium 1.0.18 h1a28f6b_0 anaconda
libssh2 1.10.0 h7a5bd25_3 conda-forge
libzlib 1.2.12 ha287fd2_2 conda-forge
llvm-openmp 14.0.4 hd125106_0 conda-forge
lz4 3.1.3 pypi_0 pypi
markdown 3.4.1 pypi_0 pypi
markupsafe 2.1.1 pypi_0 pypi
matplotlib 3.5.3 pypi_0 pypi
matplotlib-inline 0.1.6 pypi_0 pypi
mccabe 0.6.1 pypi_0 pypi
mistune 0.8.4 pypi_0 pypi
modified-ai-economist 1.1.0 dev_0
msgpack 1.0.4 pypi_0 pypi
mypy-extensions 0.4.3 pypi_0 pypi
nbclient 0.5.3 pypi_0 pypi
nbconvert 6.0.7 pypi_0 pypi
nbformat 5.1.3 pypi_0 pypi
ncurses 6.3 h1a28f6b_2 anaconda
nest-asyncio 1.5.1 pypi_0 pypi
networkx 2.8.6 pypi_0 pypi
notebook 6.4.1 pypi_0 pypi
numpy 1.21.0 pypi_0 pypi
oauthlib 3.2.0 pypi_0 pypi
openpyxl 3.0.7 pypi_0 pypi
openssl 3.0.5 h03a7124_2 conda-forge
opt-einsum 3.3.0 pypi_0 pypi
packaging 20.9 pypi_0 pypi
pandas 1.2.4 pypi_0 pypi
pandocfilters 1.4.3 pypi_0 pypi
parso 0.8.2 pypi_0 pypi
pathspec 0.10.1 pypi_0 pypi
pexpect 4.8.0 pyhd3eb1b0_3 anaconda
pickleshare 0.7.5 pyhd3eb1b0_1003 anaconda
pillow 9.0.1 pypi_0 pypi
pip 21.2.4 py310hca03da5_0 anaconda
platformdirs 2.5.2 pypi_0 pypi
pluggy 0.13.1 pypi_0 pypi
prometheus-client 0.10.1 pypi_0 pypi
prompt-toolkit 3.0.18 pypi_0 pypi
protobuf 3.19.4 pypi_0 pypi
ptyprocess 0.7.0 pyhd3eb1b0_2 anaconda
pure_eval 0.2.2 pyhd3eb1b0_0 anaconda
py 1.10.0 pypi_0 pypi
pyasn1 0.4.8 pypi_0 pypi
pyasn1-modules 0.2.8 pypi_0 pypi
pycodestyle 2.7.0 pypi_0 pypi
pycparser 2.20 pypi_0 pypi
pycryptodome 3.10.1 pypi_0 pypi
pyflakes 2.3.1 pypi_0 pypi
pygments 2.9.0 pypi_0 pypi
pylint 2.8.2 pypi_0 pypi
pyparsing 2.4.7 pypi_0 pypi
pyrsistent 0.17.3 pypi_0 pypi
pytest 6.2.4 pypi_0 pypi
python 3.10.5 h4eee789_0_cpython conda-forge
python-dateutil 2.8.1 pypi_0 pypi
python_abi 3.10 2_cp310 conda-forge
pytz 2021.1 pypi_0 pypi
pywavelets 1.3.0 pypi_0 pypi
pyyaml 5.4.1 pypi_0 pypi
pyzmq 22.0.3 pypi_0 pypi
qtconsole 5.1.0 pypi_0 pypi
qtpy 1.9.0 pypi_0 pypi
ray 2.0.0 pypi_0 pypi
readline 8.1.2 h1a28f6b_1 anaconda
regex 2021.4.4 pypi_0 pypi
requests 2.25.1 pypi_0 pypi
requests-oauthlib 1.3.1 pypi_0 pypi
rsa 4.9 pypi_0 pypi
scikit-image 0.19.3 pypi_0 pypi
scipy 1.9.1 pypi_0 pypi
seaborn 0.12.0 pypi_0 pypi
send2trash 1.5.0 pypi_0 pypi
setuptools 61.2.0 py310hca03da5_0 anaconda
six 1.16.0 pyh6c4a22f_0 conda-forge
soupsieve 2.2.1 pypi_0 pypi
sqlite 3.38.5 h1058600_0 anaconda
stack_data 0.2.0 pyhd3eb1b0_0 anaconda
tabulate 0.8.10 pypi_0 pypi
tensorboard 2.9.1 pypi_0 pypi
tensorboard-data-server 0.6.1 pypi_0 pypi
tensorboard-plugin-wit 1.8.1 pypi_0 pypi
tensorboardx 2.5.1 pypi_0 pypi
tensorflow-deps 2.8.0 0 apple
tensorflow-estimator 2.9.0 pypi_0 pypi
tensorflow-macos 2.9.2 pypi_0 pypi
tensorflow-metal 0.5.1 pypi_0 pypi
termcolor 1.1.0 pypi_0 pypi
terminado 0.10.0 pypi_0 pypi
testpath 0.5.0 pypi_0 pypi
tifffile 2022.8.12 pypi_0 pypi
tk 8.6.12 hb8d0fd4_0 anaconda
toml 0.10.2 pypi_0 pypi
tomli 2.0.1 pypi_0 pypi
tornado 6.1 py310h1a28f6b_0 anaconda
tqdm 4.60.0 pypi_0 pypi
traitlets 5.0.5 pypi_0 pypi
typing-extensions 3.10.0.0 pypi_0 pypi
tzdata 2022a hda174b7_0 anaconda
urllib3 1.26.5 pypi_0 pypi
virtualenv 20.16.4 pypi_0 pypi
wcwidth 0.2.5 pyhd3eb1b0_0 anaconda
webencodings 0.5.1 pypi_0 pypi
werkzeug 2.2.2 pypi_0 pypi
wheel 0.37.1 pyhd3eb1b0_0 anaconda
widgetsnbextension 3.5.1 pypi_0 pypi
wrapt 1.12.1 pypi_0 pypi
xz 5.2.5 h1a28f6b_1 anaconda
zeromq 4.3.4 hc377ac9_0 anaconda
zlib 1.2.12 h5a0b063_2 anaconda

As it is clear from the above list, there are some extra modules beside the required modules of the original AI-Economist and also some modules have different versions than what are mentioned in the original AI-Economist. For example, the ray version here is 2.0.0. I think with this version of ray, the parent class of any tensorflow recurrent custom model should be set RecurrentNetwork instead of RecurrentTFModelV2, so I made modifications in tf_models as required.

Then, I ran the following commands similar to what are brought in your provided notebook: multi_agent_training_with_rllib

from rllib.env_wrapper import RLlibEnvWrapper
env_obj = RLlibEnvWrapper({"env_config_dict": env_config_dict}, verbose=True)

import ray
from ray.rllib.agents.ppo import PPOTrainer

from ray.rllib.models.catalog import ModelCatalog
from rllib.tf_models import KerasConvLSTM, RandomAction

ModelCatalog.register_custom_model(KerasConvLSTM.custom_name, KerasConvLSTM)
ModelCatalog.register_custom_model(RandomAction.custom_name, RandomAction)

policies = {
"a": (
None, # uses default policy
env_obj.observation_space,
env_obj.action_space,
{'clip_param': 0.3,
'entropy_coeff': 0.025,
'entropy_coeff_schedule': None,
'gamma': 0.998,
'grad_clip': 10.0,
'kl_coeff': 0.0,
'kl_target': 0.01,
'lambda': 0.98,
'lr': 0.0003,
'lr_schedule': None,
'model': {'custom_model': 'keras_conv_lstm',
'custom_model_config': {'fc_dim': 128,
'idx_emb_dim': 4,
'input_emb_vocab': 100,
'lstm_cell_size': 128,
'num_conv': 2,
'num_fc': 2},
'max_seq_len': 25},
'use_gae': True,
'vf_clip_param': 50.0,
'vf_loss_coeff': 0.05,
'vf_share_layers': False} # define a custom agent policy configuration.
),
"p": (
None, # uses default policy
env_obj.observation_space_pl,
env_obj.action_space_pl,
{'clip_param': 0.3,
'entropy_coeff': 0.125,
'entropy_coeff_schedule': [[0, 2.0], [50000000, 0.125]],
'gamma': 0.998,
'grad_clip': 10.0,
'kl_coeff': 0.0,
'kl_target': 0.01,
'lambda': 0.98,
'lr': 0.0001,
'lr_schedule': None,
'model': {'custom_model': 'keras_conv_lstm',
'custom_model_config': {'fc_dim': 256,
'idx_emb_dim': 4,
'input_emb_vocab': 100,
'lstm_cell_size': 256,
'num_conv': 2,
'num_fc': 2},
'max_seq_len': 25},
'use_gae': True,
'vf_clip_param': 50.0,
'vf_loss_coeff': 0.05,
'vf_share_layers': False} # define a custom planner policy configuration.
)
}

policy_mapping_fun = lambda i: "a" if str(i).isdigit() else "p"

policies_to_train = ["a", "p"]

trainer_config = {
"multiagent": {
"policies": policies,
"policies_to_train": policies_to_train,
"policy_mapping_fn": policy_mapping_fun,
}
}

trainer_config.update(
{
"num_workers": 5,
"num_envs_per_worker": 1,
# Other training parameters
"train_batch_size": 4000,
"sgd_minibatch_size": 4000,
"num_sgd_iter": 1
}
)

env_config = {
"env_config_dict": env_config_dict,
"num_envs_per_worker": trainer_config.get('num_envs_per_worker'),
}

trainer_config.update(
{
"env_config": env_config
}
)

ray.init()

trainer = PPOTrainer(env=RLlibEnvWrapper, config=trainer_config )

However, the following error is generated:

RayActorError: The actor died because of an error raised in its creation task, ray::RolloutWorker.init() (pid=1353, ip=127.0.0.1, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x16a373790>)
File "/Users/asataryd/miniforge3/envs/modified-ai-economist/lib/python3.10/site-packages/ray/rllib/evaluation/rollout_worker.py", line 613, in init
self._build_policy_map(
File "/Users/asataryd/miniforge3/envs/modified-ai-economist/lib/python3.10/site-packages/ray/rllib/evaluation/rollout_worker.py", line 1784, in _build_policy_map
self.policy_map.create_policy(
File "/Users/asataryd/miniforge3/envs/modified-ai-economist/lib/python3.10/site-packages/ray/rllib/policy/policy_map.py", line 123, in create_policy
self[policy_id] = create_policy_for_framework(
File "/Users/asataryd/miniforge3/envs/modified-ai-economist/lib/python3.10/site-packages/ray/rllib/utils/policy.py", line 71, in create_policy_for_framework
return policy_class(
File "/Users/asataryd/miniforge3/envs/modified-ai-economist/lib/python3.10/site-packages/ray/rllib/algorithms/ppo/ppo_tf_policy.py", line 83, in init
base.init(
File "/Users/asataryd/miniforge3/envs/modified-ai-economist/lib/python3.10/site-packages/ray/rllib/policy/dynamic_tf_policy_v2.py", line 81, in init
self.model = self.make_model()
File "/Users/asataryd/miniforge3/envs/modified-ai-economist/lib/python3.10/site-packages/ray/rllib/policy/dynamic_tf_policy_v2.py", line 221, in make_model
return ModelCatalog.get_model_v2(
File "/Users/asataryd/miniforge3/envs/modified-ai-economist/lib/python3.10/site-packages/ray/rllib/models/catalog.py", line 587, in get_model_v2
raise ValueError(
ValueError: It looks like you are still using <rllib.tf_models.KerasConvLSTM object at 0x16a6f2ec0>.register_variables() to register your model's weights. This is no longer required, but if you are still calling this method at least once, you must make sure to register all created variables properly. The missing variables are {<Reference wrapping <tf.Variable 'a_wk4/conv2D_2_pol/kernel:0' shape=(3, 3, 16, 32) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/value/bias:0' shape=(1,) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/conv2D_1_pol/bias:0' shape=(16,) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/layer_norm_pol/beta:0' shape=(128,) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/conv2D_1_pol/kernel:0' shape=(3, 3, 26, 16) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/lstm_pol/lstm_cell/bias:0' shape=(512,) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/lstm_val/lstm_cell_1/bias:0' shape=(512,) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/value/kernel:0' shape=(128, 1) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/dense2_pol/bias:0' shape=(128,) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/logits/bias:0' shape=(80,) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/conv2D_2_val/kernel:0' shape=(3, 3, 16, 32) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/layer_norm_val/beta:0' shape=(128,) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/lstm_pol/lstm_cell/recurrent_kernel:0' shape=(128, 512) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/logits/kernel:0' shape=(128, 80) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/dense1_val/kernel:0' shape=(234, 128) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/dense1_val/bias:0' shape=(128,) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/embedding_pol/embeddings:0' shape=(100, 4) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/layer_norm_pol/gamma:0' shape=(128,) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/conv2D_2_val/bias:0' shape=(32,) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/conv2D_2_pol/bias:0' shape=(32,) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/conv2D_1_val/bias:0' shape=(16,) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/lstm_val/lstm_cell_1/recurrent_kernel:0' shape=(128, 512) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/dense2_pol/kernel:0' shape=(128, 128) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/lstm_pol/lstm_cell/kernel:0' shape=(128, 512) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/conv2D_1_val/kernel:0' shape=(3, 3, 26, 16) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/dense2_val/kernel:0' shape=(128, 128) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/dense2_val/bias:0' shape=(128,) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/dense1_pol/kernel:0' shape=(234, 128) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/dense1_pol/bias:0' shape=(128,) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/embedding_val/embeddings:0' shape=(100, 4) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/layer_norm_val/gamma:0' shape=(128,) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/lstm_val/lstm_cell_1/kernel:0' shape=(128, 512) dtype=float32>>}, and you only registered {<tf.Variable 'a_wk4/conv2D_2_pol/kernel:0' shape=(3, 3, 16, 32) dtype=float32>, <tf.Variable 'a_wk4/conv2D_1_pol/bias:0' shape=(16,) dtype=float32>, <tf.Variable 'a_wk4/value/bias:0' shape=(1,) dtype=float32>, <tf.Variable 'a_wk4/conv2D_1_pol/kernel:0' shape=(3, 3, 26, 16) dtype=float32>, <tf.Variable 'a_wk4/layer_norm_pol/beta:0' shape=(128,) dtype=float32>, <tf.Variable 'a_wk4/lstm_pol/lstm_cell/bias:0' shape=(512,) dtype=float32>, <tf.Variable 'a_wk4/lstm_val/lstm_cell_1/bias:0' shape=(512,) dtype=float32>, <tf.Variable 'a_wk4/value/kernel:0' shape=(128, 1) dtype=float32>, <tf.Variable 'a_wk4/dense2_pol/bias:0' shape=(128,) dtype=float32>, <tf.Variable 'a_wk4/conv2D_2_val/kernel:0' shape=(3, 3, 16, 32) dtype=float32>, <tf.Variable 'a_wk4/logits/bias:0' shape=(80,) dtype=float32>, <tf.Variable 'a_wk4/layer_norm_val/beta:0' shape=(128,) dtype=float32>, <tf.Variable 'a_wk4/lstm_pol/lstm_cell/recurrent_kernel:0' shape=(128, 512) dtype=float32>, <tf.Variable 'a_wk4/logits/kernel:0' shape=(128, 80) dtype=float32>, <tf.Variable 'a_wk4/dense1_val/kernel:0' shape=(234, 128) dtype=float32>, <tf.Variable 'a_wk4/dense1_val/bias:0' shape=(128,) dtype=float32>, <tf.Variable 'a_wk4/embedding_pol/embeddings:0' shape=(100, 4) dtype=float32>, <tf.Variable 'a_wk4/layer_norm_pol/gamma:0' shape=(128,) dtype=float32>, <tf.Variable 'a_wk4/conv2D_2_val/bias:0' shape=(32,) dtype=float32>, <tf.Variable 'a_wk4/conv2D_2_pol/bias:0' shape=(32,) dtype=float32>, <tf.Variable 'a_wk4/conv2D_1_val/bias:0' shape=(16,) dtype=float32>, <tf.Variable 'a_wk4/lstm_val/lstm_cell_1/recurrent_kernel:0' shape=(128, 512) dtype=float32>, <tf.Variable 'a_wk4/dense2_pol/kernel:0' shape=(128, 128) dtype=float32>, <tf.Variable 'a_wk4/lstm_pol/lstm_cell/kernel:0' shape=(128, 512) dtype=float32>, <tf.Variable 'a_wk4/conv2D_1_val/kernel:0' shape=(3, 3, 26, 16) dtype=float32>, <tf.Variable 'a_wk4/dense2_val/kernel:0' shape=(128, 128) dtype=float32>, <tf.Variable 'a_wk4/dense2_val/bias:0' shape=(128,) dtype=float32>, <tf.Variable 'a_wk4/dense1_pol/kernel:0' shape=(234, 128) dtype=float32>, <tf.Variable 'a_wk4/dense1_pol/bias:0' shape=(128,) dtype=float32>, <tf.Variable 'a_wk4/embedding_val/embeddings:0' shape=(100, 4) dtype=float32>, <tf.Variable 'a_wk4/layer_norm_val/gamma:0' shape=(128,) dtype=float32>, <tf.Variable 'a_wk4/lstm_val/lstm_cell_1/kernel:0' shape=(128, 512) dtype=float32>}. Did you forget to call register_variables() on some of the variables in question?

This happen even if I use RandomAction network. Also, in your provided notebook, you have not used these custom modelsI. Moreover, I think this is most probably not related to the changes that I made in the AI-Economist but it might be related to the compatibility issues of different versions of the modules that I have in my environment. I greatly appreciate any comments.

By the way, please let me know if you need further information. If it is required I would be happy to share the GitHub repository of my own version of AI-Economist with you. Many thanks in advance!

[sa1g]

Hi there, as you noticed there are many compatibility issues in AI-Economist...
To fix your problem you just have to comment register_variables(). Note that many other issues are ahead of this one. I think there are compatibility issues between rllib, tensorflow + non full retro-compatibility when you start upgrading stuff.

[aslansd]

Hi @sa1g, thanks, I already solved this problem. However, I have now other issue which I posted here:

https://discuss.ray.io/t/using-custom-neural-network-in-rllib/8094

I was wondering if you could be helpful in this case too! Many thanks in advance!