Poor introduction to dopamine #197
Comments
|
sorry about this! we made some changes to the dopamine API recently, which
probably broke a few things. i'll look into this next week and have it
resolved quickly.
apologies for the inconvenience, but thanks for raising it!
cheers,
…-psc
On Thu, May 19, 2022 at 9:51 AM Simon Taeter ***@***.***> wrote:
So I tried following the overview
<https://google.github.io/dopamine/docs/> and - to be honest - this has
been a pain.
The entry point to the standard Atari 2600 experiment is [dopamine/discrete_domains/train.py](https://github.com/google/dopamine/blob/master/dopamine/discrete_domains/train.py). To run the basic DQN agent,
python -um dopamine.discrete_domains.train \
--base_dir /tmp/dopamine_runs \
--gin_files dopamine/agents/dqn/configs/dqn.gin
This raises loads of errors; first all the missing imports (I fixed those
using pipreqs) then Gym complaining it doesn't distribute ROMs anymore:
gym.error.Error: We're Unable to find the game "Pong". Note: Gym no longer distributes ROMs. If you own a license to use the necessary ROMs for research purposes you can download them via `pip install gym[accept-rom-license]`. Otherwise, you should try importing "Pong" via the command `ale-import-roms`. If you believe this is a mistake perhaps your copy of "Pong" is unsupported. To check if this is the case try providing the environment variable `PYTHONWARNINGS=default::ImportWarning:ale_py.roms`. For more information see: https://github.com/mgbellemare/Arcade-Learning-Environment#rom-management
I realized all of this should run onto the core and atari containers that
I created but I don't see where this is explained.
So I started looking at the colabs hoping to find some more infos but I
got loads of errors...
agents.ipynb
<https://colab.research.google.com/github/google/dopamine/blob/master/dopamine/colab/agents.ipynb>
in Load baseline data
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
[<ipython-input-6-76246b013715>](https://localhost:8080/#) in <module>()
1 # @title Load baseline data
2 get_ipython().system('gsutil -q -m cp -R gs://download-dopamine-rl/preprocessed-benchmarks/* /content/')
----> 3 experimental_data = colab_utils.load_baselines('/content')
1 frames
[/usr/lib/python3.7/copyreg.py](https://localhost:8080/#) in _reconstructor(cls, base, state)
41 def _reconstructor(cls, base, state):
42 if base is object:
---> 43 obj = object.__new__(cls)
44 else:
45 obj = base.__new__(cls, state)
TypeError: object.__new__(BlockManager) is not safe, use BlockManager.__new__()
load_statistics.ipynb
<https://colab.research.google.com/github/google/dopamine/blob/master/dopamine/colab/load_statistics.ipynb>
in Load the baseline data
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-4-fed9ae80a060> in <module>()
2
3 get_ipython().system('gsutil -q -m cp -R gs://download-dopamine-rl/preprocessed-benchmarks/* /content/')
----> 4 experimental_data = colab_utils.load_baselines('/content')
1 frames
/usr/lib/python3.7/copyreg.py in _reconstructor(cls, base, state)
41 def _reconstructor(cls, base, state):
42 if base is object:
---> 43 obj = object.__new__(cls)
44 else:
45 obj = base.__new__(cls, state)
TypeError: object.__new__(BlockManager) is not safe, use BlockManager.__new__()
agent_visualizer.ipynb
<https://colab.research.google.com/github/google/dopamine/blob/master/dopamine/colab/agent_visualizer.ipynb>
in Generate the video
pygame 2.1.2 (SDL 2.0.16, Python 3.7.13)
Hello from the pygame community. https://www.pygame.org/contribute.html
WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/tensorflow/python/compat/v2_compat.py:107: disable_resource_variables (from tensorflow.python.ops.variable_scope) is deprecated and will be removed in a future version.
Instructions for updating:
non-resource variables are not supported in the long term
---------------------------------------------------------------------------
Exception Traceback (most recent call last)
[<ipython-input-3-05aad0365c1e>](https://localhost:8080/#) in <module>()
4 example_viz_lib.run(agent='rainbow', game='SpaceInvaders', num_steps=num_steps,
5 root_dir='/tmp/agent_viz', restore_ckpt='/tmp/tf_ckpt-199',
----> 6 use_legacy_checkpoint=True)
15 frames
[/usr/local/lib/python3.7/dist-packages/atari_py/games.py](https://localhost:8080/#) in get_game_path(game_name)
18 path = os.path.join(_games_dir, game_name) + ".bin"
19 if not os.path.exists(path):
---> 20 raise Exception('ROM is missing for %s, see https://github.com/openai/atari-py#roms for instructions' % (game_name,))
21 return path
22
Exception: ROM is missing for space_invaders, see https://github.com/openai/atari-py#roms for instructions
In call to configurable 'create_atari_environment' (<function create_atari_environment at 0x7f4b9399da70>)
In call to configurable 'Runner' (<class 'dopamine.discrete_domains.run_experiment.Runner'>)
agent_visualizer.ipynb
<https://colab.research.google.com/github/google/dopamine/blob/master/dopamine/colab/jax_agent_visualizer.ipynb>
in Generate video
WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/tensorflow/python/compat/v2_compat.py:107: disable_resource_variables (from tensorflow.python.ops.variable_scope) is deprecated and will be removed in a future version.
Instructions for updating:
non-resource variables are not supported in the long term
---------------------------------------------------------------------------
Exception Traceback (most recent call last)
[<ipython-input-5-527301e7cc72>](https://localhost:8080/#) in <module>()
12 gin.parse_config(config)
13 runner = example_viz_lib.MyRunner(base_dir, '/tmp/ckpt.199',
---> 14 AGENT_CREATORS[agent])
15 runner.visualize(base_dir / 'images', num_global_steps=200)
13 frames
[/usr/local/lib/python3.7/dist-packages/atari_py/games.py](https://localhost:8080/#) in get_game_path(game_name)
18 path = os.path.join(_games_dir, game_name) + ".bin"
19 if not os.path.exists(path):
---> 20 raise Exception('ROM is missing for %s, see https://github.com/openai/atari-py#roms for instructions' % (game_name,))
21 return path
22
Exception: ROM is missing for breakout, see https://github.com/openai/atari-py#roms for instructions
In call to configurable 'create_atari_environment' (<function create_atari_environment at 0x7f9a4a7ff200>)
In call to configurable 'Runner' (<class 'dopamine.discrete_domains.run_experiment.Runner'>)
happily tensorboard.ipynb
<https://colab.research.google.com/github/google/dopamine/blob/master/dopamine/colab/tensorboard.ipynb>
functions properly and cartpole.ipynb
<https://colab.research.google.com/github/google/dopamine/blob/master/dopamine/colab/cartpole.ipynb>
too.
Overall, all of this makes for a very poor introduction to dopamine.
—
Reply to this email directly, view it on GitHub
<#197>, or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AE3CCMNBOKQ7OCA5RRT3OCLVKZBMJANCNFSM5WMJTZGQ>
.
You are receiving this because you are subscribed to this thread.Message
ID: ***@***.***>
|
|
hi simon,
the metrics issue should be resolved now, and for the ROMs you should be
able to get that working with
`pip install gym[accept-rom-license]`
(this is an issue that is separate from us).
i'll look into the colab issues next week, but hopefully this unblocks you
for now!
…On Thu, May 19, 2022 at 9:51 AM Simon Taeter ***@***.***> wrote:
So I tried following the overview
<https://google.github.io/dopamine/docs/> and - to be honest - this has
been a pain.
The entry point to the standard Atari 2600 experiment is [dopamine/discrete_domains/train.py](https://github.com/google/dopamine/blob/master/dopamine/discrete_domains/train.py). To run the basic DQN agent,
python -um dopamine.discrete_domains.train \
--base_dir /tmp/dopamine_runs \
--gin_files dopamine/agents/dqn/configs/dqn.gin
This raises loads of errors; first all the missing imports (I fixed those
using pipreqs) then Gym complaining it doesn't distribute ROMs anymore:
gym.error.Error: We're Unable to find the game "Pong". Note: Gym no longer distributes ROMs. If you own a license to use the necessary ROMs for research purposes you can download them via `pip install gym[accept-rom-license]`. Otherwise, you should try importing "Pong" via the command `ale-import-roms`. If you believe this is a mistake perhaps your copy of "Pong" is unsupported. To check if this is the case try providing the environment variable `PYTHONWARNINGS=default::ImportWarning:ale_py.roms`. For more information see: https://github.com/mgbellemare/Arcade-Learning-Environment#rom-management
I realized all of this should run onto the core and atari containers that
I created but I don't see where this is explained.
So I started looking at the colabs hoping to find some more infos but I
got loads of errors...
agents.ipynb
<https://colab.research.google.com/github/google/dopamine/blob/master/dopamine/colab/agents.ipynb>
in Load baseline data
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
[<ipython-input-6-76246b013715>](https://localhost:8080/#) in <module>()
1 # @title Load baseline data
2 get_ipython().system('gsutil -q -m cp -R gs://download-dopamine-rl/preprocessed-benchmarks/* /content/')
----> 3 experimental_data = colab_utils.load_baselines('/content')
1 frames
[/usr/lib/python3.7/copyreg.py](https://localhost:8080/#) in _reconstructor(cls, base, state)
41 def _reconstructor(cls, base, state):
42 if base is object:
---> 43 obj = object.__new__(cls)
44 else:
45 obj = base.__new__(cls, state)
TypeError: object.__new__(BlockManager) is not safe, use BlockManager.__new__()
load_statistics.ipynb
<https://colab.research.google.com/github/google/dopamine/blob/master/dopamine/colab/load_statistics.ipynb>
in Load the baseline data
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-4-fed9ae80a060> in <module>()
2
3 get_ipython().system('gsutil -q -m cp -R gs://download-dopamine-rl/preprocessed-benchmarks/* /content/')
----> 4 experimental_data = colab_utils.load_baselines('/content')
1 frames
/usr/lib/python3.7/copyreg.py in _reconstructor(cls, base, state)
41 def _reconstructor(cls, base, state):
42 if base is object:
---> 43 obj = object.__new__(cls)
44 else:
45 obj = base.__new__(cls, state)
TypeError: object.__new__(BlockManager) is not safe, use BlockManager.__new__()
agent_visualizer.ipynb
<https://colab.research.google.com/github/google/dopamine/blob/master/dopamine/colab/agent_visualizer.ipynb>
in Generate the video
pygame 2.1.2 (SDL 2.0.16, Python 3.7.13)
Hello from the pygame community. https://www.pygame.org/contribute.html
WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/tensorflow/python/compat/v2_compat.py:107: disable_resource_variables (from tensorflow.python.ops.variable_scope) is deprecated and will be removed in a future version.
Instructions for updating:
non-resource variables are not supported in the long term
---------------------------------------------------------------------------
Exception Traceback (most recent call last)
[<ipython-input-3-05aad0365c1e>](https://localhost:8080/#) in <module>()
4 example_viz_lib.run(agent='rainbow', game='SpaceInvaders', num_steps=num_steps,
5 root_dir='/tmp/agent_viz', restore_ckpt='/tmp/tf_ckpt-199',
----> 6 use_legacy_checkpoint=True)
15 frames
[/usr/local/lib/python3.7/dist-packages/atari_py/games.py](https://localhost:8080/#) in get_game_path(game_name)
18 path = os.path.join(_games_dir, game_name) + ".bin"
19 if not os.path.exists(path):
---> 20 raise Exception('ROM is missing for %s, see https://github.com/openai/atari-py#roms for instructions' % (game_name,))
21 return path
22
Exception: ROM is missing for space_invaders, see https://github.com/openai/atari-py#roms for instructions
In call to configurable 'create_atari_environment' (<function create_atari_environment at 0x7f4b9399da70>)
In call to configurable 'Runner' (<class 'dopamine.discrete_domains.run_experiment.Runner'>)
agent_visualizer.ipynb
<https://colab.research.google.com/github/google/dopamine/blob/master/dopamine/colab/jax_agent_visualizer.ipynb>
in Generate video
WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/tensorflow/python/compat/v2_compat.py:107: disable_resource_variables (from tensorflow.python.ops.variable_scope) is deprecated and will be removed in a future version.
Instructions for updating:
non-resource variables are not supported in the long term
---------------------------------------------------------------------------
Exception Traceback (most recent call last)
[<ipython-input-5-527301e7cc72>](https://localhost:8080/#) in <module>()
12 gin.parse_config(config)
13 runner = example_viz_lib.MyRunner(base_dir, '/tmp/ckpt.199',
---> 14 AGENT_CREATORS[agent])
15 runner.visualize(base_dir / 'images', num_global_steps=200)
13 frames
[/usr/local/lib/python3.7/dist-packages/atari_py/games.py](https://localhost:8080/#) in get_game_path(game_name)
18 path = os.path.join(_games_dir, game_name) + ".bin"
19 if not os.path.exists(path):
---> 20 raise Exception('ROM is missing for %s, see https://github.com/openai/atari-py#roms for instructions' % (game_name,))
21 return path
22
Exception: ROM is missing for breakout, see https://github.com/openai/atari-py#roms for instructions
In call to configurable 'create_atari_environment' (<function create_atari_environment at 0x7f9a4a7ff200>)
In call to configurable 'Runner' (<class 'dopamine.discrete_domains.run_experiment.Runner'>)
happily tensorboard.ipynb
<https://colab.research.google.com/github/google/dopamine/blob/master/dopamine/colab/tensorboard.ipynb>
functions properly and cartpole.ipynb
<https://colab.research.google.com/github/google/dopamine/blob/master/dopamine/colab/cartpole.ipynb>
too.
Overall, all of this makes for a very poor introduction to dopamine.
—
Reply to this email directly, view it on GitHub
<#197>, or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AE3CCMNBOKQ7OCA5RRT3OCLVKZBMJANCNFSM5WMJTZGQ>
.
You are receiving this because you are subscribed to this thread.Message
ID: ***@***.***>
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
So I tried following the overview and - to be honest - this has been a pain.
This raises loads of errors:
pipreqs).from dopamine.metricstofrom metrics.I believe all of this should run onto the core and atari containers that I created following the readme but I don't see where this is explained...
So I started looking at the colabs hoping to find some more infos but I got loads of errors...
agents.ipynb in
Load baseline dataload_statistics.ipynb in
Load the baseline dataagent_visualizer.ipynb in
Generate the videoagent_visualizer.ipynb in
Generate videohappily tensorboard.ipynb functions properly and cartpole.ipynb too.
Overall, all of this makes for a very poor introduction to dopamine.
The text was updated successfully, but these errors were encountered: