docs: update readme with continuous system details #1050

RuanJohn · 2024-03-01T09:55:49Z

What?

Update the readme to mention that we now have support for continuous action space environments.

sash-a

Love it 🔥

…readme-for-cont-systems

WiemKhlifi

Lekker 🎉

The base branch was changed.

sash-a

Some very minor things

sash-a · 2024-03-01T13:33:13Z

README.md

- 🥑 **Implementations of MARL algorithms**: Implementations of multi-agent PPO systems that follow both the Centralised Training with Decentralised Execution (CTDE) and Decentralised Training with Decentralised Execution (DTDE) MARL paradigms.
- 🍬 **Environment Wrappers**: Example wrappers for mapping Jumanji environments to an environment that is compatible with Mava. At the moment, we support [Robotic Warehouse][jumanji_rware] and [Level-Based Foraging][jumanji_lbf] with plans to support more environments soon. We have also recently added support for the SMAX environment from [JaxMARL][jaxmarl].
+- 🥑 **Implementations of MARL algorithms**: Implementations of multi-agent PPO systems that follow both the Centralised Training with Decentralised Execution (CTDE) and Decentralised Training with Decentralised Execution (DTDE) MARL paradigms with support for continuous and discrete action space environments.
+- 🍬 **Environment Wrappers**: Example wrappers for mapping Jumanji environments to an environment that is compatible with Mava. At the moment, we support [Robotic Warehouse][jumanji_rware] and [Level-Based Foraging][jumanji_lbf] with plans to support more environments soon. We have also recently added support for the SMAX and MaBrax environments from [JaxMARL][jaxmarl].


while we're here can we add connector, cleaner, matrax, CVRP and gigastep? Maybe this should become a table of envs we support 🤔

➕ Something like this with action type:

This table outlines the environments supported by our library ect.... | Library/Site | Environment Supported | Action Type | |--------------|-----------------------|-------------| | Jumanji | LBF | Discrete | | | RWARE | Discrete | | | CVRP | Discrete | | Jaxmarl | MABRAX | Continuous | | | SMAX | Continuous |

If we will mention the different environments I think we need to add a comment that we are working on verifying the performance of mava on them because we may confirm that all works perfectly once we make a full benchmark 🙏

Update: Since the connector env is using a CNN network we may add another column citing the corresponding network for each env and mention that for action space we need to check the corresponding ActionHead as well in the network.yaml file:

| Library/Site | Environment Supported | Action Space | Network |--------------|-----------------------|-------------|------------- | Jumanji | LBF | Discrete | Default (mlp) | | RWARE | Discrete | Default | | Connector | Discrete | CNN | Jaxmarl | MABRAX | Continuous | Default | | SMAX | Continuous | Default

sash-a · 2024-03-01T13:34:32Z

README.md

+python mava/systems/ppo/ff_ippo.py env=rware env/scenario=tiny-4ag
+```
+
+To toggle between continuous and discrete systems, simply select the continuous action space network head. To run the same system on an `MaBrax` environment make the follow config updates from the terminal:


Just to make it clear that MaBrax is continuous

Suggested change

To toggle between continuous and discrete systems, simply select the continuous action space network head. To run the same system on an `MaBrax` environment make the follow config updates from the terminal:

To toggle between continuous and discrete systems, simply select the continuous action space network head. To run the same system on a continuous environment, like `MaBrax`, make the follow config updates from the terminal:

Suggested change

To toggle between continuous and discrete systems, simply select the continuous action space network head. To run the same system on an `MaBrax` environment make the follow config updates from the terminal:

To toggle between continuous and discrete systems, simply select the continuous action space network head. To run the same system on a `MaBrax` environment make the following config updates from the terminal:

sash-a · 2024-03-01T13:35:05Z

README.md

@@ -194,10 +200,8 @@ Please read our [contributing docs](docs/CONTRIBUTING.md) for details on how to
 We plan to iteratively expand Mava in the following increments:

 - 🌴 Support for more environments.
- 🔁 More robust recurrent systems.


Can we also close the issue Edan raised around this

WiemKhlifi

➕ Sasha's suggestions

WiemKhlifi · 2024-03-01T13:43:40Z

README.md

@@ -142,7 +142,7 @@ Furthermore, we illustrate the speed of Mava by showing the steps per second as

 ## Code Philosophy 🧘

-The current code in Mava is adapted from [PureJaxRL][purejaxrl] which provides high-quality single-file implementations with research-friendly features. In turn, PureJaxRL is inspired by the code philosophy from [CleanRL][cleanrl]. Along this vein of easy-to-use and understandable RL codebases, Mava is not designed to be a modular library and is not meant to be imported. Our repository focuses on simplicity and clarity in its implementations while utilising the advantages offered by JAX such as `pmap` and `vmap`, making it an excellent resource for researchers and practitioners to build upon.
+The current code in Mava is adapted from [PureJaxRL][purejaxrl] which provides high-quality single-file implementations with research-friendly features. In turn, PureJaxRL is inspired by the code philosophy from [CleanRL][cleanrl]. Along this vein of easy-to-use and understandable RL codebases, Mava is not designed to be a modular library and is not meant to be imported. Our repository focuses on simplicity and clarity in its implementations while utilising the advantages offered by JAX such as `pmap` and `vmap`, making it an excellent resource for researchers and practitioners to build upon. A notable difference between Mava and other single-file libraries is that Mava makes use of abstraction where relevant. In particular, this is done for network and environment creation.


Maybe network -> neural network or something clearer if exists

WiemKhlifi · 2024-03-01T13:45:58Z

README.md

+python mava/systems/ppo/ff_ippo.py env=rware env/scenario=tiny-4ag
+```
+
+To toggle between continuous and discrete systems, simply select the continuous action space network head. To run the same system on an `MaBrax` environment make the follow config updates from the terminal:


Suggested change

To toggle between continuous and discrete systems, simply select the continuous action space network head. To run the same system on an `MaBrax` environment make the follow config updates from the terminal:

To toggle between continuous and discrete systems, simply select the continuous action space network head. To run the same system on a `MaBrax` environment make the following config updates from the terminal:

docs: update readme with continuous system details

4d7cad9

RuanJohn added the documentation Improvements or additions to documentation label Mar 1, 2024

RuanJohn self-assigned this Mar 1, 2024

RuanJohn requested review from arnupretorius, DriesSmit, jcformanek, siddarthsingh1, sash-a, OmaymaMahjoub, ulricharmel, callumtilbury and WiemKhlifi as code owners March 1, 2024 09:55

pull-request-size bot added the size/XS label Mar 1, 2024

sash-a previously approved these changes Mar 1, 2024

View reviewed changes

RuanJohn added 2 commits March 1, 2024 12:35

Merge branch 'feat/remove_vampping_cont_act' into maintenance/update-…

3c1d614

…readme-for-cont-systems

docs: example on running a continuous action space system

fc5a778

pull-request-size bot added size/S and removed size/XS labels Mar 1, 2024

docs: typo fix in continuous action space example

4533408

WiemKhlifi previously approved these changes Mar 1, 2024

View reviewed changes

Base automatically changed from feat/remove_vampping_cont_act to develop March 1, 2024 12:18

RuanJohn and others added 2 commits March 1, 2024 14:28

Merge branch 'develop' into maintenance/update-readme-for-cont-systems

66fb2a4

docs: minor readme fix for abstraction description

9df3bcd

sash-a requested changes Mar 1, 2024

View reviewed changes

WiemKhlifi suggested changes Mar 1, 2024

View reviewed changes

WiemKhlifi added 2 commits March 6, 2024 10:21

Merge branch 'develop' into maintenance/update-readme-for-cont-systems

7659bcc

Merge branch 'develop' into maintenance/update-readme-for-cont-systems

529ecc5

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

docs: update readme with continuous system details #1050

docs: update readme with continuous system details #1050

RuanJohn commented Mar 1, 2024

sash-a left a comment

WiemKhlifi left a comment

sash-a left a comment

sash-a Mar 1, 2024

WiemKhlifi Mar 1, 2024 •

edited

OmaymaMahjoub Mar 3, 2024

WiemKhlifi Mar 5, 2024

sash-a Mar 1, 2024

WiemKhlifi Mar 1, 2024

sash-a Mar 1, 2024

WiemKhlifi left a comment

WiemKhlifi Mar 1, 2024 •

edited

WiemKhlifi Mar 1, 2024

	To toggle between continuous and discrete systems, simply select the continuous action space network head. To run the same system on an `MaBrax` environment make the follow config updates from the terminal:
	To toggle between continuous and discrete systems, simply select the continuous action space network head. To run the same system on a continuous environment, like `MaBrax`, make the follow config updates from the terminal:

docs: update readme with continuous system details #1050

Are you sure you want to change the base?

docs: update readme with continuous system details #1050

Conversation

RuanJohn commented Mar 1, 2024

What?

sash-a left a comment

Choose a reason for hiding this comment

WiemKhlifi left a comment

Choose a reason for hiding this comment

sash-a left a comment

Choose a reason for hiding this comment

sash-a Mar 1, 2024

Choose a reason for hiding this comment

WiemKhlifi Mar 1, 2024 • edited

Choose a reason for hiding this comment

OmaymaMahjoub Mar 3, 2024

Choose a reason for hiding this comment

WiemKhlifi Mar 5, 2024

Choose a reason for hiding this comment

sash-a Mar 1, 2024

Choose a reason for hiding this comment

WiemKhlifi Mar 1, 2024

Choose a reason for hiding this comment

sash-a Mar 1, 2024

Choose a reason for hiding this comment

WiemKhlifi left a comment

Choose a reason for hiding this comment

WiemKhlifi Mar 1, 2024 • edited

Choose a reason for hiding this comment

WiemKhlifi Mar 1, 2024

Choose a reason for hiding this comment

WiemKhlifi Mar 1, 2024 •

edited

WiemKhlifi Mar 1, 2024 •

edited