Benchmark downloader and processor

[TimotheeMathieu]

It would be nice to have a function to fetch and process benchmark results from various places like stablebaselines3-zoo or cleanrl, so they can be used easily to compare with the results of a rlberry user.

[TimotheeMathieu]

Google Atari bucket: https://console.cloud.google.com/storage/brow ser/rl-benchmark-data/ALE
SB3: https://github.com/DLR-RM/rl-baselines3-zoo/tree/master/logs/benchmark WARNING only one seed
cleanrl: https://wandb.ai/openrlbenchmark/openrlbenchmark/reportlist (no idea what data they have)

There are other.

Goal is to fit entry format for comparator (https://rlberry-py.github.io/rlberry/basics/comparison.html).

@YannBerthelot , @KohlerHECTOR, @mmcenta, @AleShi94 do you have any other idea of benchmark that we could use ?

[JulienT01]

link to Google Atari bucket: https://console.cloud.google.com/storage/browser/rl-benchmark-data/atari_100k
associated colab : https://colab.research.google.com/drive/1a0pSD-1tWhMmeJeeoyZM1A-HCW3yf1xR?usp=sharing#scrollTo=eV8ci5frgPFO
associated paper : https://arxiv.org/pdf/2108.13264.pdf

[TimotheeMathieu]

There is also https://github.com/DLR-RM/rl-trained-agents/
And maybe we can also have importer for HuggingFace

[YannBerthelot]

Just came out: https://arxiv.org/abs/2402.03046, associated github : https://github.com/openrlbenchmark/openrlbenchmark Le mer. 7 févr. 2024 à 09:44, TimotheeMathieu ***@***.***> a écrit :

…

There is also https://github.com/DLR-RM/rl-trained-agents/ — Reply to this email directly, view it on GitHub <#395 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALWSWPWWFAETWHGN4GSTPGDYSM5FTAVCNFSM6AAAAAA752V552VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMZRGU2DENBYGM> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[KohlerHECTOR]

La classe @YannBerthelot !!! Viens tu présentes aujourd'hui ?