A curated list of open-ended learning AI resources. The aim of open-ended algorithms is to keep on inventing new and ever-more complex tasks and solving them continually, even endlessly. From the invention of the wheel, to farming, vaccines, computers, and even rock and roll. These so-far uniquely human advancements and discoveries are the hallmark of civilization. What does AI need to possess to discover such new paradigms, as only humans have until now? Let's take a look at our progress on this frontier. How close are we to AGI?
When submitting a pull request, please put the new paper at the correct chronological position as the following format:
* **Paper Title** <br>
*Author(s)* <br>
Conference, Year. [[Paper]](link) [[Code]](link) [[Website]](link)
-
Minimal Criterion Coevolution: A New Approach to Open-Ended Search
Jonathan C. Brant, Kenneth O. Stanley
GECCO, 2017. [Paper] [Code] -
Paired Open-Ended Trailblazer (POET): Endlessly Generating Increasingly Complex and Diverse Learning Environments and Their Solutions
Rui Wang, Joel Lehman, Jeff Clune, Kenneth O. Stanley
GECCO, 2019. [Paper] [Code] [Website] -
Enhanced POET: Open-Ended Reinforcement Learning through Unbounded Invention of Learning Challenges and their Solutions
Rui Wang, Joel Lehman, Aditya Rawal, Jiale Zhi, Yulun Li, Jeff Clune, Kenneth O. Stanley
ICML, 2020. [Paper] [Code] [Website] -
Co-generation of game levels and game-playing agents
Aaron Dharna, Julian Togelius, L.B.Soros
AIIDE 2020. [Paper] [Code] -
Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Design
Michael Dennis, Natasha Jaques, Eugene Vinitsky, Alexandre Bayen, Stuart Russell, Andrew Critch, Sergey Levine
NeurIPS, 2020. [Paper] [Code] [Website] -
Co-optimising Robot Morphology and Controller in a Simulated Open-Ended Environment
Emma Hjellbrekke Stensby, Kai Olav Ellefsen, Kyrre Glette
EvoStar 2021. [Paper] [Code] -
Prioritized Level Replay
Minqi Jiang, Edward Grefenstette, Tim Rocktäschel
ICML, 2021. [Paper] [Code] -
Replay-Guided Adversarial Environment Design
Minqi Jiang*, Michael Dennis*, Jack Parker-Holder, Jakob Foerster, Edward Grefenstette, Tim Rocktäschel
NeurIPS, 2021. [Paper] [Code] -
Environment Generation for Zero-Shot Compositional Reinforcement Learning
Izzeddin Gur, Natasha Jaques, Yingjie Miao, Jongwook Choi, Manoj Tiwari, Honglak Lee, Aleksandra Faust
NeurIPS, 2021. [Paper] -
MiniHack the Planet: A Sandbox for Open-Ended Reinforcement Learning Research
Mikayel Samvelyan, Robert Kirk, Vitaly Kurin, Jack Parker-Holder, Minqi Jiang, Eric Hambro, Fabio Petroni, Heinrich Küttler, Edward Grefenstette, Tim Rocktäschel
NeurIPS, 2021. [Paper] [Code] [Website] -
Open-Ended Learning Leads to Generally Capable Agents
Open Ended Learning Team, Adam Stooke, Anuj Mahajan, Catarina Barros, Charlie Deck, Jakob Bauer, Jakub Sygnowski, Maja Trebacz, Max Jaderberg, Michael Mathieu, Nat McAleese, Nathalie Bradley-Schmieg, Nathaniel Wong, Nicolas Porcel, Roberta Raileanu, Steph Hughes-Fitt, Valentin Dalibard, Wojciech Marian Czarnecki
arXiv, 2021. [Paper] [Website] -
SPOTTER: Extending Symbolic Planning Operators through Targeted Reinforcement Learning
Vasanth Sarathy, Daniel Kasenberg, Shivam Goel, Jivko Sinapov, Matthias Scheutz
arXiv, 2021. [Paper] [Code] -
EvoCraft: A New Challenge for Open-Endedness
Djordje Grbic, Rasmus Berg Palm, Elias Najarro, Claire Glanois, Sebastian Risi
EvoStar, 2021. [Paper] [Website] -
Video Games as a Testbed for Open-Ended Phenomena
Sam Earle; Julian Togelius; L. B. Soros
IEEE Conference on Games, 2021. [Paper] -
Open-ended search for environments and adapted agents using map-elites
Emma Stensby Norstein, Kai Olav Ellefsen, Kyrre Glette
EvoStar, 2022. [Paper] [Code] -
Minimal Criterion Artist Collective
Kai Arulkumaran; Thu Nguyen-Phuoc
GECCO, 2022. [Paper] [Code] -
Evolving Curricula with Regret-Based Environment Design
Jack Parker-Holder*, Minqi Jiang*, Michael Dennis, Mikayel Samvelyan, Jakob Foerster, Edward Grefenstette, Tim Rocktäschel
ICML, 2022. [Paper] [Code] [Demo] -
Evolution through Large Models
Joel Lehman, Jonathan Gordon, Shawn Jain, Kamal Ndousse, Cathy Yeh, Kenneth Stanley
arXiv, 2022. [Paper] [Code] -
RAPid-Learn: A Framework for Learning to Recover for Handling Novelties in Open-World Environments
Shivam Goel, Yash Shukla, Vasanth Sarathy, Matthias Scheutz, Jivko Sinapov
arXiv, 2022. [Paper] [Code] -
Transfer Dynamics in Emergent Evolutionary Curricula
Aaron Dharna, Amy K. Hoover, Julian Togelius, Lisa Soros
IEEE Transactions on Games, 2022. [Paper] [Code] -
Watts: Infrastructure for Open-Ended Learning
Aaron Dharna, Charlie Summers, Rohin Dasari, Julian Togelius, Amy K. Hoover
ALOE Workshop 2022 [Paper] [Code] -
MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge
Linxi Fan, Guanzhi Wang, Yunfan Jiang, Ajay Mandlekar, Yuncong Yang, Haoyi Zhu, Andrew Tang, De-An Huang, Yuke Zhu, Anima Anandkumar
NeurIPS, 2022. [Paper] [Code] [Website] -
Grounding Aleatoric Uncertainty in Unsupervised Environment Design
Minqi Jiang, Michael Dennis, Jack Parker-Holder, Andrei Lupu, Heinrich Küttler, Edward Grefenstette, Tim Rocktäschel, Jakob Foerster
NeurIPS 2022. [Paper] -
Language and Culture Internalisation for Human-Like Autotelic AI
Cédric Colas, Tristan Karch, Clément Moulin-Frier, Pierre-Yves Oudeyer
Nature Machine Intelligence, 2022. [Paper] [Website] -
Flow-Lenia: Towards open-ended evolution in cellular automata through mass conservation and parameter localization
Erwan Plantec, Gautier Hamon, Mayalen Etcheverry, Pierre-Yves Oudeyer, Clément Moulin-Frier, Bert Wang-Chak Chan
ALife 2023. [Paper] -
MAESTRO: Open-Ended Environment Design for Multi-Agent Reinforcement Learning
Mikayel Samvelyan, Akbir Khan, Michael Dennis, Minqi Jiang, Jack Parker-Holder, Jakob Foerster, Roberta Raileanu, Tim Rocktäschel
ICLR, 2023. [Paper] [Website] -
Powderworld: A Platform for Understanding Generalization via Rich Task Distributions
Kevin Frans, Philip Isola
ICLR, 2023. [Paper] [Website] [Code] -
Human-Timescale Adaptation in an Open-Ended Task Space
Adaptive Agent Team, Jakob Bauer, Kate Baumli, Satinder Baveja, Feryal Behbahani, Avishkar Bhoopchand, Nathalie Bradley-Schmieg, Michael Chang, Natalie Clay, Adrian Collister, Vibhavari Dasagi, Lucy Gonzalez, Karol Gregor, Edward Hughes, Sheleem Kashem, Maria Loks-Thompson, Hannah Openshaw, Jack Parker-Holder, Shreya Pathak, Nicolas Perez-Nieves, Nemanja Rakicevic, Tim Rocktäschel, Yannick Schroecker, Jakub Sygnowski, Karl Tuyls, Sarah York, Alexander Zacherl, Lei Zhang
ICML, 2023. [Paper] [Website] -
Deep Laplacian-based Options for Temporally-Extended Exploration
Martin Klissarov, Marlos C. Machado
ICML, 2023. [Paper] [Blogpost 1] [Blogpost2] -
Discovering General Reinforcement Learning Algorithms with Adversarial Environment Design
Matthew T. Jackson, Minqi Jiang, Jack Parker-Holder, Risto Vuorio, Chris Lu, Gregory Farquhar, Shimon Whiteson, Jakob N. Foerster
NeurIPS, 2023. [Paper] [Code] -
Voyager: An Open-Ended Embodied Agent with Large Language Models
Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, Anima Anandkumar
arXiv, 2023. [Paper] [Code] [Website] -
OMNI: Open-endedness via Models of human Notions of Interestingness
Jenny Zhang, Joel Lehman, Kenneth Stanley, Jeff Clune
arXiv, 2023. [Paper] [Code] [Website] -
Augmenting Autotelic Agents with Large Language Models
Cédric Colas, Laetitia Teodorescu, Pierre-Yves Oudeyer, Xingdi Yuan, Marc-Alexandre Côté
arXiv, 2023. [Paper] -
Reward-Free Curricula for Training Robust World Models
Marc Rigter, Minqi Jiang, Ingmar Posner
arXiv, 2023. [Paper] -
Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolution
Chrisantha Fernando, Dylan Banarse, Henryk Michalewski, Simon Osindero, Tim Rocktäschel
arXiv, 2023. [Paper] -
Self-Taught Optimizer (STOP): Recursively Self-Improving Code Generation
Eric Zelikman, Eliana Lorch, Lester Mackey, Adam Tauman Kalai
arXiv, 2023. [Paper] -
Motif: Intrinsic Motivation from Artificial Intelligence Feedback
Martin Klissarov, Pierluca D'Oro, Shagun Sodhani, Roberta Raileanu, Pierre-Luc Bacon, Pascal Vincent, Amy Zhang, Mikael Henaff
arXiv, 2023. [Paper] [Code] -
Quality-Diversity through AI Feedback
Herbie Bradley, Andrew Dai, Hannah Teufel, Jenny Zhang, Koen Oostermeijer, Marco Bellagente, Jeff Clune, Kenneth Stanley, Grégory Schott, Joel Lehman
arXiv, 2023. [Paper] [Website] -
Quality Diversity through Human Feedback
Li Ding, Jenny Zhang, Jeff Clune, Lee Spector, Joel Lehman
arXiv, 2023. [Paper] -
Eureka: Human-Level Reward Design via Coding Large Language Models
Yecheng Jason Ma, William Liang, Guanzhi Wang, De-An Huang, Osbert Bastani, Dinesh Jayaraman, Yuke Zhu, Linxi Fan, Anima Anandkumar
arXiv, 2023. [Paper] [Code] [Website] -
OS-Copilot: Towards Generalist Computer Agents with Self-Improvement
Zhiyong Wu, Chengcheng Han*, Zichen Ding, Zhenmin Weng, Zhoumianze Liu, Shunyu Yao, Tao Yu, Lingpeng Kong
arXiv, 2024. [Paper] [Code] [Website]
- Open Questions in Creating Safe Open-ended AI: Tensions Between Control and Creativity
Adrien Ecoffet, Jeff Clune, Joel Lehman
arXiv, 2020. [Paper]
-
Why Greatness Cannot Be Planned: The Myth of the Objective
Kenneth O. Stanley, Joel Lehman
Springer, 2015. [Book] -
Open-endedness: The last grand challenge you’ve never heard of
Kenneth O. Stanley, Joel Lehman, Lisa Soros
O'Reilly Radar, 2017. [Paper] -
AI-GAs: AI-generating algorithms, an alternate paradigm for producing general artificial intelligence
Jeff Clune
arXiv, 2019. [Paper] -
Creative Problem Solving in Artificially Intelligent Agents: A Survey and Framework
Evana Gizzi, Lakshmi Nair, Sonia Chernova, Jivko Sinapov
arXiv, 2022. [Paper] -
Executive Function: A Contrastive Value Policy for Resampling and Relabeling Perceptions via Hindsight Summarization?
Chris Lengerich, Ben Lengerich.
arXiv, 2022. [Paper] -
General Intelligence Requires Rethinking Exploration
Minqi Jiang, Tim Rocktäschel, Edward Grefenstette
Royal Society Open Science, 2023. [Paper]