examples

Examples

Here's a brief description of the examples in this folder:

Prerequisites:

• Intro to dataclasses & simple-parsing
• Basics of openai gym

Basic examples:

• pl_example.py: Recommended entry-point for ML Practicioners. Shows an example method and model using PyTorch Lightning. This is the best way to get started if you don't mind some level of abstraction in your code (a good thing in general!)

• quick_demo.ipynb: Recommended entry-point for new users. Simple demo showing how to create a Method from scratch that targets a Supervised CL Setting, as well as how to improve this simple Method using a simple regularization loss.

  • quick_demo.py: First part of the above notebook: shows how to create a Method from scratch that targets a Supervised CL Setting.
  • quick_demo_ewc.py: Second part of the above notebook: shows how to improve upon an existing Method by adding a CL regularization loss.

• baseline_demo.py: Shows how the BaseMethod can be applied to get results in both RL and SL Settings.

CLVision Workshop Submission Examples:

Examples in this folder are aimed at solving the supervised learning track of the competition.

Each example builds on top of the previous, in a manner that improves the overall performance you can expect on any given CL setting.

As such, it is recommended that you take a look at the examples in the following order:

0. DummyMethod Non-parametric method that simply returns a random prediction for each observation.

1. Simple Classifier: Standard neural net classifier without any CL-related mechanism. Works in the SL track, but has very poor performance.

2. Multi-Head / Task Inference Classifier: Performs multi-head prediction, and a simple form of task inference. Gets better results that the example.

3. CL Regularized Classifier: Adds a simple CL regularization loss to the multihead classifier above.

Advanced examples:

• RL_and_SL_demo.py:

  Example that shows how the BaseMethod can easily be extended by adding AuxiliaryTasks to it, allows you to get results in both RL and SL.

• continual_rl_demo.py:

  Demonstrates how to create Reinforcement Learning (RL) Settings, as well as how methods from stable-baselines3 can be applied to these settings.

• Extending Stable-Baselines3 (RL Settings only):

  (Not recommended for new users!) Very specific example which shows how, if you really wanted to, you could extend one or more of the Methods from SB3 with some kind of regularization loss hooking into the internal optimization loop of SB3.