VizSuite breaks with length-1 episodes, does not support arbitrary action spaces/distributions

[jordis-ai2]

Problem

Some visualization types assume episode lengths > 1. VizSuite further assumes action spaces are Discrete(n).

Steps to reproduce

Modify projects/tutorials/running_inference_tutorial.py with:

    def machine_params(self, mode="train", **kwargs):
        res = super().machine_params(mode, **kwargs)
        if mode in ["test", "valid"]:
            res.set_visualizer(self.get_viz(mode))

        return res

    @classmethod
    def training_pipeline(cls, **kwargs):
        pipeline = super().training_pipeline(**kwargs)
        pipeline.save_interval = 1
        return pipeline

Then run training with
python main.py projects/tutorials/running_inference_tutorial.py -o debug_valid_viz -l debug