Skip to content

christianhidber/easyagents

Repository files navigation

Reinforcement Learning for Practitioners (v1.4.1, 20Q1)

Travis_Status Coverage Status License Downloads Docs

Status: under active development, breaking changes may occur. Release notes.

EasyAgents logo

EasyAgents is a high level reinforcement learning api focusing on ease of use and simplicity. Written in Python and running on top of established reinforcement learning libraries like tf-Agents, tensorforce or keras-rl. Environments are implemented in OpenAI gym. For an example of an industrial application of reinforcement learning see here.

In collaboration with Oliver Zeigermann.

Features


  • provides the same, simple api across all libraries. Thus you can easily switch between different implementations and you don't have to learn for each of them a new api.
  • to create and run any algorithm you only need 2 lines of code, all the parameters are named consistently across all algorithms.
  • supports a broad set of different algorithms
  • runs inside jupyter notebooks as well as stand-alone, easy to install requiring only a single 'pip install easyagents'.
  • easy to understand, ready-made plots and logs to investigate the algorithms and environments behaviour

Examples


from easyagents.agents import PpoAgent
from easyagents.callbacks import plot

ppoAgent = PpoAgent('CartPole-v0')
ppoAgent.train([plot.State(), plot.Loss(), plot.Rewards()])

Scenario_Simple

More Detailed

from easyagents.agents import PpoAgent
from easyagents.callbacks import plot

ppoAgent = PpoAgent( 'Orso-v1',fc_layers=(500,500,500))
ppoAgent.train([plot.State(), plot.Loss(), plot.Rewards(), plot.Actions(), 
                plot.StepRewards(), plot.Steps(), plot.ToMovie()], 
                learning_rate = 0.0001, num_iterations = 500, max_steps_per_episode=50 )

Scenario_Detailed

Tutorials


Available Algorithms and Backends


algorithm tf-Agents tensorforce keras-rl (suspended) easyagents class name
CEM not available not available yes CemAgent
Dqn yes yes yes DqnAgent
Double Dqn open not available yes DoubleDqnAgent
Dueling Dqn not available yes yes DuelingDqnAgent
Ppo yes yes not available PpoAgent
Random yes yes not available RandomAgent
REINFORCE yes yes not available ReinforceAgent
SAC preview not available not available SacAgent

[191001]

  • if you are interested in other algorithms, backends or hyperparameters let us know by creating an issue. We'll try our best to support you.
  • for a documentation of the agents api see here.
  • starting with easyagents 1.3 (191102) the backend for keras-rl is suspended until support for tensorflow 2.0 is available.

Industrial Application


Geberit - a sanitary technology company with > 12'000 employees - produces in particular pipes and other parts to get rain-water of flat roofs - so called syphonic roof drainage systems. They warrant that large buildings like stadiums, airports or shopping malls do not collapse during heavy rainfalls. However it is surprisingly difficult to find the right dimensions for the pipes. It is actually so difficult, that as of today no feasable, deterministic algorithm is known. Thus traditional heuristics and classic machine learning were used to support the users in finding a suitable solution.

Using reinforcement learning the failrate of the previous solution was reduced by 70%, resulting in an end-to-end success-rate of > 98%.

For more details take a look at this talk.

Installation


Install from pypi using pip:

pip install easyagents

More


Documentation

for release notes & class diagram, for agents & api.

Guiding Principles

  • easily train, evaluate & debug policies for (you own) gym environment over "designing new algorithms"
  • simple & consistent over "flexible & powerful"
  • inspired by keras:
    • same api across all algorithms
    • support different implementations of the same algorithm
    • extensible (pluggable backends, plots & training schemes)

EasyAgents may not be ideal if

  • you would like to leverage implementation specific advantages of an algorithm
  • you want to do distributed or in parallel reinforcement learning

Note

  • If you have any difficulties in installing or using easyagents please let us know by creating an issue. We'll try our best to help you.
  • Any ideas, help, suggestions, comments etc in python / open source development / reinforcement learning / whatever are more than welcome.