Improving Shogun Observable Framework
Being able to monitor model internals of Machine Learning algorithms at runtime is a crucial tool for both scientific investigations of data, and for debugging or development of new algorithms. Many Machine Learning frameworks are written in a high level, often interpreted languages (such as Python), which allow this type of monitoring in a relatively simple manner. Once algorithms are implemented in a lower (more efficient) level, however, observing parameter changes is much more challenging. For example, Google developed the Tensorboard framework as a tool to monitor, visualise and analyse internals of tensorflow programs (which are, while often instantiated in Python, implemented in C++). In previous Google Summer of Code projects, techniques from reactive programming have been used to build the foundations of an Observable framework which allows for monitoring state changes at runtime- in Shogun. While functional, the new features have not been deployed to the code base in a systematic manner. The purpose of this project is to deploy, improve and extend Shogun’s Observable architecture, and to provide glueing code for frameworks such as Tensorboard. The project will also provide crucial documentation and real-world use-cases of how those features improve scientific workflows.
Extended Proposal: https://docs.google.com/document/d/17LioVPBuyn4G2uDy0PbmpZAjJxt_wg105SblKcPyLtY/edit?usp=sharing
Trello Board: https://trello.com/b/vSRsje9A/observable-shoguns-models
Deliverables: a series of observable algorithms (at least 50% of the previously given list).
- Start by porting each of the chosen classes to the observable framework;
- Write unit tests to ensure that the observables are working;
Deliverables: a new set of ParameterObservers with an associate meta example.
- Write new ParameterObservers and example to showcase the new functionalities;
- Write unit tests to ensure their correctness;
- Write meta examples to show how they are used;
Deliverables: two blog posts/notebooks which showcase a more complete usage of the new ParameterObservers.
- Apply the newly parameter observers to the previously decided projects in the Applications section;
- Write a Jupyter Notebook (one for each application) so to showcase the newly added functionalities to the users.