This repository has been archived by the owner on Jan 6, 2022. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 6
/
README.Rmd
80 lines (58 loc) · 3.48 KB
/
README.Rmd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
---
output: github_document
---
# mlrHyperopt
[](https://travis-ci.org/jakob-r/mlrHyperopt)
[](https://ci.appveyor.com/project/jakob-r/mlrhyperopt)
[](https://coveralls.io/github/jakob-r/mlrHyperopt?branch=master)
```{r, setup, include=FALSE}
set.seed(123)
knitr::opts_chunk$set(cache = TRUE, collapse = FALSE, warning = FALSE, error = FALSE)
library(mlrHyperopt)
configureMlr(show.learner.output = FALSE)
```
Easy Hyper Parameter Optimization with [mlr](https://github.com/mlr-org/mlr/#-machine-learning-in-r) and [mlrMBO](http://mlr-org.github.io/mlrMBO/).
* [UseR 2017 Talk](https://github.com/jakob-r/mlrHyperopt/raw/master/meta/useR2017/beamer/jakob_richter_mlrHyperopt.pdf)
* [Issues and Bugs](https://github.com/jakob-r/mlrHyperopt/issues)
* [Tutorial and Documentation](https://jakob-r.github.io/mlrHyperopt)
* [Webservice](http://mlrhyperopt.jakob-r.de/parconfigs) (Work in progress!)
* [Status](http://mlrhyperopt.jakob-r.de/parconfigs) (Experimental)
## Installation
```{r, eval=FALSE}
devtools::install_github("berndbischl/ParamHelpers") # version >= 1.11 needed.
devtools::install_github("jakob-r/mlrHyperopt", dependencies = TRUE)
```
## Purpose
_mlrHyperopt_ aims at making hyperparameter optimization of machine learning methods super simple.
It offers tuning in one line:
```{r, warning=FALSE, message=FALSE}
library(mlrHyperopt)
res = hyperopt(iris.task, learner = "classif.svm")
res
```
Mainly it uses the [learner implemented in _mlr_](https://mlr-org.github.io/mlr/articles/tutorial/integrated_learners/index.html) and uses the [tuning methods also available in _mlr_](http://mlr-org.github.io/mlr-tutorial/devel/html/tune.html).
Unfortunately _mlr_ lacks of well defined _search spaces_ for each learner to make hyperparameter tuning easy.
_mlrHyperopt_ includes default _search spaces_ for the most common machine learning methods like _random forest_, _svm_ and _boosting_.
As the developer can not be an expert on all machine learning methods available for _R_ and _mlr_, _mlrHyperopt_ also offers a web service to share, upload and download improved _search spaces_.
## Development Status
### Web Server
_ParConfigs_ are up- and downloaded via JSON and stored on the server in a database.
It's a very basic Ruby on Rails CRUD App generated via scaffolding with tiny modifications <https://github.com/jakob-r/mlrHyperoptServer>.
ToDo:
* Voting System
* Upload-/Download Count
* Improve API
* Return existing ID when a duplicate is uploaded (instead of error).
* Allow a combined search (instead of one key value pair).
### R package
Basic functionality works reliable.
Maybe I will improve the optimization heuristics in the future.
It still *needs more default search spaces* for popular learners!
#### Reproducibility
This package is still under construction and the inner workings might change without a version number update.
Thus I do not recommend the usage for reproducible research until it is on CRAN.
For reproducible research you might want to stick to the more lengthly but more precise [mlr tuning workflow](https://mlr-org.github.io/mlr/articles/tutorial/tune.html).
You can still use the Parameter Sets recommended in _mlrHyperopt_.
Just make sure to write them in your source code.
### Collaboration
Is encouraged! 👍