You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
resample$score(predict_sets = 'test') gives same output as resample$score(predict_sets = 'train'). Also, is there any functionality where results from both train and test can be accessed?
#951
Open
mermast opened this issue
Jul 24, 2023
· 1 comment
(1) You have to set the predict sets in the learners (Line 8). We should throw an error if the measure requests an non-existent predict set. (2) The predict_sets argument seems not to work in $score(). You have to define the measures like in the example above (Line 13-15).
Description
In both the situations, i think it just outputs test results ...
Reproducible example
tasks = tsks(c("german_credit", "sonar"))
learners = lrns(c("classif.rpart", "classif.ranger","classif.featureless"), predict_type = "prob")
rsmp_cv5 = rsmp("cv", folds = 5)
set.seed(1234)
design = benchmark_grid(tasks, learners, rsmp_cv5)
bmr = benchmark(design)
bmr$score(predict_sets = 'train')
bmr$score(predict_sets = 'test')
The text was updated successfully, but these errors were encountered: